Segment Reporting - Schedule of Segment Data (Detail) - USD ($) $ in Thousands		3 Months Ended
		Mar. 31, 2020		Mar. 31, 2019		Dec. 31, 2019
Segment Reporting Disclosure [Line Items]
Total interest income		$ 35,542		$ 30,043
Total interest expense	[1]	9,000		7,722
Net interest income		26,542		22,321
Provision for loan losses		16,541		13,343
Net interest income (loss) after loss provision		10,001		8,978
Sponsorship and race winnings		2,573		3,179
Race team related expenses		(2,130)		(1,998)
Other income (expense)		(26,694)		(9,020)
Income (loss) before income taxes		(16,250)		1,139
Income tax benefit (provision)		3,249		256
Net income (loss)		(13,001)		1,395
Balance Sheet Data
Total loans, net		1,129,722		987,338		$ 1,114,762
Total assets		1,534,395		1,428,728		1,541,667
Total funds borrowed		$ 1,176,645	[2]	$ 1,098,716		1,169,593
Selected Financial Ratios
Return on average assets		(3.57%)		0.36%
Return on average equity		(16.56%)		1.72%
Interest yield		11.82%		11.52%
Net interest margin		8.80%		8.56%
Reserve coverage		4.57%		3.60%
Delinquency status	[3]	0.60%		0.81%
Charge-off ratio		3.08%		5.33%
RPAC [Member]
Segment Reporting Disclosure [Line Items]
Total interest expense		$ 40		$ 36
Net interest income		(40)		(36)
Net interest income (loss) after loss provision		(40)		(36)
Sponsorship and race winnings		2,573		3,179
Race team related expenses		(2,130)		(1,998)
Other income (expense)		(1,845)		(1,797)
Income (loss) before income taxes		(1,442)		(652)
Income tax benefit (provision)		359		157
Net income (loss)		(1,083)		(495)
Balance Sheet Data
Total assets		30,171		30,952		31,538
Total funds borrowed		$ 7,830		$ 7,681		7,794
Selected Financial Ratios
Return on average assets		(14.12%)		(6.60%)
Return on average equity				(65.48%)
Operating Segments [Member] | Consumer Lending [Member] | Recreation [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ 26,334		$ 22,479
Total interest expense		3,566		2,774
Net interest income		22,768		19,705
Provision for loan losses		10,601		7,005
Net interest income (loss) after loss provision		12,167		12,700
Other income (expense)		(7,372)		(5,382)
Income (loss) before income taxes		4,795		7,318
Income tax benefit (provision)		(1,226)		(1,895)
Net income (loss)		3,569		5,423
Balance Sheet Data
Total loans, net		712,881		601,067		695,257
Total assets		725,337		611,702		707,377
Total funds borrowed		$ 577,715		$ 487,165		563,805
Selected Financial Ratios
Return on average assets		2.00%		3.66%
Return on average equity		10.02%		13.83%
Interest yield		15.08%		15.50%
Net interest margin		13.04%		13.58%
Reserve coverage		3.03%		1.46%
Delinquency status	[3]	0.73%		0.56%
Charge-off ratio		3.65%		3.40%
Operating Segments [Member] | Consumer Lending [Member] | Home Improvement [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ 5,887		$ 4,325
Total interest expense		1,287		906
Net interest income		4,600		3,419
Provision for loan losses		1,536		549
Net interest income (loss) after loss provision		3,064		2,870
Other income (expense)		(2,340)		(1,637)
Income (loss) before income taxes		724		1,233
Income tax benefit (provision)		(185)		(319)
Net income (loss)		539		914
Balance Sheet Data
Total loans, net		252,392		191,089		244,716
Total assets		261,743		199,999		252,704
Total funds borrowed		$ 208,519		$ 159,251		201,605
Selected Financial Ratios
Return on average assets		0.84%		2.38%
Return on average equity		4.20%		9.53%
Interest yield		9.53%		9.42%
Net interest margin		7.43%		7.45%
Reserve coverage		1.37%		1.13%
Delinquency status	[3]	0.08%		0.08%
Charge-off ratio		1.03%		0.35%
Operating Segments [Member] | Commercial Lending [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ 1,758		$ 1,876
Total interest expense		657		701
Net interest income		1,101		1,175
Net interest income (loss) after loss provision		1,101		1,175
Other income (expense)		(895)		(315)
Income (loss) before income taxes		206		860
Income tax benefit (provision)		(51)		(206)
Net income (loss)		155		654
Balance Sheet Data
Total loans, net		64,911		51,180		66,405
Total assets		83,864		91,329		84,924
Total funds borrowed		$ 68,469		$ 72,976		68,666
Selected Financial Ratios
Return on average assets		0.74%		2.92%
Return on average equity		3.69%		14.61%
Interest yield		10.40%		12.93%
Net interest margin		6.51%		8.10%
Reserve coverage		0.00%	[4]	0.82%
Delinquency status	[3],[4]	0.16%		1.29%
Charge-off ratio		0.00%	[5]	0.00%	[4]
Operating Segments [Member] | Medallion Lending [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ 1,002		$ 841
Total interest expense		1,849		1,909
Net interest income		(847)		(1,068)
Provision for loan losses		4,404		5,334
Net interest income (loss) after loss provision		(5,251)		(6,402)
Other income (expense)		(8,573)		1,214
Income (loss) before income taxes		(13,824)		(5,188)
Income tax benefit (provision)		3,445		1,251
Net income (loss)		(10,379)		(3,937)
Balance Sheet Data
Total loans, net		96,192		140,426		105,022
Total assets		201,959		254,714		217,483
Total funds borrowed		$ 160,812		$ 202,255		176,825
Selected Financial Ratios
Return on average assets		(19.90%)		(6.05%)
Return on average equity		(98.50%)		(30.23%)
Interest yield		3.93%		2.33%
Net interest margin		(3.32%)		(2.96%)
Reserve coverage		22.71%		15.26%
Delinquency status	[3]	1.21%		2.47%
Charge-off ratio		6.11%		21.59%
Intersegment Eliminations [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ 561		$ 522
Total interest expense		1,601		1,396
Net interest income		(1,040)		(874)
Provision for loan losses				455
Net interest income (loss) after loss provision		(1,040)		(1,329)
Other income (expense)		(5,669)		(1,103)
Income (loss) before income taxes		(6,709)		(2,432)
Income tax benefit (provision)		907		1,268
Net income (loss)		(5,802)		(1,164)
Balance Sheet Data
Total loans, net		3,346		3,576		3,362
Total assets		231,321		240,032		247,641
Total funds borrowed		$ 153,300		$ 169,388		$ 150,898
Selected Financial Ratios
Return on average assets		(9.74%)		(2.10%)
Return on average equity		(29.89%)		(7.49%)

[1] Average borrowings outstanding were $1,164,483 and $1,067,075, and the related average borrowing costs were 3.11% and 2.93%, for the three months ended March 31, 2020 and 2019.

[2] Excludes deferred financing costs of $4,674 and $5,105 as of March 31, 2020 and December 31, 2019.

[3] Loans 90 days or more past due.

[4] Ratio is based on total commercial lending balances, and relates solely to the legacy commercial loan business.

[5] Ratio is based on total commercial lending balances, and relates to the total loan business.