Segment Reporting - Schedule of Segment Data (Detail) - USD ($) $ in Thousands		3 Months Ended						6 Months Ended
		Jun. 30, 2020		Mar. 31, 2020	Jun. 30, 2019		Mar. 31, 2019	Jun. 30, 2020		Jun. 30, 2019		Dec. 31, 2019
Segment Reporting Disclosure [Line Items]
Total interest income		$ 35,588			$ 32,015			$ 71,130		$ 62,058
Total interest expense	[1]	8,835			8,821			17,835		16,543
Net interest income		26,753			23,194			53,295		45,515
Provision for loan losses		16,941			15,171			33,482		28,514
Net interest income (loss) after loss provision		9,812			8,023			19,813		17,001
Sponsorship and race winnings		3,626			4,889			6,199		8,068
Race team related expenses		(1,818)			(2,550)			(3,948)		(4,548)
Other income (expense), net		(14,738)			(18,840)			(41,432)		(27,860)
Loss before income taxes		(3,118)			(8,478)			(19,368)		(7,339)
Income tax benefit (provision)		853			1,835			4,102		2,091
Net income (loss)		(2,265)		$ (13,001)	(6,643)		$ 1,395	(15,266)		(5,248)
Balance Sheet Data
Total loans, net		1,193,617			1,047,805			1,193,617		1,047,805		$ 1,114,762
Total assets		1,651,743			1,481,953			1,651,743		1,481,953		1,541,667
Total funds borrowed		$ 1,295,794			$ 1,155,336			$ 1,295,794		$ 1,155,336		1,169,593
Selected Financial Ratios
Return on average assets		(0.98%)			(2.06%)			(2.23%)		(0.89%)
Return on average equity		(4.97%)			(10.34%)			(10.82%)		(4.36%)
Interest yield		10.95%			11.67%			11.31%		11.58%
Net interest margin		8.23%			8.46%			8.48%		8.49%
Reserve coverage		5.31%			3.74%			5.31%		3.74%
Delinquency status	[2]	1.26%			0.78%			1.26%		0.78%
Charge-off ratio		1.39%			4.46%			2.21%		4.88%
RPAC [Member]
Segment Reporting Disclosure [Line Items]
Total interest expense		$ 40			$ 36			$ 80		$ 72
Net interest income		(40)			(36)			(80)		(72)
Net interest income (loss) after loss provision		(40)			(36)			(80)		(72)
Sponsorship and race winnings		3,626			4,889			6,199		8,068
Race team related expenses		(1,818)			(2,550)			(3,948)		(4,548)
Other income (expense), net		(1,378)			(1,717)			(3,223)		(3,514)
Loss before income taxes		390			586			(1,052)		(66)
Income tax benefit (provision)		(97)			(141)			262		16
Net income (loss)		293			445			(790)		(50)
Balance Sheet Data
Total assets		30,542			33,526			30,542		33,526		31,538
Total funds borrowed		$ 8,615			$ 7,713			$ 8,615		$ 7,713		7,794
Selected Financial Ratios
Return on average assets		3.88%			5.54%			(5.17%)		(0.32%)
Return on average equity		(53.94%)			(47.72%)			81.74%		(3.13%)
Operating Segments [Member] | Consumer Lending [Member] | Recreation [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ 27,229			$ 24,370			$ 53,563		$ 46,849
Total interest expense		3,226			3,189			6,792		5,963
Net interest income		24,003			21,181			46,771		40,886
Provision for loan losses		8,292			6,176			18,893		13,181
Net interest income (loss) after loss provision		15,711			15,005			27,878		27,705
Other income (expense), net		(6,497)			(5,938)			(13,869)		(11,320)
Loss before income taxes		9,214			9,067			14,009		16,385
Income tax benefit (provision)		(2,356)			(2,349)			(3,582)		(4,244)
Net income (loss)		6,858			6,718			10,427		12,141
Balance Sheet Data
Total loans, net		759,764			655,868			759,764		655,868		695,257
Total assets		775,151			667,600			775,151		667,600		707,377
Total funds borrowed		$ 617,066			$ 531,708			$ 617,066		$ 531,708		563,805
Selected Financial Ratios
Return on average assets		3.68%			4.21%			2.85%		3.93%
Return on average equity		18.38%			16.16%			14.25%		16.26%
Interest yield		14.91%			15.53%			14.98%		15.49%
Net interest margin		13.15%			13.50%			13.08%		13.52%
Reserve coverage		3.43%			1.90%			3.43%		1.90%
Delinquency status	[2]	0.44%			0.56%			0.44%		0.56%
Charge-off ratio		1.95%			1.55%			2.78%		2.43%
Operating Segments [Member] | Consumer Lending [Member] | Home Improvement [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ 6,326			$ 4,678			$ 12,213		$ 9,003
Total interest expense		1,236			1,037			2,523		1,943
Net interest income		5,090			3,641			9,690		7,060
Provision for loan losses		760			813			2,296		1,362
Net interest income (loss) after loss provision		4,330			2,828			7,394		5,698
Other income (expense), net		(1,962)			(1,719)			(4,302)		(3,356)
Loss before income taxes		2,368			1,109			3,092		2,342
Income tax benefit (provision)		(606)			(288)			(791)		(607)
Net income (loss)		1,762			821			2,301		1,735
Balance Sheet Data
Total loans, net		278,000			206,636			278,000		206,636		244,716
Total assets		288,501			217,757			288,501		217,757		252,704
Total funds borrowed		$ 229,237			$ 173,226			$ 229,237		$ 173,226		201,605
Selected Financial Ratios
Return on average assets		2.58%			1.94%			1.72%		1.98%
Return on average equity		12.88%			7.88%			8.62%		8.65%
Interest yield		9.66%			9.46%			9.58%		9.44%
Net interest margin		7.77%			7.36%			7.58%		7.40%
Reserve coverage		1.44%			1.39%			1.44%		1.39%
Delinquency status	[2]	0.05%			0.08%			0.05%		0.08%
Charge-off ratio		0.30%			0.17%			0.65%		0.26%
Operating Segments [Member] | Commercial Lending [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ 1,726			$ 1,641			$ 3,484		$ 3,517
Total interest expense		617			666			1,274		1,367
Net interest income		1,109			975			2,210		2,150
Net interest income (loss) after loss provision		1,109			975			2,210		2,150
Other income (expense), net		(584)			(780)			(1,479)		(1,095)
Loss before income taxes		525			195			731		1,055
Income tax benefit (provision)		(131)			(48)			(182)		(254)
Net income (loss)		394			147			549		801
Balance Sheet Data
Total loans, net		68,140			60,395			68,140		60,395		66,405
Total assets		86,831			86,725			86,831		86,725		84,924
Total funds borrowed		$ 70,567			$ 68,654			$ 70,567		$ 68,654		68,666
Selected Financial Ratios
Return on average assets		1.86%			0.66%			1.30%		1.81%
Return on average equity		9.28%			3.31%			6.48%		9.03%
Interest yield		10.67%			11.02%			10.97%		11.85%
Net interest margin		6.86%			6.55%			6.96%		7.24%
Reserve coverage		0.00%	[3]		0.71%	[3]		0.00%	[4]	0.71%	[3]
Delinquency status	[2]	0.15%	[3]		1.13%	[3]		0.15%	[4]	1.13%	[3]
Charge-off ratio	[5]	0.00%			0.00%			0.00%		0.00%
Operating Segments [Member] | Medallion Lending [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ (7)			$ 666			$ 995		$ 1,507
Total interest expense		988			1,591			2,837		3,500
Net interest income		(995)			(925)			(1,842)		(1,993)
Provision for loan losses		7,889			8,182			12,293		13,516
Net interest income (loss) after loss provision		(8,884)			(9,107)			(14,135)		(15,509)
Other income (expense), net		(2,292)			(6,558)			(10,865)		(5,344)
Loss before income taxes		(11,176)			(15,665)			(25,000)		(20,853)
Income tax benefit (provision)		2,785			3,779			6,230		5,030
Net income (loss)		(8,391)			(11,886)			(18,770)		(15,823)
Balance Sheet Data
Total loans, net		84,369			121,314			84,369		121,314		105,022
Total assets		190,657			235,948			190,657		235,948		217,483
Total funds borrowed		$ 151,614			$ 187,575			$ 151,614		$ 187,575		176,825
Selected Financial Ratios
Return on average assets		(17.19%)			(19.43%)			(18.56%)		(12.53%)
Return on average equity		(85.96%)			(97.16%)			(92.14%)		(62.63%)
Interest yield		(0.03%)			1.99%			2.04%		2.17%
Net interest margin		(4.08%)			(2.77%)			(3.78%)		(2.87%)
Reserve coverage		29.84%			16.88%			29.84%		16.88%
Delinquency status	[2]	10.29%			2.66%			10.29%		2.66%
Charge-off ratio		1.12%			26.47%			3.73%		23.94%
Intersegment Eliminations [Member]
Segment Reporting Disclosure [Line Items]
Total interest income		$ 314			$ 660			$ 875		$ 1,182
Total interest expense		2,728			2,302			4,329		3,698
Net interest income		(2,414)			(1,642)			(3,454)		(2,516)
Provision for loan losses										455
Net interest income (loss) after loss provision		(2,414)			(1,642)			(3,454)		(2,971)
Other income (expense), net		(2,025)			(2,128)			(7,694)		(3,231)
Loss before income taxes		(4,439)			(3,770)			(11,148)		(6,202)
Income tax benefit (provision)		1,258			882			2,165		2,150
Net income (loss)		(3,181)			(2,888)			(8,983)		(4,052)
Balance Sheet Data
Total loans, net		3,344			3,592			3,344		3,592		3,362
Total assets		280,061			240,397			280,061		240,397		247,641
Total funds borrowed		$ 218,695			$ 186,460			$ 218,695		$ 186,460		$ 150,898
Selected Financial Ratios
Return on average assets		(8.96%)			(4.82%)			(7.14%)		(3.16%)
Return on average equity		(38.05%)			(20.68%)			(26.03%)		(12.54%)

[1] Average borrowings outstanding were $1,290,318 and $1,227,413, and the related average borrowing costs were 2.75% and 2.92%, for the three and six months ended June 30, 2020, and were $1,127,509 and $1,108,512 and 3.14% and 3.01% for the three and six months ended June 30, 2019.  

[2] Loans 90 days or more past due.

[3] Ratio is based on total commercial lending balances, and relates solely to the legacy commercial loan business.

[4] Ratio is based on total commercial lending balances, and relates solely to the legacy commercial loan business

[5] Ratio is based on total commercial lending balances, and relates to the total loan business.