Loans and Allowance for Credit Losses - Additional Information (Details)	3 Months Ended		9 Months Ended
	Sep. 30, 2021 USD ($) investment	Sep. 30, 2020 USD ($) Loan	Sep. 30, 2021 USD ($) loan	Sep. 30, 2020 USD ($) loan Loan	Dec. 31, 2020 USD ($)	Sep. 23, 2020 USD ($)	Jul. 08, 2020 USD ($)	Dec. 31, 2019 USD ($)
Accounts, Notes, Loans and Financing Receivable [Line Items]
Premiums and discounts on acquired loans	$ 12,119,000		$ 12,119,000		$ 18,511,000
Net deferred origination and factoring fees	7,824,000		7,824,000		4,887,000
Accrued interest on loans	15,235,000		15,235,000		18,198,000
Allowance for credit losses	41,017,000		41,017,000		95,739,000
Loans	4,782,730,000		4,782,730,000		4,996,776,000
Pledged loans	1,652,900,000		1,652,900,000		2,255,441,000
Loans held for investment transferred to loans held for sale	12,373,000	$ 56,934,000	76,976,000	$ 172,565,000
Proceeds from sale of loans	17,446,000	58,313,000	63,028,000	145,513,000
Net (gains) losses on transfer of loans to loans held for sale	(210,000)	515,000	(1,676,000)	466,000
Increase (decrease) in allowances for credit losses attributable to non-PCD net charge-offs	3,718,000		4,067,000
Nonaccrual loans	14,534,000		14,534,000		34,073,000
Factored receivables greater than 90 days past due	37,020,000		37,020,000		72,774,000
Interest earned on nonaccrual loans	0	$ 0	0	$ 0
Recorded investments in troubled debt restructurings	$ 5,065,000		$ 5,065,000		13,324,000
Financing receivable, modifications, number of contracts	0			23
Number of defaults on modified loans | loan			3	2
Recorded investments in troubled debt restructurings			$ 1,681,000	$ 18,000
Troubled Debt Restructuring
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance for credit losses	$ 1,292,000		1,292,000		2,469,000
Nonaccrual loans	5,048,000		5,048,000		13,321,000
Over-Formula Advances | Transport Financial Solutions
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans						$ 62,200,000	$ 62,200,000
Changes in Allowance for Credit Losses
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance for credit losses change due to loss drivers	177,000		10,319,000
Factored receivables
Accounts, Notes, Loans and Financing Receivable [Line Items]
Customer reserves	189,565,000		189,565,000		145,892,000
Allowance for credit losses					56,463,000			$ 7,657,000
Loans	1,607,028,000		1,607,028,000		1,120,770,000
Charge-offs			45,683,000	3,027,000
Credit loss expense			8,547,000	4,475,000
Nonaccrual loans	0		0		0
Factored receivables greater than 90 days past due	36,936,000		36,936,000		72,717,000
Factored receivables | Over-Formula Advances
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance for credit losses	10,077,000		10,077,000		48,485,000
Loans					62,100,000
Charge-offs			41,265,000
Credit loss expense			2,844,000
Factored receivables | Over-Formula Advances | Transport Financial Solutions
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans					62,200,000
Factored receivables | Over-Formula Advances | Past Due 90 Days or More
Accounts, Notes, Loans and Financing Receivable [Line Items]
Nonaccrual loans	10,077,000		10,077,000		62,100,000
Factored receivables | Misdirected Payments Receivable | United States Postal Service
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	19,361,000		19,361,000		19,600,000
Factored receivables greater than 90 days past due					6,000,000
1-4 family residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance for credit losses					1,225,000			308,000
Loans	127,073,000		127,073,000		157,147,000
Charge-offs			26,000	27,000
Credit loss expense			(804,000)	1,138,000
Nonaccrual loans	1,310,000		1,310,000		1,848,000
Factored receivables greater than 90 days past due	0		0		0
1-4 family residential | Real Eatate Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Residential real estate loans in process of foreclosure	375,000		375,000		251,000
Commercial
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance for credit losses					22,040,000			$ 12,566,000
Loans	1,398,497,000		1,398,497,000		1,562,957,000
Charge-offs			426,000	1,173,000
Credit loss expense			(7,936,000)	$ 11,004,000
Nonaccrual loans	7,828,000		7,828,000		17,202,000
Factored receivables greater than 90 days past due	$ 84,000		$ 84,000		$ 35,000
Financing receivable, modifications, number of contracts | Loan		14		19