Loans and Allowance for Credit Losses - Individual and Collective Allowance for Credit Losses on Financing Receivables and Loan Balances (Details) - USD ($) $ in Thousands	Dec. 31, 2020	Dec. 31, 2019
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit loss	$ 95,739	$ 29,092
Commercial real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	12,616
Allowance for credit loss	1,334
Construction, land development, land
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	2,317
Allowance for credit loss	271
1-4 family residential
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	2,196
Allowance for credit loss	34
Farmland
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	2,530
Allowance for credit loss	0
Commercial
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	17,007
Allowance for credit loss	5,163
Factored receivables
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	92,437
Allowance for credit loss	51,371
Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	253
Allowance for credit loss	37
Mortgage warehouse
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Allowance for credit loss	0
Collateral Dependent Loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	129,356
Allowance for credit loss	58,210
Real Estate | Commercial real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	12,454
Real Estate | Construction, land development, land
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	2,317
Real Estate | 1-4 family residential
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	1,948
Real Estate | Farmland
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	2,189
Real Estate | Commercial
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	1,813
Real Estate | Factored receivables
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Real Estate | Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Real Estate | Mortgage warehouse
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Real Estate | Collateral Dependent Loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	20,721
Accounts Receivable | Commercial real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Accounts Receivable | Construction, land development, land
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Accounts Receivable | 1-4 family residential
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Accounts Receivable | Farmland
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Accounts Receivable | Commercial
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Accounts Receivable | Factored receivables
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	92,437
Accounts Receivable | Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Accounts Receivable | Mortgage warehouse
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Accounts Receivable | Collateral Dependent Loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	92,437
Equipment | Commercial real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Equipment | Construction, land development, land
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Equipment | 1-4 family residential
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Equipment | Farmland
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	143
Equipment | Commercial
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	5,842
Equipment | Factored receivables
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Equipment | Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Equipment | Mortgage warehouse
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Equipment | Collateral Dependent Loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	5,985
Other | Commercial real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	162
Other | Construction, land development, land
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Other | 1-4 family residential
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	248
Other | Farmland
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	198
Other | Commercial
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	9,352
Other | Factored receivables
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Other | Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	253
Other | Mortgage warehouse
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	0
Other | Collateral Dependent Loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Financing Receivable Collateral Dependent Amortized Cost	$ 10,213
Commercial real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit loss		5,353
Construction, land development, land
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit loss		1,382
1-4 family residential properties
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit loss		308
Farmland
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit loss		670
Commercial
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit loss		12,566
Factored receivables
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit loss		7,657
Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit loss		488
Mortgage warehouse
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit loss		$ 668