Fair Value Measurements (Details Textual) - USD ($)	3 Months Ended	12 Months Ended			24 Months Ended
	Sep. 30, 2018	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017	Dec. 31, 2019	Dec. 31, 2016
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		$ 3,797,187,000	$ 4,009,431,000		$ 3,797,187,000
Threshold for loans that are considered individually significant for allowance analysis purposes			250,000
Equity Method Investment, Other than Temporary Impairment		1,000,000.0			764,000
Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Warrants Not Settleable in Cash, Fair Value Disclosure			4,793,000	$ 5,161,000		$ 5,497,000
Equity Warrants Transfers to Available For Sale Securities	$ 78,000	113,000	78,000	(1,182,000)
Fair Value, Recurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		3,797,187,000			3,797,187,000
Warrants Not Settleable in Cash, Fair Value Disclosure		3,434,000	4,793,000		3,434,000
Fair Value, Recurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Warrants Not Settleable in Cash, Fair Value Disclosure		3,434,000	4,793,000		3,434,000
Covered Private Label Collateralized Mortgage Obligations Member [Domain] | Fair Value, Recurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Transfers Into Level 3		0	0	574,000
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3		0	0	$ 21,165,000
Private Label Collateralized Mortgage Obligations [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		99,483,000	101,205,000		99,483,000
Private Label Collateralized Mortgage Obligations [Member] | Fair Value, Recurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		99,483,000	101,205,000		99,483,000
Private Label Collateralized Mortgage Obligations [Member] | Fair Value, Recurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		6,264,000	7,288,000		6,264,000
Asset-backed Securities [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		214,783,000	81,385,000		214,783,000
Asset-backed Securities [Member] | Fair Value, Recurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		214,783,000	81,385,000		214,783,000
Asset-backed Securities [Member] | Fair Value, Recurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		16,435,000	$ 39,945,000		$ 16,435,000
Warrant [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3		$ 113,000