Fair Value Measurements (Details Textual) - USD ($)	3 Months Ended	12 Months Ended
	Sep. 30, 2017	Dec. 31, 2018	Dec. 31, 2017	Dec. 31, 2016	Dec. 31, 2015
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		$ 4,009,431,000	$ 3,774,431,000
Warrants Not Settleable in Cash, Fair Value Disclosure			5,161,000
Threshold for loans that are considered individually significant for allowance analysis purposes			250,000
Equity Method Investment, Other than Temporary Impairment		5,900,000
Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value			64,983,000
Warrants Not Settleable in Cash, Fair Value Disclosure			5,161,000	$ 5,497,000	$ 4,914,000
Equity Warrants Transfers to Available For Sale Securities	$ 1,182,000		1,182,000	(836,000)
Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		4,009,431,000	3,774,431,000
Warrants Not Settleable in Cash, Fair Value Disclosure		4,793,000	5,161,000
Fair Value, Measurements, Recurring [Member] | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		47,233,000	64,983,000
Warrants Not Settleable in Cash, Fair Value Disclosure		4,793,000	5,161,000
Covered Private Label Collateralized Mortgage Obligations Member [Domain] | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3			21,200,000
Covered Private Label Collateralized Mortgage Obligations Member [Domain] | Fair Value, Measurements, Recurring [Member] | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Transfers Into Level 3		0	574,000	$ 0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3		0	(21,165,000)
Private Label Collateralized Mortgage Obligations [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		101,205,000	125,987,000
Asset-backed Securities [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		81,385,000	88,710,000
Asset-backed Securities [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		81,385,000	88,710,000
Asset-backed Securities [Member] | Fair Value, Measurements, Recurring [Member] | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securities available-for-sale, at fair value		39,945,000	$ 42,109,000
Warrant [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3		$ (78,000)