Fair Value Measurements (Details 2) - Level 3 - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2018	Dec. 31, 2017	Dec. 31, 2016
Covered Private Label Collateralized Mortgage Obligations Member [Domain]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3		$ (21,200)
Covered Private Label Collateralized Mortgage Obligations Member [Domain] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value, beginning balance	$ 22,874	56,902	$ 81,241
Fair Value, Measured on Recurring Basis, Gain (Loss) Included in Earnings	1,737	2,256	1,636
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Other Comprehensive Income (Loss)	(1,146)	(742)	(1,648)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales	(4,880)	(4,732)	0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Transfers Into Level 3	0	574	0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3	0	21,165
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Settlements	(11,297)	(10,219)	(24,327)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value, ending balance	7,288	22,874	56,902
Other securities [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value, beginning balance	42,109	8,373	18,200
Fair Value, Measured on Recurring Basis, Gain (Loss) Included in Earnings	(32)	367	96
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Other Comprehensive Income (Loss)	495	(937)	94
Stock and Warrants Purchases During Period	15,158	42,910	0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Settlements	(17,785)	(8,604)	(10,017)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value, ending balance	$ 39,945	$ 42,109	$ 8,373