Fair Value Measurements (Details 3)	9 Months Ended
	Sep. 30, 2018 USD ($)
Level 3 CMO [Roll Forward]
Equity warrants, Balance, beginning of period	$ 5,161,000
Level 3
Level 3 CMO [Roll Forward]
Total included in earnings, Equity Warrants	5,291,000
Total included in other comprehensive income	0
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants	557,000
Sales, Equity Warrants	(5,972,000)
Net settlements, Equity Warrants	0
Fair Value, Measurements, Recurring
Level 3 CMO [Roll Forward]
Equity warrants, Balance, beginning of period	5,161,000
Equity warrants, Balance, end of period	4,962,000
Fair Value, Measurements, Recurring | Level 3
Level 3 CMO [Roll Forward]
Equity warrants, Balance, beginning of period	5,161,000
Equity warrants, Balance, end of period	4,962,000
Private Label CMOs | Fair Value, Measurements, Recurring | Level 3
Level 3 CMO [Roll Forward]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	22,874,000
Total included in earnings	595,000
Total included in other comprehensive income	(511,000)
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants	0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales	0
Net settlements	(9,835,000)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	13,123,000
Asset-backed | Fair Value, Measurements, Recurring | Level 3
Level 3 CMO [Roll Forward]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	42,109,000
Total included in earnings	(35,000)
Total included in other comprehensive income	220,000
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants	0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales	0
Net settlements	(13,195,000)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	29,099,000
Private label collateralized mortgage obligations
Level 3 CMO [Roll Forward]
Fair value transfers of assets measured on a recurring basis	0
SBA securities
Level 3 CMO [Roll Forward]
Fair value transfers of assets measured on a recurring basis	0
Warrant
Level 3 CMO [Roll Forward]
Fair value transfers of assets measured on a recurring basis	$ 75,000