Fair Value Measurements (Details 1) - USD ($) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2017	Dec. 31, 2016
Level 3 CMO Roll Forward [Abstract]
Equity warrants		$ 5,497
Fair Value, Measurements, Recurring
Level 3 CMO Roll Forward [Abstract]
Equity warrants	$ 5,368	5,497
Level 3
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation [Line Items]
Warrants, Acquired From Acquisition		5,497
Level 3 CMO Roll Forward [Abstract]
Total included in other comprehensive income	0
Gain on Sale of Investments	155
Proceeds from Issuance of Warrants	(518)
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants	234
Transfers to Available-For-Sale	0
Level 3 | Fair Value, Measurements, Recurring
Level 3 CMO Roll Forward [Abstract]
Equity warrants	5,368	5,497
Level 3 | Private Label CMOs | Fair Value, Measurements, Recurring
Level 3 CMO Roll Forward [Abstract]
Balance, December 31, 2016	56,902
Total included in earnings	1,063
Total included in other comprehensive income	(534)
Net settlements	(7,418)
Balance, March 31, 2017	45,281	56,902
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales	(4,732)
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants	0
Level 3 | Asset-backed and other securities | Fair Value, Measurements, Recurring
Level 3 CMO Roll Forward [Abstract]
Balance, December 31, 2016	8,373
Total included in earnings	25
Total included in other comprehensive income	(42)
Net settlements	(480)
Balance, March 31, 2017	7,876	$ 8,373
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales	0
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants	$ 0
Level 3 | Minimum | Private label collateralized mortgage obligations | Fair Value, Measurements, Recurring
Unobservable Inputs Quantitative Information [Abstract]
Voluntary annual prepayment speeds	2.40%
Annual default rates	0.00%
Loss severity rates	0.00%
Discount rates	0.00%
Level 3 | Minimum | SBA securities | Fair Value, Measurements, Recurring
Unobservable Inputs Quantitative Information [Abstract]
Voluntary annual prepayment speeds	5.00%
Annual default rates	1.00%
Loss severity rates	15.00%
Discount rates	3.50%
Level 3 | Maximum | Private label collateralized mortgage obligations | Fair Value, Measurements, Recurring
Unobservable Inputs Quantitative Information [Abstract]
Voluntary annual prepayment speeds	32.60%
Annual default rates	14.50%
Loss severity rates	92.90%
Discount rates	10.30%
Level 3 | Maximum | SBA securities | Fair Value, Measurements, Recurring
Unobservable Inputs Quantitative Information [Abstract]
Voluntary annual prepayment speeds	5.00%
Annual default rates	1.00%
Loss severity rates	15.00%
Discount rates	3.50%
Level 3 | Weighted Average | Private label collateralized mortgage obligations | Fair Value, Measurements, Recurring
Unobservable Inputs Quantitative Information [Abstract]
Voluntary annual prepayment speeds	8.80%
Annual default rates	2.30%
Loss severity rates	41.00%
Discount rates	4.40%
Level 3 | Weighted Average | SBA securities | Fair Value, Measurements, Recurring
Unobservable Inputs Quantitative Information [Abstract]
Voluntary annual prepayment speeds	5.00%
Annual default rates	1.00%
Loss severity rates	15.00%
Discount rates	3.50%
Level 3 | Weighted Average | Warrant | Fair Value, Measurements, Recurring
Weighted Average Input
Remaining life assumption (in years)	3 years 9 months 20 days
Warrant | Level 3
Weighted Average Input
Volatility	17.80%
Warrant | Level 3 | Weighted Average
Weighted Average Input
Risk-free interest rate	1.70%