Fair Value Measurements - Activity for Level 3 Assets Measured at Fair Value on a Recurring Basis (Details) - USD ($)	3 Months Ended			9 Months Ended
	Sep. 30, 2020	Jun. 30, 2020	Sep. 30, 2019	Sep. 30, 2020	Sep. 30, 2019
Level 3 CMO [Roll Forward]
Unrealized net holding gains (losses) on securities available-for-sale arising during the period	$ 19,745,000	$ 82,780,000	$ 32,759,000	$ 119,708,000	$ 167,566,000
Level 3
Level 3 CMO [Roll Forward]
Equity warrants, Balance, beginning of period				3,434,000
Total included in earnings, Equity Warrants				3,310,000
Total included in other comprehensive income				0
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants				240,000
Sales, Equity Warrants				(2,234,000)
Net settlements, Equity Warrants				0
Fair Value, Equity, Transfers to Level 1, Amount				(8,000)
Stock And Warrants Purchased During Period Value Preferred Stock And Warrants				0
Fair Value, Measurements, Recurring
Level 3 CMO [Roll Forward]
Equity warrants, Balance, beginning of period				3,434,000
Equity warrants, Balance, end of period	4,742,000			4,742,000
Fair Value, Measurements, Recurring | Level 3
Level 3 CMO [Roll Forward]
Equity warrants, Balance, beginning of period				3,434,000
Equity warrants, Balance, end of period	4,742,000			4,742,000
Private Label CMOs | Fair Value, Measurements, Recurring | Level 3
Level 3 CMO [Roll Forward]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value				6,264,000
Total included in earnings				326,000
Total included in other comprehensive income				(447,000)
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants				0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales				0
Net settlements				(1,092,000)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	5,051,000			5,051,000
Unrealized net holding gains (losses) on securities available-for-sale arising during the period				1,239,000
Fair Value, Equity, Transfers to Level 1, Amount				0
Stock And Warrants Purchased During Period Value Preferred Stock And Warrants				0
Asset-backed | Fair Value, Measurements, Recurring | Level 3
Level 3 CMO [Roll Forward]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value				16,435,000
Total included in earnings				42,000
Total included in other comprehensive income				(389,000)
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants				0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales				0
Net settlements				(5,966,000)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	$ 30,222,000			30,222,000
Unrealized net holding gains (losses) on securities available-for-sale arising during the period				(192,000)
Fair Value, Equity, Transfers to Level 1, Amount				0
Stock And Warrants Purchased During Period Value Preferred Stock And Warrants				20,100,000
Warrant
Level 3 CMO [Roll Forward]
Fair value transfers of assets measured on a recurring basis				$ 8,000