Fair Value Measurements - Activity for Level 3 Assets Measured at Fair Value on a Recurring Basis (Details) - USD ($)	3 Months Ended
	Mar. 31, 2021	Dec. 31, 2020	Mar. 31, 2020
Level 3 CMO [Roll Forward]
Unrealized net holding gains (losses) on securities available-for-sale arising during the period	$ (91,523,000)	$ 22,988,000	$ 17,183,000
Level 3
Level 3 CMO [Roll Forward]
Equity warrants, Balance, beginning of period	4,520,000
Total included in earnings, Equity Warrants	6,123,000
Total included in other comprehensive income	0
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants	57,000
Sales, Equity Warrants	(6,044,000)
Net settlements, Equity Warrants	0
Equity warrants, Balance, end of period		4,520,000
Fair Value, Equity, Transfers to Level 1, Amount	(67,000)
Fair Value, Measurements, Recurring
Level 3 CMO [Roll Forward]
Equity warrants, Balance, beginning of period	4,520,000
Equity warrants, Balance, end of period	4,589,000	4,520,000
Fair Value, Measurements, Recurring | Level 3
Level 3 CMO [Roll Forward]
Equity warrants, Balance, beginning of period	4,520,000
Equity warrants, Balance, end of period	4,589,000	4,520,000
Private Label CMOs | Fair Value, Measurements, Recurring | Level 3
Level 3 CMO [Roll Forward]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	4,647,000
Total included in earnings	73,000
Total included in other comprehensive income	(87,000)
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants	0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales	0
Net settlements	(219,000)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	4,414,000	4,647,000
Unrealized net holding gains (losses) on securities available-for-sale arising during the period	1,007,000
Fair Value, Equity, Transfers to Level 1, Amount	0
Asset-backed | Fair Value, Measurements, Recurring | Level 3
Level 3 CMO [Roll Forward]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	25,725,000
Total included in earnings	(22,000)
Total included in other comprehensive income	16,000
Stock and Warrants Issued During Period, Value, Preferred Stock and Warrants	0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Sales	0
Net settlements	(1,837,000)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	23,882,000	$ 25,725,000
Unrealized net holding gains (losses) on securities available-for-sale arising during the period	171,000
Fair Value, Equity, Transfers to Level 1, Amount	0
Warrant
Level 3 CMO [Roll Forward]
Fair value transfers of assets measured on a recurring basis	$ 67,000