Securities - Available for Sale Securities With Gross Unrealized Losses and Their Fair Value (Details) $ in Thousands	Dec. 31, 2020 USD ($)	Dec. 31, 2019 USD ($)
Securities have been in a continuous unrealized loss position, number of positions	37	62
Securities have been in a continuous unrealized loss position, fair value	$ 290,820	$ 441,043
Securities have been in a continuous unrealized loss position, unrealized losses	7,766	10,101
Securities have been in a continuous unrealized loss position, less than 12 months, fair value	76,729	146,957
Securities have been in a continuous unrealized loss position, less than 12 months, unrealized losses	186	1,363
Securities have been in a continuous unrealized loss position, 12 months or more, fair value	214,091	294,086
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses	$ 7,580	$ 8,738
US Government Agencies Debt Securities [Member]
Securities have been in a continuous unrealized loss position, number of positions	1
Securities have been in a continuous unrealized loss position, fair value	$ 4,988
Securities have been in a continuous unrealized loss position, unrealized losses	1
Securities have been in a continuous unrealized loss position, less than 12 months, fair value	4,988
Securities have been in a continuous unrealized loss position, less than 12 months, unrealized losses	1
Securities have been in a continuous unrealized loss position, 12 months or more, fair value	0
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses	$ 0
Corporate Debt Securities [Member]
Securities have been in a continuous unrealized loss position, number of positions	14	16
Securities have been in a continuous unrealized loss position, fair value	$ 113,734	$ 123,050
Securities have been in a continuous unrealized loss position, unrealized losses	6,266	6,950
Securities have been in a continuous unrealized loss position, 12 months or more, fair value	113,734	123,050
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses	$ 6,266	$ 6,950
Collateralized Loan Obligations [Member]
Securities have been in a continuous unrealized loss position, number of positions	13	13
Securities have been in a continuous unrealized loss position, fair value	$ 99,199	$ 99,137
Securities have been in a continuous unrealized loss position, unrealized losses	1,363	1,212
Securities have been in a continuous unrealized loss position, less than 12 months, fair value	7,441	25,451
Securities have been in a continuous unrealized loss position, less than 12 months, unrealized losses	52	108
Securities have been in a continuous unrealized loss position, 12 months or more, fair value	91,758	73,686
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses	$ 1,311	$ 1,104
Other Debt Obligations [Member]
Securities have been in a continuous unrealized loss position, number of positions	28	29
Securities have been in a continuous unrealized loss position, fair value	$ 217,921	$ 222,187
Securities have been in a continuous unrealized loss position, unrealized losses	7,630	8,162
Securities have been in a continuous unrealized loss position, less than 12 months, fair value	12,429	25,451
Securities have been in a continuous unrealized loss position, less than 12 months, unrealized losses	53	108
Securities have been in a continuous unrealized loss position, 12 months or more, fair value	205,492	196,736
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses	$ 7,577	$ 8,054
Collateralized Mortgage Backed Securities [Member]
Securities have been in a continuous unrealized loss position, number of positions	9	33
Securities have been in a continuous unrealized loss position, fair value	$ 72,899	$ 218,856
Securities have been in a continuous unrealized loss position, unrealized losses	136	1,939
Securities have been in a continuous unrealized loss position, less than 12 months, fair value	64,300	121,506
Securities have been in a continuous unrealized loss position, less than 12 months, unrealized losses	133	1,255
Securities have been in a continuous unrealized loss position, 12 months or more, fair value	8,599	97,350
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses	$ 3	$ 684
REMIC and CMO [Member]
Securities have been in a continuous unrealized loss position, number of positions		23
Securities have been in a continuous unrealized loss position, fair value		$ 120,989
Securities have been in a continuous unrealized loss position, unrealized losses		1,440
Securities have been in a continuous unrealized loss position, less than 12 months, fair value		102,384
Securities have been in a continuous unrealized loss position, less than 12 months, unrealized losses		1,117
Securities have been in a continuous unrealized loss position, 12 months or more, fair value		18,605
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses		$ 323
GNMA [Member]
Securities have been in a continuous unrealized loss position, number of positions	1	1
Securities have been in a continuous unrealized loss position, fair value	$ 10,341	$ 49
Securities have been in a continuous unrealized loss position, unrealized losses	22
Securities have been in a continuous unrealized loss position, less than 12 months, fair value	10,341	$ 49
Securities have been in a continuous unrealized loss position, less than 12 months, unrealized losses	22
Securities have been in a continuous unrealized loss position, 12 months or more, fair value	0
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses	$ 0
FNMA [Member]
Securities have been in a continuous unrealized loss position, number of positions	5	8
Securities have been in a continuous unrealized loss position, fair value	$ 32,463	$ 67,618
Securities have been in a continuous unrealized loss position, unrealized losses	31	426
Securities have been in a continuous unrealized loss position, less than 12 months, fair value	23,864	19,073
Securities have been in a continuous unrealized loss position, less than 12 months, unrealized losses	28	138
Securities have been in a continuous unrealized loss position, 12 months or more, fair value	8,599	48,545
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses	$ 3	$ 288
FHLMC [Member]
Securities have been in a continuous unrealized loss position, number of positions	3	1
Securities have been in a continuous unrealized loss position, fair value	$ 30,095	$ 30,200
Securities have been in a continuous unrealized loss position, unrealized losses	83	73
Securities have been in a continuous unrealized loss position, less than 12 months, fair value	30,095
Securities have been in a continuous unrealized loss position, less than 12 months, unrealized losses	83
Securities have been in a continuous unrealized loss position, 12 months or more, fair value	0	30,200
Securities have been in a continuous unrealized loss position, 12 months or more, unrealized losses	$ 0	$ 73