Investment Securities (Details 2) - USD ($) $ in Thousands	Dec. 31, 2020	Dec. 31, 2019
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	$ 457,559	$ 716,081
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	90,077	1,756
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	547,636	717,837
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(2,474)	(3,545)
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	(904)	(10)
Available-for-sale Securities, Continuous Unrealized Loss Position, Accumulated Loss	(3,378)	(3,555)
Available-for-sale Securities, Accumulated Gross Unrealized Loss, before Tax	(3,378)	(3,555)
US States and Political Subdivisions Debt Securities [Member]
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	5,919	38,667
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	5,919	38,667
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(18)	(339)
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Accumulated Loss	(18)	(339)
Available-for-sale Securities, Accumulated Gross Unrealized Loss, before Tax	(18)	(339)
Government agency and government-sponsored enterprise pass through securities [Member]
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	58,408	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	186
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	58,408	186
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(37)	0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	0	(1)
Available-for-sale Securities, Continuous Unrealized Loss Position, Accumulated Loss	(37)	(1)
Available-for-sale Securities, Accumulated Gross Unrealized Loss, before Tax	(37)	(537)
Government agency and government-sponsored enterprise collateralized mortgage obligations [Member]
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	97,863	180,071
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	1,456
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	97,863	181,527
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(280)	(572)
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	0	(7)
Available-for-sale Securities, Continuous Unrealized Loss Position, Accumulated Loss	(280)	(579)
Available-for-sale Securities, Accumulated Gross Unrealized Loss, before Tax	(280)	(579)
Commercial Mortgage Backed Securities [Member]
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	90,722	214,862
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	90,722	214,862
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(897)	(537)
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Accumulated Loss	(897)	(537)
Corporate Debt Securities [Member]
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	87,596
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	87,596
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(404)
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	0
Available-for-sale Securities, Accumulated Gross Unrealized Loss, before Tax	(404)	0
Asset-backed Securities [Member]
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	17,694	165,575
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	61,031	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	78,725	165,575
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(63)	(1,670)
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	(707)	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Accumulated Loss	(770)	(1,670)
Available-for-sale Securities, Accumulated Gross Unrealized Loss, before Tax	(770)	(1,670)
Collateralized Loan Obligations [Member]
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	96,442	102,469
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	28,972	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	125,414	102,469
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(729)	(403)
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	(195)	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Accumulated Loss	(924)	(403)
Available-for-sale Securities, Accumulated Gross Unrealized Loss, before Tax	(924)	(403)
Private Label Collateralized Mortgage Obligations [Member]
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	788	9,872
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	74	114
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	862	9,986
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(19)	(11)
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	(2)	(2)
Available-for-sale Securities, Continuous Unrealized Loss Position, Accumulated Loss	(21)	(13)
Available-for-sale Securities, Accumulated Gross Unrealized Loss, before Tax	(21)	(13)
SBA asset-backed securities [Member]
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	2,127	4,565
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	2,127	4,565
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Accumulated Loss	(27)	(13)
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Accumulated Loss	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Accumulated Loss	(27)	(13)
Available-for-sale Securities, Accumulated Gross Unrealized Loss, before Tax	$ (27)	$ (13)