Investment Securities (Details 2) - USD ($) $ in Thousands	Jun. 30, 2016	Dec. 31, 2015
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	$ 455,898	$ 1,377,664
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(4,013)	(13,632)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	82,494	41,832
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(1,881)	(1,473)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	538,392	1,419,496
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(5,894)	(15,105)
Government agency and government-sponsored enterprise pass through securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	142,069	352,042
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(465)	(3,480)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	4,618	9,342
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(19)	(266)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	146,687	361,384
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(484)	(3,746)
Government agency and government-sponsored enterprise collateralized mortgage obligations
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	31,379	117,786
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(154)	(1,026)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	31,379	117,786
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(154)	(1,026)
Private label collateralized mortgage obligations
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	49,862	93,533
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(534)	(1,000)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	3,058	1,638
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(217)	(106)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	52,920	95,171
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(751)	(1,106)
Municipal securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	7,465	126,892
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(1)	(1,061)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	531
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	0	(11)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	7,465	127,423
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(1)	(1,072)
Government agency and government-sponsored enterprise commercial mortgage-backed securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	43,039	236,098
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(281)	(2,156)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	14,230
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	0	(641)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	43,039	250,328
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(281)	(2,797)
US Treasury securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value		69,380
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss		(816)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value		0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss		0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value		69,380
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss		(816)
Corporate debt securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	28,792	29,379
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(1,426)	(950)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	28,792	29,379
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(1,426)	(950)
Collateralized loan obligations
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	58,145	100,993
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(685)	(1,131)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	53,825	0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(1,113)	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	111,970	100,993
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(1,798)	(1,131)
SBA securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	53,519	179,942
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(297)	(830)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	0	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	53,519	179,942
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(297)	(830)
Asset-backed and other securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	41,628	71,619
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(170)	(1,182)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	20,993	16,091
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(532)	(449)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	62,621	87,710
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	$ (702)	$ (1,631)