Investment Securities (Details 2) - USD ($) $ in Thousands	Sep. 30, 2016	Dec. 31, 2015
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	$ 155,773	$ 1,377,664
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(1,527)	(13,632)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	280,333	41,832
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(1,818)	(1,473)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	436,106	1,419,496
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(3,345)	(15,105)
Government agency and government-sponsored enterprise pass through securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	33,817	352,042
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(100)	(3,480)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	130,786	9,342
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(594)	(266)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	164,603	361,384
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(694)	(3,746)
Government agency and government-sponsored enterprise collateralized mortgage obligations
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	18,893	117,786
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(95)	(1,026)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	26,476	0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(86)	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	45,369	117,786
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(181)	(1,026)
Private label collateralized mortgage obligations
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	14,552	93,533
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(98)	(1,000)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	29,058	1,638
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(571)	(106)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	43,610	95,171
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(669)	(1,106)
Municipal securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	5,338	126,892
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(29)	(1,061)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	531
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	0	(11)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	5,338	127,423
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(29)	(1,072)
Government agency and government-sponsored enterprise commercial mortgage-backed securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	77,706	236,098
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(1,189)	(2,156)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	0	14,230
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	0	(641)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	77,706	250,328
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(1,189)	(2,797)
US Treasury securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value		69,380
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss		(816)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value		0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss		0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value		69,380
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss		(816)
Corporate debt securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value		29,379
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss		(950)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value		0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss		0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value		29,379
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss		(950)
Collateralized loan obligations
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	2,506	100,993
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(5)	(1,131)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	39,197	0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(232)	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	41,703	100,993
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(237)	(1,131)
SBA securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	2,956	179,942
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(9)	(830)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	39,842	0
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(239)	0
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	42,798	179,942
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	(248)	(830)
Asset-backed and other securities
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	5	71,619
Available-for-sale Securities, Continuous Unrealized Loss Position, Less than 12 Months, Aggregate Loss	(2)	(1,182)
Available-for-sale Securities, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	14,974	16,091
Available-for-sale Securities, Continuous Unrealized Loss Position, 12 Months or Longer, Aggregate Loss	(96)	(449)
Available-for-sale Securities, Continuous Unrealized Loss Position, Fair Value	14,979	87,710
Available-for-sale Securities, Continuous Unrealized Loss Position, Aggregate Loss	$ (98)	$ (1,631)