Segment Information - Schedule of Operating Results by Segment (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2012	Dec. 31, 2011		Dec. 31, 2010
Segment Reporting Information [Line Items]
Net interest income	$ 235,315	$ 273,234		$ 274,044
Provision for loan losses	35,419	63,272		87,080
Other income (loss)	92,403	94,910		74,880
Other expense	187,968	188,317		187,107
Income (loss) before taxes	104,331	116,555		74,737
Income before income taxes	104,331	116,555		74,737
Income taxes (benefit)	25,701	34,415		16,636
Net income	78,630	82,140		58,101
Assets	6,642,803	6,972,245		7,282,261
Assets held for sale (1)		382,462	[1]
Loans	4,450,322	4,317,099		4,732,685
Total loans	4,450,322	4,317,099		4,732,685
Deposits	4,716,032	4,465,114		5,095,420
Liabilities held for sale (2)		536,186	[2]
PNB
Segment Reporting Information [Line Items]
Net interest income	221,758	236,282		237,281
Provision for loan losses	16,678	30,220		23,474
Other income (loss)	70,739	90,982		80,512
Other expense	156,516	146,235		144,051
Income (loss) before taxes	119,303	150,809		150,268
Income before income taxes	119,303	150,809		150,268
Income taxes (benefit)	32,197	43,958		47,320
Net income	87,106	106,851		102,948
Assets	6,502,579	6,281,747		6,495,558
Assets held for sale (1)		0	[1]
Loans	4,369,173	4,172,424		4,074,775
Total loans	4,369,173	4,172,424		4,074,775
Deposits	4,814,107	4,611,646		4,622,693
Liabilities held for sale (2)		0	[2]
VB
Segment Reporting Information [Line Items]
Net interest income	0	27,078		27,867
Provision for loan losses	0	31,052		61,407
Other income (loss)	0	6,617		(6,024)
Other expense	0	31,379		31,623
Income (loss) before taxes	0	(28,736)		(71,187)
Income before income taxes	0	(28,736)		(71,187)
Income taxes (benefit)	0	(6,210)		(25,773)
Net income	0	(22,526)		(45,414)
Assets	0	650,935		791,945
Assets held for sale (1)		382,462	[1]
Loans	0	123,883		640,580
Total loans	0	123,883		640,580
Deposits	0	32		633,432
Liabilities held for sale (2)		536,186	[2]
GFSC
Segment Reporting Information [Line Items]
Net interest income	9,156	8,693		7,611
Provision for loan losses	859	2,000		2,199
Other income (loss)	0	0		2
Other expense	2,835	2,506		2,326
Income (loss) before taxes	5,462	4,187		3,088
Income before income taxes	5,462	4,187		3,088
Income taxes (benefit)	1,912	1,466		1,082
Net income	3,550	2,721		2,006
Assets	49,926	46,682		43,209
Assets held for sale (1)		0	[1]
Loans	50,082	47,111		43,714
Total loans	50,082	47,111		43,714
Deposits	8,358	8,013		7,062
Liabilities held for sale (2)		0	[2]
SEPH
Segment Reporting Information [Line Items]
Net interest income	(341)	(974)		0
Provision for loan losses	17,882	0		0
Other income (loss)	21,431	(3,039)		0
Other expense	22,032	1,082		0
Income (loss) before taxes	(18,824)	(5,095)		0
Income before income taxes	(18,824)	(5,095)		0
Income taxes (benefit)	(6,603)	(1,784)		0
Net income	(12,221)	(3,311)		0
Assets	104,428	34,989		0
Assets held for sale (1)		0	[1]
Loans	59,178	0		0
Total loans	59,178	0		0
Deposits	0	0		0
Liabilities held for sale (2)		0	[2]
All Other
Segment Reporting Information [Line Items]
Net interest income	4,742	2,155		1,285
Provision for loan losses	0	0		0
Other income (loss)	233	350		390
Other expense	6,585	7,115		9,107
Income (loss) before taxes	(1,610)	(4,610)		(7,432)
Income before income taxes	(1,610)	(4,610)		(7,432)
Income taxes (benefit)	(1,805)	(3,015)		(5,993)
Net income	195	(1,595)		(1,439)
Assets	(14,130)	(42,108)		(48,451)
Assets held for sale (1)		0	[1]
Loans	(28,111)	(26,319)		(26,384)
Total loans	(28,111)	(26,319)		(26,384)
Deposits	(106,433)	(154,577)		(167,767)
Liabilities held for sale (2)		$ 0	[2]

[1] The assets held for sale represent the loans and other assets at Vision Bank that were sold in the first quarter of 2012.

[2] The liabilities held for sale represent the deposits and other liabilities at Vision Bank that were sold in the first quarter of 2012.