Loans - Average Recorded Investment and Interest Income Recognized on Loans Individually Evaluated for Impairment (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011
Financing Receivable, Impaired [Line Items]
Recorded investment	$ 112,315	$ 137,339	$ 187,135
Average recorded investment	124,089	164,164	214,045
Interest income recognized	2,640	2,813	1,592
Commercial, financial and agricultural
Financing Receivable, Impaired [Line Items]
Recorded investment	20,727	22,587	40,621
Average recorded investment	20,523	35,305	23,518
Interest income recognized	412	529	209
Commercial real estate
Financing Receivable, Impaired [Line Items]
Recorded investment	41,822	44,278	51,978
Average recorded investment	41,426	44,541	49,927
Interest income recognized	1,151	968	829
SEPH commercial land and development
Financing Receivable, Impaired [Line Items]
Recorded investment	4,777
Average recorded investment	8,723
Interest income recognized
SEPH commercial land and development
Financing Receivable, Impaired [Line Items]
Recorded investment		13,260	24,328
Average recorded investment		17,277	58,792
Interest income recognized
Remaining commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	10,782	21,574	25,912
Average recorded investment	17,829	27,774	29,152
Interest income recognized	616	818	339
Commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	33,408	35,622	44,276
Average recorded investment	34,972	39,248	52,640
Interest income recognized	461	497	214
Consumer
Financing Receivable, Impaired [Line Items]
Recorded investment	799	18	20
Average recorded investment	616	19	16
Interest income recognized		$ 1	$ 1