Loans - Average Recorded Investment and Interest Income Recognized on Loans Individually Evaluated for Impairment (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Financing Receivable, Impaired [Line Items]
Recorded investment	$ 80,664	$ 73,712	$ 112,315
Average recorded investment	70,762	92,614	124,089
Interest income recognized	1,963	2,678	2,640
Commercial, financial and agricultural
Financing Receivable, Impaired [Line Items]
Recorded investment	30,595	19,106	20,727
Average recorded investment	20,179	19,518	20,523
Interest income recognized	340	360	412
Commercial real estate
Financing Receivable, Impaired [Line Items]
Recorded investment	18,025	21,989	41,822
Average recorded investment	17,883	31,945	41,426
Interest income recognized	550	1,027	1,151
SEPH commercial land and development
Financing Receivable, Impaired [Line Items]
Recorded investment	2,044	2,078	4,777
Average recorded investment	2,066	3,658	8,723
Interest income recognized	21	146
Remaining commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	4,676	5,609	10,782
Average recorded investment	5,666	8,784	17,829
Interest income recognized	26	61	616
Commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	25,324	24,930	33,408
Average recorded investment	24,968	28,306	34,972
Interest income recognized	$ 1,026	1,084	461
Consumer
Financing Receivable, Impaired [Line Items]
Recorded investment			799
Average recorded investment		$ 403	$ 616