Loans - Average Recorded Investment and Interest Income Recognized on Loans Individually Evaluated for Impairment (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Financing Receivable, Impaired [Line Items]
Recorded investment	$ 70,426	$ 80,664	$ 73,712
Average recorded investment	79,496	70,762	92,614
Interest income recognized	4,777	1,963	2,678
Commercial, financial and agricultural
Financing Receivable, Impaired [Line Items]
Recorded investment	20,624	30,595	19,106
Average recorded investment	26,821	20,179	19,518
Interest income recognized	885	340	360
Commercial real estate
Financing Receivable, Impaired [Line Items]
Recorded investment	24,474	18,025	21,989
Average recorded investment	22,828	17,883	31,945
Interest income recognized	884	550	1,027
SEPH commercial land and development
Financing Receivable, Impaired [Line Items]
Recorded investment		2,044	2,078
Average recorded investment	1,597	2,066	3,658
Interest income recognized		21	146
Remaining commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	2,226	4,676	5,609
Average recorded investment	3,906	5,666	8,784
Interest income recognized	66	26	61
Commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	23,102	25,324	24,930
Average recorded investment	24,341	24,968	28,306
Interest income recognized	2,942	$ 1,026	1,084
Consumer
Financing Receivable, Impaired [Line Items]
Average recorded investment	$ 3		$ 403