Loans - Loans Individually Evaluated for Impairment by Class of Loans (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	$ 175,578	$ 242,345
Recorded investment	112,315	137,339	187,135
Allowance for loan losses allocated	10,451	8,276
Impaired Financing Receivable, Average Recorded Investment	124,089	164,164	214,045
Impaired Financing Receivable, Interest Income, Cash Basis Method	2,640	2,813	1,592
Commercial, financial and agricultural
Financing Receivable, Impaired [Line Items]
Recorded investment	20,727	22,587	40,621
Impaired Financing Receivable, Average Recorded Investment	20,523	35,305	23,518
Impaired Financing Receivable, Interest Income, Cash Basis Method	412	529	209
Commercial, financial and agricultural | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	22,429	23,782
Recorded investment	12,885	14,683
Allowance for loan losses allocated
Commercial, financial and agricultural | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	12,616	12,268
Recorded investment	7,842	7,904
Allowance for loan losses allocated	3,268	3,180
Commercial real estate
Financing Receivable, Impaired [Line Items]
Recorded investment	41,822	44,278	51,978
Impaired Financing Receivable, Average Recorded Investment	41,426	44,541	49,927
Impaired Financing Receivable, Interest Income, Cash Basis Method	1,151	968	829
Commercial real estate | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	56,870	56,258
Recorded investment	34,149	35,097
Allowance for loan losses allocated
Commercial real estate | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	7,966	11,412
Recorded investment	7,673	9,181
Allowance for loan losses allocated	5,496	1,540
SEPH commercial land and development | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	23,722	57,346
Recorded investment	4,777	13,260
Allowance for loan losses allocated
Remaining commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	10,782	21,574	25,912
Impaired Financing Receivable, Average Recorded Investment	17,829	27,774	29,152
Impaired Financing Receivable, Interest Income, Cash Basis Method	616	818	339
Remaining commercial | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	8,429	29,328
Recorded investment	6,872	14,093
Allowance for loan losses allocated
Remaining commercial | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	3,909	8,071
Recorded investment	3,910	7,481
Allowance for loan losses allocated	1,132	2,277
Commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	33,408	35,622	44,276
Impaired Financing Receivable, Average Recorded Investment	34,972	39,248	52,640
Impaired Financing Receivable, Interest Income, Cash Basis Method	461	497	214
Commercial | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	36,709	39,918
Recorded investment	31,461	31,957
Allowance for loan losses allocated
Commercial | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	2,129	3,944
Recorded investment	1,947	3,665
Allowance for loan losses allocated	555	1,279
Consumer
Financing Receivable, Impaired [Line Items]
Recorded investment	799	18	20
Impaired Financing Receivable, Average Recorded Investment	616	19	16
Impaired Financing Receivable, Interest Income, Cash Basis Method		1	1
Consumer | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	799	18
Recorded investment	799	18
Allowance for loan losses allocated
Consumer | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance
Recorded investment
Allowance for loan losses allocated
Construction Real Estate Vision Commercial Land And Development [Member]
Financing Receivable, Impaired [Line Items]
Recorded investment		13,260	24,328
Impaired Financing Receivable, Average Recorded Investment		17,277	58,792
Impaired Financing Receivable, Interest Income, Cash Basis Method