Loans - Loans Individually Evaluated for Impairment by Class of Loans (Details) (USD $) In Thousands, unless otherwise specified	Dec. 31, 2012	Dec. 31, 2011
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	$ 242,345	$ 290,908
Recorded investment	137,339	187,135
Allowance for loan losses allocated	8,276	15,935
Commercial, financial and agricultural
Financing Receivable, Impaired [Line Items]
Recorded investment	22,587	40,621
Commercial, financial and agricultural | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	23,782	23,164
Recorded investment	14,683	18,098
Allowance for loan losses allocated
Commercial, financial and agricultural | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	12,268	23,719
Recorded investment	7,904	22,523
Allowance for loan losses allocated	3,180	5,819
Commercial real estate
Financing Receivable, Impaired [Line Items]
Recorded investment	44,278	51,978
Commercial real estate | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	56,258	58,242
Recorded investment	35,097	41,506
Allowance for loan losses allocated
Commercial real estate | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	11,412	12,183
Recorded investment	9,181	10,472
Allowance for loan losses allocated	1,540	4,431
SEPH/Vision commercial land and development | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	56,075	54,032
Recorded investment	12,740	17,786
Allowance for loan losses allocated
SEPH/Vision commercial land and development | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	1,271	20,775
Recorded investment	520	6,542
Allowance for loan losses allocated		1,540
Remaining commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	21,574	25,912
Remaining commercial | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	29,328	33,319
Recorded investment	14,093	18,372
Allowance for loan losses allocated
Remaining commercial | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	8,071	9,711
Recorded investment	7,481	7,540
Allowance for loan losses allocated	2,277	1,874
Commercial
Financing Receivable, Impaired [Line Items]
Recorded investment	35,622	44,276
Commercial | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	39,918	49,341
Recorded investment	31,957	38,686
Allowance for loan losses allocated
Commercial | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	3,944	6,402
Recorded investment	3,665	5,590
Allowance for loan losses allocated	1,279	2,271
Consumer
Financing Receivable, Impaired [Line Items]
Recorded investment	18	20
Consumer | With no related allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance	18
Recorded investment	18
Allowance for loan losses allocated
Consumer | With an allowance recorded
Financing Receivable, Impaired [Line Items]
Unpaid principal balance		20
Recorded investment		20
Allowance for loan losses allocated