Allowance for Loan Losses - Allowance Calculation at the Segment Level and Adjusted Beginning Balance for the Allowance for Credit Losses (Details) (USD $) In Thousands, unless otherwise specified	6 Months Ended	9 Months Ended	12 Months Ended
	Jun. 30, 2013	Sep. 30, 2013	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011
Financing Receivable, Allowance for Credit Losses [Line Items]
Allowance for Loan and Lease Losses, Reserve Estimate Methodology, Historical Loss Period Factor	48 months	54 months	60 months
General reserves as a percent of performing loans	1.10%	1.10%
Beginning balance	$ 55,537	$ 55,537	$ 55,537	$ 68,444	$ 143,575
Transfer of loans at fair value					219
Transfer of allowance to held for sale					13,100
Charge-offs			19,153	61,268	133,882
Recoveries			(19,669)	(12,942)	(8,798)
Net Charge-offs			(516)	48,326	125,084
Provision (Recovery)			3,415	35,419	63,272
Ending balance			59,468	55,537	68,444
Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning balance	15,635	15,635	15,635	16,950	11,555
Transfer of loans at fair value					2
Transfer of allowance to held for sale					1,184
Charge-offs			6,160	26,847	18,350
Recoveries			(1,314)	(1,066)	(1,402)
Net Charge-offs			4,846	25,781	16,948
Provision (Recovery)			3,429	24,466	23,529
Ending balance			14,218	15,635	16,950
Commercial real estate
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning balance	11,736	11,736	11,736	15,539	24,369
Transfer of loans at fair value					150
Transfer of allowance to held for sale					4,327
Charge-offs			1,832	10,454	23,063
Recoveries			(726)	(783)	(1,825)
Net Charge-offs			1,106	9,671	21,238
Provision (Recovery)			5,269	5,868	16,885
Ending balance			15,899	11,736	15,539
Construction real estate
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning balance	6,841	6,841	6,841	14,433	70,462
Transfer of loans at fair value					63
Transfer of allowance to held for sale					1,998
Charge-offs			1,791	9,985	64,166
Recoveries			(9,378)	(2,979)	(1,463)
Net Charge-offs			(7,587)	7,006	62,703
Provision (Recovery)			(7,573)	(586)	8,735
Ending balance			6,855	6,841	14,433
Residential real estate
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning balance	14,759	14,759	14,759	15,692	30,259
Transfer of loans at fair value					4
Transfer of allowance to held for sale					5,450
Charge-offs			3,207	8,607	20,691
Recoveries			(6,000)	(5,559)	(1,719)
Net Charge-offs			(2,793)	3,048	18,972
Provision (Recovery)			(3,301)	2,115	9,859
Ending balance			14,251	14,759	15,692
Consumer
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning balance	6,566	6,566	6,566	5,830	6,925
Transfer of loans at fair value					0
Transfer of allowance to held for sale					141
Charge-offs			6,163	5,375	7,612
Recoveries			(2,249)	(2,555)	(2,385)
Net Charge-offs			3,914	2,820	5,227
Provision (Recovery)			5,593	3,556	4,273
Ending balance			8,245	6,566	5,830
Leases
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning balance	0	0	0	0	5
Transfer of loans at fair value					0
Transfer of allowance to held for sale					0
Charge-offs			0	0	0
Recoveries			(2)	0	(4)
Net Charge-offs			(2)	0	(4)
Provision (Recovery)			(2)	0	(9)
Ending balance			$ 0	$ 0	$ 0