Loans (Granite Purchased Loans) (Details) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2012	Mar. 31, 2012	Dec. 31, 2011	Jun. 30, 2011	Mar. 31, 2011	Dec. 31, 2010
Financing Receivable [Line Items]
Allowance of loan losses	$ 38,551	$ 39,795	$ 39,360	$ 59,366	$ 68,729	$ 93,687
Loans held for investment	1,281,823		1,217,535
Commercial and agricultural [Member]
Financing Receivable [Line Items]
Loans held for investment	78,367		95,089
Real estate - construction [Member]
Financing Receivable [Line Items]
Loans held for investment	75,576		92,806
Real estate - mortgage, 1-4 family residential [Member]
Financing Receivable [Line Items]
Loans held for investment	589,972		453,725
Real estate - mortgage, commercial [Member]
Financing Receivable [Line Items]
Loans held for investment	493,721		531,383
Consumer [Member]
Financing Receivable [Line Items]
Loans held for investment	44,187		44,532
Granite Corp [Member]
Financing Receivable [Line Items]
Allowance of loan losses	3,336		0
Purchased impaired	287,092		330,836	330,836
Purchased contractual	36,656		42,652
Loans held for investment	323,748		373,488
Unpaid principal balance	338,516		404,130
Granite Corp [Member] | Commercial and agricultural [Member]
Financing Receivable [Line Items]
Purchased impaired	20,277		28,872
Purchased contractual	6,283		8,061
Loans held for investment	26,560		36,933
Unpaid principal balance	26,793		39,531
Granite Corp [Member] | Real estate - construction [Member]
Financing Receivable [Line Items]
Purchased impaired	5,671		7,641
Purchased contractual	0		0
Loans held for investment	5,671		7,641
Unpaid principal balance	6,101		8,413
Granite Corp [Member] | Real estate - mortgage, 1-4 family residential [Member]
Financing Receivable [Line Items]
Purchased impaired	46,150		52,894
Purchased contractual	30,373		34,590
Loans held for investment	76,523		87,484
Unpaid principal balance	79,980		93,310
Granite Corp [Member] | Real estate - mortgage, commercial [Member]
Financing Receivable [Line Items]
Purchased impaired	213,750		239,621
Purchased contractual	0		0
Loans held for investment	213,750		239,621
Unpaid principal balance	224,208		261,076
Granite Corp [Member] | Consumer [Member]
Financing Receivable [Line Items]
Purchased impaired	1,244		1,808
Purchased contractual	0		1
Loans held for investment	1,244		1,809
Unpaid principal balance	1,434		1,800
Granite Corp [Member] | PI [Member]
Financing Receivable [Line Items]
Allowance of loan losses	3,300
Granite Corp [Member] | PC [Member]
Financing Receivable [Line Items]
Allowance of loan losses	$ 400