Loans and Allowance for Loan Losses (Granite Purchased Loans) (Details) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2013	Mar. 31, 2013	Dec. 31, 2012		Sep. 30, 2012	Mar. 31, 2012	Dec. 31, 2011
Financing Receivable [Line Items]
Allowance of loan losses	$ 25,387	$ 25,085	$ 29,314	[1]	$ 30,859	$ 38,551	$ 39,360
Loans held for investment	1,195,142		1,177,035	[1]
PI [Member]
Financing Receivable [Line Items]
Loans held for investment	176,593		228,392
PC [Member]
Financing Receivable [Line Items]
Loans held for investment	1,018,549		948,643
Granite Corp [Member]
Financing Receivable [Line Items]
Purchased impaired	176,593		228,392		252,205
Purchased contractual	28,088		34,844
Loans held for investment	204,681		263,236
Unpaid principal balance	205,230		271,060
Granite Corp [Member] | Commercial and agricultural [Member]
Financing Receivable [Line Items]
Purchased impaired	11,418		14,666
Purchased contractual	4,874		7,311
Loans held for investment	16,292		21,977
Unpaid principal balance	15,597		21,692
Granite Corp [Member] | Real estate - construction [Member]
Financing Receivable [Line Items]
Purchased impaired	10,196		2,682
Purchased contractual	0		0
Loans held for investment	10,196		2,682
Unpaid principal balance	9,902		2,677
Granite Corp [Member] | Real estate - mortgage, 1-4 family residential [Member]
Financing Receivable [Line Items]
Purchased impaired	29,712		39,215
Purchased contractual	22,833		27,484
Loans held for investment	52,545		66,699
Unpaid principal balance	53,476		69,200
Granite Corp [Member] | Real estate - mortgage, commercial [Member]
Financing Receivable [Line Items]
Purchased impaired	124,178		170,467
Purchased contractual	381		49
Loans held for investment	124,559		170,516
Unpaid principal balance	125,395		176,347
Granite Corp [Member] | Consumer [Member]
Financing Receivable [Line Items]
Purchased impaired	1,089		1,362
Purchased contractual	0		0
Loans held for investment	1,089		1,362
Unpaid principal balance	860		1,144
Granite Corp [Member] | PI [Member]
Financing Receivable [Line Items]
Allowance of loan losses	4,100		5,400
Granite Corp [Member] | PC [Member]
Financing Receivable [Line Items]
Allowance of loan losses	$ 700		$ 500

[1] Derived from audited consolidated financial statements