Loans and Allowance for Loan Losses (Granite Purchased Loans) (Details) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2014	Mar. 31, 2014	Dec. 31, 2013		Jun. 30, 2013	Mar. 31, 2013	Dec. 31, 2012
Financing Receivable [Line Items]
Allowance of loan losses	$ 23,975	$ 26,039	$ 26,785	[1]	$ 25,085	$ 29,641	$ 29,314
Loans held for investment	1,269,865		1,212,248	[1]
PI [Member]
Financing Receivable [Line Items]
Loans held for investment	141,924		161,652
PC [Member]
Financing Receivable [Line Items]
Loans held for investment	1,127,941		1,050,596
Granite Corp [Member]
Financing Receivable [Line Items]
Purchased impaired	141,924	149,033	161,652		189,159	205,884
Purchased contractual	27,970		28,448
Loans held for investment	169,894		190,100
Unpaid principal balance	179,232		202,174
Granite Corp [Member] | Commercial and agricultural [Member]
Financing Receivable [Line Items]
Purchased impaired	9,024		10,729
Purchased contractual	3,727		5,948
Loans held for investment	12,751		16,677
Unpaid principal balance	11,627		16,452
Granite Corp [Member] | Real estate - construction [Member]
Financing Receivable [Line Items]
Purchased impaired	10,437		9,893
Purchased contractual	0		0
Loans held for investment	10,437		9,893
Unpaid principal balance	11,521		11,368
Granite Corp [Member] | Real estate - mortgage, 1-4 family residential [Member]
Financing Receivable [Line Items]
Purchased impaired	19,528		26,853
Purchased contractual	24,232		22,127
Loans held for investment	43,760		48,980
Unpaid principal balance	46,072		51,359
Granite Corp [Member] | Real estate - mortgage, commercial [Member]
Financing Receivable [Line Items]
Purchased impaired	101,847		113,178
Purchased contractual	11		373
Loans held for investment	101,858		113,551
Unpaid principal balance	109,300		122,197
Granite Corp [Member] | Consumer [Member]
Financing Receivable [Line Items]
Purchased impaired	1,088		999
Purchased contractual	0		0
Loans held for investment	1,088		999
Unpaid principal balance	712		798
Granite Corp [Member] | PI [Member]
Financing Receivable [Line Items]
Allowance of loan losses	4,100		5,600
Granite Corp [Member] | PC [Member]
Financing Receivable [Line Items]
Allowance of loan losses	$ 300		$ 300

[1] Derived from audited consolidated financial statements