Loans and Allowance for Loan Losses (Granite Purchased Loans) (Details) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2014	Jun. 30, 2014	Dec. 31, 2013		Sep. 30, 2013	Jun. 30, 2013	Dec. 31, 2012
Financing Receivable [Line Items]
Allowance of loan losses	$ 21,525	$ 23,975	$ 26,785	[1]	$ 25,387	$ 25,085	$ 29,314
Loans held for investment	1,318,117		1,212,248	[1]
PI [Member]
Financing Receivable [Line Items]
Loans held for investment	130,665		161,652
PC [Member]
Financing Receivable [Line Items]
Loans held for investment	1,187,452		1,050,596
Granite Corp [Member]
Financing Receivable [Line Items]
Purchased impaired	130,665	141,924	161,652		176,593	189,159
Purchased contractual	26,927		28,448
Loans held for investment	157,592		190,100
Unpaid principal balance	166,639		202,174
Granite Corp [Member] | Commercial and agricultural [Member]
Financing Receivable [Line Items]
Purchased impaired	8,258		10,729
Purchased contractual	3,786		5,948
Loans held for investment	12,044		16,677
Unpaid principal balance	10,840		16,452
Granite Corp [Member] | Real estate - construction [Member]
Financing Receivable [Line Items]
Purchased impaired	9,277		9,893
Purchased contractual	0		0
Loans held for investment	9,277		9,893
Unpaid principal balance	10,886		11,368
Granite Corp [Member] | Real estate - mortgage, 1-4 family residential [Member]
Financing Receivable [Line Items]
Purchased impaired	18,318		26,853
Purchased contractual	23,141		22,127
Loans held for investment	41,459		48,980
Unpaid principal balance	43,659		51,359
Granite Corp [Member] | Real estate - mortgage, commercial [Member]
Financing Receivable [Line Items]
Purchased impaired	93,830		113,178
Purchased contractual	0		373
Loans held for investment	93,830		113,551
Unpaid principal balance	100,567		122,197
Granite Corp [Member] | Consumer [Member]
Financing Receivable [Line Items]
Purchased impaired	982		999
Purchased contractual	0		0
Loans held for investment	982		999
Unpaid principal balance	687		798
Granite Corp [Member] | PI [Member]
Financing Receivable [Line Items]
Allowance of loan losses	3,900		5,600
Granite Corp [Member] | PC [Member]
Financing Receivable [Line Items]
Allowance of loan losses	$ 600		$ 900

[1] Derived from audited consolidated financial statements