Loans and Allowance for Loan Losses (Granite Purchased Loans) (Details) - USD ($) $ in Thousands	Sep. 30, 2015	Jun. 30, 2015	Dec. 31, 2014		Sep. 30, 2014	Jun. 30, 2014	Dec. 31, 2013
Financing Receivable [Line Items]
Allowance of loan losses	$ 17,188	$ 17,989	$ 20,345	[1]	$ 21,525	$ 23,975	$ 26,785
Total Loans held for investment	1,522,455		1,357,788	[1]
PI [Member]
Financing Receivable [Line Items]
Total Loans held for investment	92,677		122,842
PC [Member]
Financing Receivable [Line Items]
Total Loans held for investment	1,429,778		1,234,946
Granite Corp [Member]
Financing Receivable [Line Items]
Purchased impaired	92,677	$ 102,115	122,842		$ 130,665	$ 141,924	$ 161,652
Purchased contractual	20,748		25,948
Total Loans held for investment	113,425		148,790
Unpaid principal balance	114,599		153,774
Granite Corp [Member] | Commercial and agricultural [Member]
Financing Receivable [Line Items]
Purchased impaired	5,848		7,535
Purchased contractual	1,228		4,288
Total Loans held for investment	7,076		11,823
Unpaid principal balance	5,964		10,508
Granite Corp [Member] | Real estate - construction [Member]
Financing Receivable [Line Items]
Purchased impaired	7,995		9,197
Purchased contractual	0		0
Total Loans held for investment	7,995		9,197
Unpaid principal balance	8,821		10,463
Granite Corp [Member] | Real estate - mortgage, 1-4 family residential [Member]
Financing Receivable [Line Items]
Purchased impaired	14,791		17,737
Purchased contractual	19,520		21,660
Total Loans held for investment	34,311		39,397
Unpaid principal balance	35,303		41,295
Granite Corp [Member] | Real estate - mortgage, commercial [Member]
Financing Receivable [Line Items]
Purchased impaired	63,085		87,345
Purchased contractual	0		0
Total Loans held for investment	63,085		87,345
Unpaid principal balance	63,890		90,830
Granite Corp [Member] | Consumer [Member]
Financing Receivable [Line Items]
Purchased impaired	958		1,028
Purchased contractual	0		0
Total Loans held for investment	958		1,028
Unpaid principal balance	621		678
Granite Corp [Member] | PI [Member]
Financing Receivable [Line Items]
Allowance of loan losses	3,200		3,200
Granite Corp [Member] | PC [Member]
Financing Receivable [Line Items]
Allowance of loan losses	$ 400		$ 300

[1] Derived from audited consolidated financial statements