Loans and Allowance for Loan Losses (Narrative) (Details) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2016 USD ($) loan	Jun. 30, 2015 USD ($) loan	Jun. 30, 2016 USD ($) loan	Jun. 30, 2015 USD ($) loan	Mar. 31, 2016 USD ($)	Dec. 31, 2015 USD ($)		Mar. 31, 2015 USD ($)	Dec. 31, 2014 USD ($)
Financing Receivable [Line Items]
Net loan premiums	$ 4,000		$ 4,000			$ 3,700
Unpaid principal balance of loans serviced to others	376,500		376,500			304,600
Loans pledged to collateralize FHLB advances and letters of credit	91,900		91,900			102,900
Investment securities pledged to collateralize FHLB advances and letters of credit	119,800		119,800			113,000
Credit availability for borrowing under FHLB advances and letters of credit	82,000		82,000			80,800
Loans pledged to collateralize potential borrowings from Federal Reserve Discount Window	3,600		3,600
Securities pledged to collateralize potential borrowings from Federal Reserve Discount Window	42,800		42,800
Borrowing capacity under Federal Reserve Discount Window	42,400		42,400
Additional borrowings from FHLB	326,900		326,900
Interest lost on nonaccrual loans			500	$ 700
Loans on nonaccrual status	14,700		14,700			18,000
Restructured loans	16,100		16,100			17,900
Allowance of loan losses	13,705	$ 17,989	13,705	17,989	$ 14,240	$ 15,195	[1]	$ 19,008	$ 20,345
Charge-offs	2,041	1,567	3,135	2,562
Recoveries	999	1,336	1,673	2,132
Net charge-offs	$ 1,042	$ 231	$ 1,462	$ 430
ALL, as a percentage of loans held for investment	0.90%	1.24%	0.90%	1.24%		0.98%
Number of loans | loan	2	2	4	2
Number of loans with modified terms | loan			2
Number of loans with modified terms and interest rate | loan			2
Reserve for unfunded commitments	$ 1,200		$ 1,200			$ 800
Minimum [Member]
Financing Receivable [Line Items]
Straight lines of credit expected funding rate			50.00%
Maximum [Member]
Financing Receivable [Line Items]
Straight lines of credit expected funding rate			100.00%
Letter of Credit [Member]
Financing Receivable [Line Items]
Expected usage percentage of remaining unfunded commitments	10.00%
Granite Corp [Member]
Financing Receivable [Line Items]
Loans purchased	$ 69,844	$ 98,934	$ 69,844	$ 98,934
Substandard or Doubtful [Member]
Financing Receivable [Line Items]
Loans purchased	16,200		16,200			27,000
Performing Financial Instruments [Member]
Financing Receivable [Line Items]
Restructured loans	12,000		12,000			13,100
PI loans [Member] | Granite Corp [Member]
Financing Receivable [Line Items]
Allowance of loan losses	2,800		2,800			2,800
PC and Originated Loans [Member] | Granite Corp [Member]
Financing Receivable [Line Items]
Allowance of loan losses	$ 300		$ 300			$ 300

[1] Derived from audited consolidated financial statements