Loans and Credit Quality Loans and Credit Quality (Recorded investment in loans by impairment methodology) (Details) - USD ($) $ in Thousands		Jun. 30, 2016	Mar. 31, 2016	Dec. 31, 2015	Jun. 30, 2015	Mar. 31, 2015	Dec. 31, 2014
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance collectively evaluated for impairment		$ 33,410		$ 30,092
Allowance individually evaluated for impairment		591		567
Allowance for credit losses		34,001	$ 32,423	30,659	$ 26,448	$ 25,628	$ 22,524
Loans collectively evaluated for impairment		3,614,861		3,109,018
Loans individually evaluated for impairment		93,613		93,673
Total		3,708,474		3,202,691
Fair value of loans held for investment	[1]	22,362		21,544
Total loans held for investment		3,730,836		3,224,235
Residential Mortgage [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance collectively evaluated for impairment		8,078		8,723
Allowance individually evaluated for impairment		216		219
Allowance for credit losses		8,294	9,026	8,942	8,997	9,959	9,447
Loans collectively evaluated for impairment		1,112,375		1,101,891
Loans individually evaluated for impairment		83,479		79,745
Total		1,195,854		1,181,636
Total loans held for investment	[2]	1,218,216		1,203,180
Home Equity Line of Credit [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance collectively evaluated for impairment		5,334		4,545
Allowance individually evaluated for impairment		66		75
Allowance for credit losses		5,400	4,852	4,620	3,882	3,331	3,322
Loans collectively evaluated for impairment		307,808		254,762
Loans individually evaluated for impairment		1,396		1,611
Total		309,204		256,373
Total loans held for investment		309,204		256,373
Consumer Portfolio Segment [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance collectively evaluated for impairment		13,412		13,268
Allowance individually evaluated for impairment		282		294
Allowance for credit losses		13,694	13,878	13,562	12,879	13,290	12,769
Loans collectively evaluated for impairment		1,420,183		1,356,653
Loans individually evaluated for impairment		84,875		81,356
Total		1,505,058		1,438,009
Total loans held for investment		1,527,420		1,459,553
Commercial Real Estate [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance collectively evaluated for impairment		6,045		4,847
Allowance individually evaluated for impairment		0		0
Allowance for credit losses		6,045	5,175	4,847	5,046	4,551	3,846
Loans collectively evaluated for impairment		759,055		597,571
Loans individually evaluated for impairment		3,115		3,132
Total		762,170		600,703
Total loans held for investment		762,170		600,703
Multifamily Residential [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance collectively evaluated for impairment		2,048		1,194
Allowance individually evaluated for impairment		0		0
Allowance for credit losses		2,048	1,832	1,194	780	661	673
Loans collectively evaluated for impairment		562,107		423,424
Loans individually evaluated for impairment		621		3,133
Total		562,728		426,557
Total loans held for investment		562,728		426,557
Construction Loans [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance collectively evaluated for impairment		9,369		9,271
Allowance individually evaluated for impairment		0		0
Allowance for credit losses		9,369	9,286	9,271	5,943	5,003	3,818
Loans collectively evaluated for impairment		637,108		579,446
Loans individually evaluated for impairment		2,333		3,714
Total		639,441		583,160
Total loans held for investment		639,441		583,160
Commercial Loan [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance collectively evaluated for impairment		2,536		1,512
Allowance individually evaluated for impairment		309		273
Allowance for credit losses		2,845	2,252	1,785	1,800	2,123	1,418
Loans collectively evaluated for impairment		236,408		151,924
Loans individually evaluated for impairment		2,669		2,338
Total		239,077		154,262
Total loans held for investment		239,077		154,262
Commercial Portfolio Segment [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance collectively evaluated for impairment		19,998		16,824
Allowance individually evaluated for impairment		309		273
Allowance for credit losses		20,307	$ 18,545	17,097	$ 13,569	$ 12,338	$ 9,755
Loans collectively evaluated for impairment		2,194,678		1,752,365
Loans individually evaluated for impairment		8,738		12,317
Total		2,203,416		1,764,682
Total loans held for investment		$ 2,203,416		$ 1,764,682

[1] Comprised of single family loans where a fair value option election was made at the time of origination and, therefore, are carried at fair value with changes recognized in the consolidated statements of operations.

[2] Includes $22.4 million and $21.5 million at June 30, 2016 and December 31, 2015, respectively, of loans where a fair value option election was made at the time of origination and, therefore, are carried at fair value with changes recognized in the consolidated statements of operations.