Loans and Credit Quality - Loans by Loan Grade (Details) - USD ($) $ in Thousands	Dec. 31, 2018	Dec. 31, 2017
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	$ 5,093,747	$ 4,529,627
Fair value of loans held for investment	4,057	5,477
Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	4,964,980	4,359,445
Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	66,819	128,563
Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	42,800	22,410
Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	19,148	19,209
Consumer Portfolio Segment [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	1,929,098	1,834,855
Consumer Portfolio Segment [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	1,907,395	1,807,159
Consumer Portfolio Segment [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	2,977	3,125
Consumer Portfolio Segment [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	9,285	12,079
Consumer Portfolio Segment [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	9,441	12,492
Consumer Portfolio Segment [Member] | Residential Mortgage [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	1,358,175	1,381,366
Consumer Portfolio Segment [Member] | Residential Mortgage [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	1,338,025	1,355,965
Consumer Portfolio Segment [Member] | Residential Mortgage [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	2,882	2,982
Consumer Portfolio Segment [Member] | Residential Mortgage [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	8,775	11,328
Consumer Portfolio Segment [Member] | Residential Mortgage [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	8,493	11,091
Consumer Portfolio Segment [Member] | Home Equity Line of Credit [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	570,923	453,489
Consumer Portfolio Segment [Member] | Home Equity Line of Credit [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	569,370	451,194
Consumer Portfolio Segment [Member] | Home Equity Line of Credit [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	95	143
Consumer Portfolio Segment [Member] | Home Equity Line of Credit [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	510	751
Consumer Portfolio Segment [Member] | Home Equity Line of Credit [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	948	1,401
Commercial Real Estate Portfolio Segment [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	2,404,487	2,038,450
Commercial Real Estate Portfolio Segment [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	2,366,087	1,970,396
Commercial Real Estate Portfolio Segment [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	26,583	64,901
Commercial Real Estate Portfolio Segment [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	7,001	1,973
Commercial Real Estate Portfolio Segment [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	4,816	1,180
Commercial Real Estate Portfolio Segment [Member] | Non-Owner Occupied Commercial Real Estate [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	701,928	622,782
Commercial Real Estate Portfolio Segment [Member] | Non-Owner Occupied Commercial Real Estate [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	695,077	613,181
Commercial Real Estate Portfolio Segment [Member] | Non-Owner Occupied Commercial Real Estate [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	1,426	8,801
Commercial Real Estate Portfolio Segment [Member] | Non-Owner Occupied Commercial Real Estate [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	5,425	0
Commercial Real Estate Portfolio Segment [Member] | Non-Owner Occupied Commercial Real Estate [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	0	800
Commercial Real Estate Portfolio Segment [Member] | Multifamily [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	908,015	728,037
Commercial Real Estate Portfolio Segment [Member] | Multifamily [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	903,897	693,190
Commercial Real Estate Portfolio Segment [Member] | Multifamily [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	3,626	34,038
Commercial Real Estate Portfolio Segment [Member] | Multifamily [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	492	507
Commercial Real Estate Portfolio Segment [Member] | Multifamily [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	0	302
Commercial Real Estate Portfolio Segment [Member] | Construction Loans [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	794,544	687,631
Commercial Real Estate Portfolio Segment [Member] | Construction Loans [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	767,113	664,025
Commercial Real Estate Portfolio Segment [Member] | Construction Loans [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	21,531	22,062
Commercial Real Estate Portfolio Segment [Member] | Construction Loans [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	1,084	1,466
Commercial Real Estate Portfolio Segment [Member] | Construction Loans [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	4,816	78
Commercial Portfolio Segment [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	760,162	656,322
Commercial Portfolio Segment [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	691,498	581,890
Commercial Portfolio Segment [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	37,259	60,537
Commercial Portfolio Segment [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	26,514	8,358
Commercial Portfolio Segment [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	4,891	5,537
Commercial Portfolio Segment [Member] | Owner Occupied Commercial Real Estate [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	429,158	391,613
Commercial Portfolio Segment [Member] | Owner Occupied Commercial Real Estate [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	392,273	361,429
Commercial Portfolio Segment [Member] | Owner Occupied Commercial Real Estate [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	22,928	20,949
Commercial Portfolio Segment [Member] | Owner Occupied Commercial Real Estate [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	11,087	6,399
Commercial Portfolio Segment [Member] | Owner Occupied Commercial Real Estate [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	2,870	2,836
Commercial Portfolio Segment [Member] | Commercial Business [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	331,004	264,709
Commercial Portfolio Segment [Member] | Commercial Business [Member] | Pass [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	299,225	220,461
Commercial Portfolio Segment [Member] | Commercial Business [Member] | Watch [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	14,331	39,588
Commercial Portfolio Segment [Member] | Commercial Business [Member] | Special Mention [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	15,427	1,959
Commercial Portfolio Segment [Member] | Commercial Business [Member] | Substandard [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Total loans receivable before fees and costs	2,021	2,701
Fair Value, Measurements, Recurring [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Fair value of loans held for investment	4,057	5,477
Fair Value, Inputs, Level 3 [Member] | Fair Value, Measurements, Recurring [Member]
Designated Loan Grades by Loan Portfolio Segment and Loan Class [Abstract]
Fair value of loans held for investment	$ 4,057	$ 5,477