Loans (Aging Analysis of Past Due and Nonaccrual Loans by Class) (Details) - USD ($) $ in Thousands	Dec. 31, 2017	Dec. 31, 2016
Financing Receivable, Recorded Investment [Line Items]
Total past due	$ 8,308	$ 13,279
Nonaccrual	2,994	7,237
Current	1,162,366	960,135
Total loans held for investment, current	1,139,556	945,362
Total	1,147,864	958,641
SBA loans held for sale	22,810	14,773
Total loans	1,170,674	973,414
Nonaccrual loans, TDRs		153
SBA loans held for investment
Financing Receivable, Recorded Investment [Line Items]
Total past due	1,185	2,056
Nonaccrual	632	1,168
Current	42,814	40,436
Total	43,999	42,492
SBA 504 loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	513
Nonaccrual	0	513
Current	21,871	25,831
Total	21,871	26,344
Commercial loans
Financing Receivable, Recorded Investment [Line Items]
Total	606,994	509,171
Commercial loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	108	75
Nonaccrual	25	25
Current	82,717	58,372
Total	82,825	58,447
Commercial loans | Commercial real estate
Financing Receivable, Recorded Investment [Line Items]
Total past due	1,674	2,083
Nonaccrual	43	401
Current	468,022	420,335
Total	469,696	422,418
Commercial loans | Commercial real estate construction
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
Nonaccrual	0	0
Current	54,473	28,306
Total	54,473	28,306
Residential mortgage loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	4,457	5,867
Nonaccrual	1,669	2,672
Current	360,688	283,226
Total	365,145	289,093
Consumer loans
Financing Receivable, Recorded Investment [Line Items]
Total	109,855	91,541
Consumer loans | Home equity
Financing Receivable, Recorded Investment [Line Items]
Total past due	881	709
Nonaccrual	625	482
Current	54,936	46,702
Total	55,817	47,411
Consumer loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	3	1,976
Nonaccrual	0	1,976
Current	54,035	42,154
Total	54,038	44,130
SBA [Member]
Financing Receivable, Recorded Investment [Line Items]
Nonaccrual	27	60
30-59 days past due
Financing Receivable, Recorded Investment [Line Items]
Total past due	2,705	4,808
30-59 days past due | SBA loans held for investment
Financing Receivable, Recorded Investment [Line Items]
Total past due	240	491
30-59 days past due | SBA 504 loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
30-59 days past due | Commercial loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	23	50
30-59 days past due | Commercial loans | Commercial real estate
Financing Receivable, Recorded Investment [Line Items]
Total past due	558	1,108
30-59 days past due | Commercial loans | Commercial real estate construction
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
30-59 days past due | Residential mortgage loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	1,830	2,932
30-59 days past due | Consumer loans | Home equity
Financing Receivable, Recorded Investment [Line Items]
Total past due	51	227
30-59 days past due | Consumer loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	3	0
60-89 days past due
Financing Receivable, Recorded Investment [Line Items]
Total past due	2,549	1,234
60-89 days past due | SBA loans held for investment
Financing Receivable, Recorded Investment [Line Items]
Total past due	313	397
60-89 days past due | SBA 504 loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
60-89 days past due | Commercial loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
60-89 days past due | Commercial loans | Commercial real estate
Financing Receivable, Recorded Investment [Line Items]
Total past due	1,073	574
60-89 days past due | Commercial loans | Commercial real estate construction
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
60-89 days past due | Residential mortgage loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	958	263
60-89 days past due | Consumer loans | Home equity
Financing Receivable, Recorded Investment [Line Items]
Total past due	205	0
60-89 days past due | Consumer loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing
Financing Receivable, Recorded Investment [Line Items]
Total past due	60	0
90 days and still accruing | SBA loans held for investment
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | SBA 504 loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Commercial loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	60	0
90 days and still accruing | Commercial loans | Commercial real estate
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Commercial loans | Commercial real estate construction
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Residential mortgage loans
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Consumer loans | Home equity
Financing Receivable, Recorded Investment [Line Items]
Total past due	0	0
90 days and still accruing | Consumer loans | Other
Financing Receivable, Recorded Investment [Line Items]
Total past due	$ 0	$ 0