Allowance for Loan Losses and Reserve for Unfunded Loan Commitments (Activity in Allowance for Loan Losses by Portfolio Segment) (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2016	Mar. 31, 2015
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning Balance	$ 12,759	$ 12,551
Charge-offs	(342)	(847)
Recoveries	17	277
Net (charge-offs) recoveries	(325)	(570)
Provision for loan losses charged to expense	200	200
Ending Balance	12,634	12,181
Small Business Administration Portfolio Segment [Member]
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning Balance	1,961	1,883
Charge-offs	(86)	(128)
Recoveries	11	37
Net (charge-offs) recoveries	(75)	(91)
Provision for loan losses charged to expense	(173)	81
Ending Balance	1,713	1,873
Small Business Administration, 504 Portfolio Segment [Member]
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning Balance	741	1,337
Charge-offs		(589)
Net (charge-offs) recoveries		(589)
Provision for loan losses charged to expense	(143)	414
Ending Balance	598	1,162
Commercial Portfolio Segment [Member]
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning Balance	6,309	6,270
Charge-offs	(228)	(100)
Recoveries	6	201
Net (charge-offs) recoveries	(222)	101
Provision for loan losses charged to expense	224	(747)
Ending Balance	6,311	5,624
Residential Portfolio Segment [Member]
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning Balance	2,769	2,289
Recoveries		39
Net (charge-offs) recoveries		39
Provision for loan losses charged to expense	123	13
Ending Balance	2,892	2,341
Consumer Portfolio Segment [Member]
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning Balance	817	667
Charge-offs	(28)	(30)
Net (charge-offs) recoveries	(28)	(30)
Provision for loan losses charged to expense	47	73
Ending Balance	836	710
Unallocated Financing Receivables [Member]
Financing Receivable, Allowance for Credit Losses [Line Items]
Beginning Balance	162	105
Provision for loan losses charged to expense	122	366
Ending Balance	$ 284	$ 471