Investment Securities (Details)	3 Months Ended
	Mar. 31, 2016 USD ($) item	Mar. 31, 2015 USD ($)	Dec. 31, 2015 USD ($) item
Carrying value and fair value of available for sale investment securities [Abstract]
Total available for sale securities	$ 198,074,000		$ 224,108,000
Gross unrealized gains	3,686,000		2,205,000
Gross unrealized losses	(532,000)		(2,246,000)
Available-for-sale Securities, Total	201,228,000		224,067,000
Amortized cost [Abstract]
Within 1 year	0
After 1 but within 5 years	11,644,000
After 5 but within 10 years	11,587,000
After 10 years	4,113,000
Total available for sale debt securities	195,931,000
Available-for-sale Equity Securities, Amortized Cost Basis	2,143,000
Total available for sale securities	198,074,000		224,108,000
Fair value [Abstract]
Within 1 year	0
After 1 but within 5 years	11,622,000
After 5 but within 10 years	11,649,000
After 10 years	4,179,000
Total available for sale debt securities	198,771,000
No contractual maturity	2,457,000
Fair value	201,228,000		224,067,000
Proceeds from sales of AFS investment securities	10,022,000	$ 18,698,000
Gross realized gains and losses on the sale of securities recognized in earnings [Abstract]
Gross realized gains	367,000	187,000
Gross realized losses	0	0
Net realized gains	367,000	187,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	10,491,000		106,022,000
12 months or longer, Fair value	18,651,000		37,100,000
Total, Fair value	29,142,000		143,122,000
Less than 12 months, Gross unrealized losses	(145,000)		(1,134,000)
12 months or longer, Gross unrealized losses	(387,000)		(1,112,000)
Total, Gross unrealized losses	$ (532,000)		$ (2,246,000)
Less than 12 months, number of positions | item	7		43
12 months or longer, number of positions | item	7		12
Total , number of positions | item	14		55
Fair value	$ 201,228,000		$ 224,067,000
Number of securities in an unrealized loss position for less than twelve months | item	7		43
Number of securities in an unrealized loss position for more than twelve months | item	7		12
Credit related impairment losses on debt securities held for which a portion of OTTI was recognized in other comprehensive income [Roll Forward]
Reductions for securities sold during the period (realized)	$ 0	$ 0
U.S. Government Agencies [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Total available for sale securities	15,949,000		$ 26,127,000
Gross unrealized gains	11,000		0
Gross unrealized losses	(69,000)		(564,000)
Available-for-sale Securities, Total	15,891,000		25,563,000
Amortized cost [Abstract]
Total available for sale securities	15,949,000		26,127,000
Fair value [Abstract]
Fair value	15,891,000		25,563,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value			6,681,000
12 months or longer, Fair value	4,288,000		18,882,000
Total, Fair value	4,288,000		25,563,000
Less than 12 months, Gross unrealized losses			(57,000)
12 months or longer, Gross unrealized losses	(69,000)		(507,000)
Total, Gross unrealized losses	$ (69,000)		$ (564,000)
Less than 12 months, number of positions | item			2
12 months or longer, number of positions | item	2		6
Total , number of positions | item	2		8
Fair value	$ 15,891,000		$ 25,563,000
Number of securities in an unrealized loss position for less than twelve months | item			2
Number of securities in an unrealized loss position for more than twelve months | item	2		6
Mortgage-backed Securities-residential [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Total available for sale securities	$ 10,673,000		$ 11,002,000
Gross unrealized gains	250,000		106,000
Gross unrealized losses	0		(50,000)
Available-for-sale Securities, Total	10,923,000		11,058,000
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	10,673,000
Total available for sale securities	10,673,000		11,002,000
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	10,923,000
Fair value	10,923,000		11,058,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value			5,140,000
12 months or longer, Fair value			2,574,000
Total, Fair value			7,714,000
Less than 12 months, Gross unrealized losses			(5,000)
12 months or longer, Gross unrealized losses			(45,000)
Total, Gross unrealized losses			$ (50,000)
Less than 12 months, number of positions | item			2
12 months or longer, number of positions | item			1
Total , number of positions | item			3
Fair value	10,923,000		$ 11,058,000
Number of securities in an unrealized loss position for less than twelve months | item			2
Number of securities in an unrealized loss position for more than twelve months | item			1
Collateralized Mortgage Obligations, Issued or Guaranteed by U.S. Government Agencies [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Total available for sale securities	155,465,000		$ 170,764,000
Gross unrealized gains	2,852,000		1,524,000
Gross unrealized losses	(397,000)		(1,554,000)
Available-for-sale Securities, Total	157,920,000		170,734,000
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	155,465,000
Total available for sale securities	155,465,000		170,764,000
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	157,920,000
Fair value	157,920,000		170,734,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	6,970,000		87,020,000
12 months or longer, Fair value	13,372,000		15,644,000
Total, Fair value	20,342,000		102,664,000
Less than 12 months, Gross unrealized losses	(82,000)		(994,000)
12 months or longer, Gross unrealized losses	(315,000)		(560,000)
Total, Gross unrealized losses	$ (397,000)		$ (1,554,000)
Less than 12 months, number of positions | item	3		31
12 months or longer, number of positions | item	4		5
Total , number of positions | item	7		36
Fair value	$ 157,920,000		$ 170,734,000
Number of securities in an unrealized loss position for less than twelve months | item	3		31
Number of securities in an unrealized loss position for more than twelve months | item	4		5
Mortgage-backed Securities, Issued by Private Enterprises [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Total available for sale securities	$ 2,449,000		$ 2,729,000
Gross unrealized gains	29,000		1,000
Gross unrealized losses	0		(26,000)
Available-for-sale Securities, Total	2,478,000		2,704,000
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	2,449,000
Total available for sale securities	2,449,000		2,729,000
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	2,478,000
Fair value	2,478,000		2,704,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value			1,101,000
12 months or longer, Fair value			0
Total, Fair value			1,101,000
Less than 12 months, Gross unrealized losses			(26,000)
12 months or longer, Gross unrealized losses			0
Total, Gross unrealized losses			$ (26,000)
Less than 12 months, number of positions | item			1
12 months or longer, number of positions | item			0
Total , number of positions | item			1
Fair value	2,478,000		$ 2,704,000
Number of securities in an unrealized loss position for less than twelve months | item			1
Number of securities in an unrealized loss position for more than twelve months | item			0
Corporate Bonds [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Total available for sale securities	1,500,000		$ 1,500,000
Gross unrealized gains	58,000		56,000
Gross unrealized losses	0		0
Available-for-sale Securities, Total	1,558,000		1,556,000
Amortized cost [Abstract]
Total available for sale securities	1,500,000		1,500,000
Fair value [Abstract]
Fair value	1,558,000		1,556,000
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	1,558,000		1,556,000
Municipal Bonds [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Total available for sale securities	9,895,000		9,910,000
Gross unrealized gains	172,000		73,000
Gross unrealized losses	(66,000)		(52,000)
Available-for-sale Securities, Total	10,001,000		9,931,000
Amortized cost [Abstract]
Total available for sale securities	9,895,000		9,910,000
Fair value [Abstract]
Fair value	10,001,000		9,931,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	3,521,000		6,080,000
12 months or longer, Fair value	991,000		0
Total, Fair value	4,512,000		6,080,000
Less than 12 months, Gross unrealized losses	(63,000)		(52,000)
12 months or longer, Gross unrealized losses	(3,000)		0
Total, Gross unrealized losses	$ (66,000)		$ (52,000)
Less than 12 months, number of positions | item	4		7
12 months or longer, number of positions | item	1		0
Total , number of positions | item	5		7
Fair value	$ 10,001,000		$ 9,931,000
Number of securities in an unrealized loss position for less than twelve months | item	4		7
Number of securities in an unrealized loss position for more than twelve months | item	1		0
Private Equity Funds [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Total available for sale securities	$ 2,117,000		$ 2,050,000
Gross unrealized gains	314,000		445,000
Gross unrealized losses	0		0
Available-for-sale Securities, Total	2,431,000		2,495,000
Amortized cost [Abstract]
Total available for sale securities	2,117,000		2,050,000
Fair value [Abstract]
Fair value	2,431,000		2,495,000
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	2,431,000		2,495,000
Common Stock [Member]
Carrying value and fair value of available for sale investment securities [Abstract]
Total available for sale securities	26,000		26,000
Gross unrealized gains	0		0
Gross unrealized losses	0		0
Available-for-sale Securities, Total	26,000		26,000
Amortized cost [Abstract]
Total available for sale securities	26,000		26,000
Fair value [Abstract]
Fair value	26,000		26,000
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	$ 26,000		$ 26,000