Investment Securities - Gross Realized Gains and Losses - (Details)	12 Months Ended
	Dec. 31, 2016 USD ($) security	Dec. 31, 2015 USD ($) security
Gross realized gains and losses on the sale of securities recognized in earnings [Abstract]
Gross realized gains	$ 1,475,000	$ 925,000
Gross realized losses	(44,000)	(25,000)
Net realized gains	1,431,000	900,000
Other than Temporary Impairment Losses, Investments, Portion Recognized in Earnings, Net, Available-for-sale Securities	190,000	14,000
Investment securities pledged as collateral	74,800,000
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	84,780,000	106,022,000
12 months or longer, Fair value	14,359,000	37,100,000
Total, Fair value	99,139,000	143,122,000
Less than 12 months, Gross unrealized losses	(1,800,000)	(1,134,000)
12 months or longer, Gross unrealized losses	(529,000)	(1,112,000)
Total, Gross unrealized losses	$ (2,329,000)	$ (2,246,000)
Less than 12 months, number of positions | security	40	43
12 months or longer, number of positions | security	7	12
Total , number of positions | security	47	55
Fair value	$ 169,854,000	$ 224,067,000
Number of securities in an unrealized loss position for less than twelve months | security	40	43
Number of securities in an unrealized loss position for more than twelve months | security	7	12
Credit related impairment losses on debt securities held for which a portion of OTTI was recognized in other comprehensive income [Roll Forward]
Reductions for securities sold during the period (realized)	$ 0
U.S. Government Agencies [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	966,000	$ 6,681,000
12 months or longer, Fair value		18,882,000
Total, Fair value	966,000	25,563,000
Less than 12 months, Gross unrealized losses	(22,000)	(57,000)
12 months or longer, Gross unrealized losses		(507,000)
Total, Gross unrealized losses	$ (22,000)	$ (564,000)
Less than 12 months, number of positions | security	1	2
12 months or longer, number of positions | security		6
Total , number of positions | security	1	8
Fair value	$ 966,000	$ 25,563,000
Number of securities in an unrealized loss position for less than twelve months | security	1	2
Number of securities in an unrealized loss position for more than twelve months | security		6
Mortgage-backed Securities-residential [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	$ 4,237,000	$ 5,140,000
12 months or longer, Fair value		2,574,000
Total, Fair value	4,237,000	7,714,000
Less than 12 months, Gross unrealized losses	(17,000)	(5,000)
12 months or longer, Gross unrealized losses		(45,000)
Total, Gross unrealized losses	$ (17,000)	$ (50,000)
Less than 12 months, number of positions | security	2	2
12 months or longer, number of positions | security		1
Total , number of positions | security	2	3
Fair value	$ 7,038,000	$ 11,058,000
Number of securities in an unrealized loss position for less than twelve months | security	2	2
Number of securities in an unrealized loss position for more than twelve months | security		1
Collateralized Mortgage Obligations, Issued or Guaranteed by U.S. Government Agencies [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	$ 73,864,000	$ 87,020,000
12 months or longer, Fair value	12,045,000	15,644,000
Total, Fair value	85,909,000	102,664,000
Less than 12 months, Gross unrealized losses	(1,630,000)	(994,000)
12 months or longer, Gross unrealized losses	(499,000)	(560,000)
Total, Gross unrealized losses	$ (2,129,000)	$ (1,554,000)
Less than 12 months, number of positions | security	31	31
12 months or longer, number of positions | security	4	5
Total , number of positions | security	35	36
Fair value	$ 145,559,000	$ 170,734,000
Number of securities in an unrealized loss position for less than twelve months | security	31	31
Number of securities in an unrealized loss position for more than twelve months | security	4	5
Mortgage-backed Securities, Issued by Private Enterprises [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	$ 4,035,000	$ 1,101,000
Total, Fair value	4,035,000	1,101,000
Less than 12 months, Gross unrealized losses	(86,000)	(26,000)
Total, Gross unrealized losses	$ (86,000)	$ (26,000)
Less than 12 months, number of positions | security	3	1
Total , number of positions | security	3	1
Fair value	$ 4,035,000	$ 2,704,000
Number of securities in an unrealized loss position for less than twelve months | security	3	1
Corporate Bonds [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	$ 1,701,000	$ 1,556,000
Municipal Bonds [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	1,678,000	6,080,000
12 months or longer, Fair value	2,314,000
Total, Fair value	3,992,000	6,080,000
Less than 12 months, Gross unrealized losses	(45,000)	(52,000)
12 months or longer, Gross unrealized losses	(30,000)
Total, Gross unrealized losses	$ (75,000)	$ (52,000)
Less than 12 months, number of positions | security	3	7
12 months or longer, number of positions | security	3
Total , number of positions | security	6	7
Fair value	$ 8,708,000	$ 9,931,000
Number of securities in an unrealized loss position for less than twelve months | security	3	7
Number of securities in an unrealized loss position for more than twelve months | security	3
Private Equity Funds [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	$ 1,821,000	$ 2,495,000
Number of securities invested in real estate funds | security	5
Common Stock [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	$ 26,000	$ 26,000