Investment Securities - Gross Realized Gains and Losses - (Details) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2017 USD ($) item	Jun. 30, 2016 USD ($)	Jun. 30, 2017 USD ($) item	Jun. 30, 2016 USD ($)	Dec. 31, 2016 USD ($) item
Gross realized gains and losses on the sale of securities recognized in earnings [Abstract]
Gross realized gains	$ 293	$ 779	$ 293	$ 1,146
Gross realized losses	(17)	(44)	(17)	(44)
Net realized gains	276	735	276	1,102
Other than Temporary Impairment Losses, Investments, Portion Recognized in Earnings, Net, Available-for-sale Securities	0	$ 146	0	146
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	49,977		49,977		$ 84,780
12 months or longer, Fair value	17,798		17,798		14,359
Total, Fair value	67,775		67,775		99,139
Less than 12 months, Gross unrealized losses	(428)		(428)		(1,800)
12 months or longer, Gross unrealized losses	(473)		(473)		(529)
Total, Gross unrealized losses	$ (901)		$ (901)		$ (2,329)
Less than 12 months, number of positions | item	22		22		40
12 months or longer, number of positions | item	9		9		7
Total , number of positions | item	31		31		47
Fair value	$ 135,936		$ 135,936		$ 169,854
Number of securities in an unrealized loss position for less than twelve months | item	22		22		40
Number of securities in an unrealized loss position for more than twelve months | item	9		9		7
Credit related impairment losses on debt securities held for which a portion of OTTI was recognized in other comprehensive income [Roll Forward]
Reductions for securities sold during the period (realized)			$ 0	$ 0
U.S. Government Agencies [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	$ 2,122		2,122		$ 966
12 months or longer, Fair value	0		0		0
Total, Fair value	2,122		2,122		966
Less than 12 months, Gross unrealized losses	(5)		(5)		(22)
12 months or longer, Gross unrealized losses	0		0		0
Total, Gross unrealized losses	$ (5)		$ (5)		$ (22)
Less than 12 months, number of positions | item	2		2		1
12 months or longer, number of positions | item	0		0		0
Total , number of positions | item	2		2		1
Fair value	$ 2,122		$ 2,122		$ 966
Number of securities in an unrealized loss position for less than twelve months | item	2		2		1
Number of securities in an unrealized loss position for more than twelve months | item	0		0		0
Mortgage-backed Securities-residential [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value					$ 4,237
12 months or longer, Fair value					0
Total, Fair value					4,237
Less than 12 months, Gross unrealized losses					(17)
12 months or longer, Gross unrealized losses					0
Total, Gross unrealized losses					$ (17)
Less than 12 months, number of positions | item					2
12 months or longer, number of positions | item					0
Total , number of positions | item					2
Fair value	$ 7,207		$ 7,207		$ 7,038
Number of securities in an unrealized loss position for less than twelve months | item					2
Number of securities in an unrealized loss position for more than twelve months | item					0
Collateralized Mortgage Obligations, Issued or Guaranteed by U.S. Government Agencies [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	47,191		47,191		$ 73,864
12 months or longer, Fair value	15,647		15,647		12,045
Total, Fair value	62,838		62,838		85,909
Less than 12 months, Gross unrealized losses	(422)		(422)		(1,630)
12 months or longer, Gross unrealized losses	(432)		(432)		(499)
Total, Gross unrealized losses	$ (854)		$ (854)		$ (2,129)
Less than 12 months, number of positions | item	18		18		31
12 months or longer, number of positions | item	6		6		4
Total , number of positions | item	24		24		35
Fair value	$ 115,589		$ 115,589		$ 145,559
Number of securities in an unrealized loss position for less than twelve months | item	18		18		31
Number of securities in an unrealized loss position for more than twelve months | item	6		6		4
Mortgage-backed Securities, Issued by Private Enterprises [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value					$ 4,035
12 months or longer, Fair value					0
Total, Fair value					4,035
Less than 12 months, Gross unrealized losses					(86)
12 months or longer, Gross unrealized losses					0
Total, Gross unrealized losses					$ (86)
Less than 12 months, number of positions | item					3
12 months or longer, number of positions | item					0
Total , number of positions | item					3
Fair value					$ 4,035
Number of securities in an unrealized loss position for less than twelve months | item					3
Number of securities in an unrealized loss position for more than twelve months | item					0
Corporate Bonds [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	$ 1,728		$ 1,728		$ 1,701
Municipal Bonds [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	664		664		1,678
12 months or longer, Fair value	2,151		2,151		2,314
Total, Fair value	2,815		2,815		3,992
Less than 12 months, Gross unrealized losses	(1)		(1)		(45)
12 months or longer, Gross unrealized losses	(41)		(41)		(30)
Total, Gross unrealized losses	$ (42)		$ (42)		$ (75)
Less than 12 months, number of positions | item	2		2		3
12 months or longer, number of positions | item	3		3		3
Total , number of positions | item	5		5		6
Fair value	$ 7,558		$ 7,558		$ 8,708
Number of securities in an unrealized loss position for less than twelve months | item	2		2		3
Number of securities in an unrealized loss position for more than twelve months | item	3		3		3
Private Equity Funds [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	$ 1,706		$ 1,706		$ 1,821
Common Stock
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	$ 26		$ 26		$ 26