Investment Securities - Gross Realized Gains and Losses - (Details) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2017 USD ($) item	Sep. 30, 2016 USD ($) security	Sep. 30, 2017 USD ($) item	Sep. 30, 2016 USD ($) security	Dec. 31, 2016 USD ($) item
Gross realized gains and losses on the sale of securities recognized in earnings [Abstract]
Gross realized gains	$ 32	$ 280	$ 325	$ 1,426
Gross realized losses	0	0	(17)	(44)
Net realized gains	32	280	308	1,382
Other than Temporary Impairment Losses, Investments, Portion Recognized in Earnings, Net, Available-for-sale Securities	0	$ 35	0	181
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	56,968		56,968		$ 84,780
12 months or longer, Fair value	27,119		27,119		14,359
Total, Fair value	84,087		84,087		99,139
Less than 12 months, Gross unrealized losses	(363)		(363)		(1,800)
12 months or longer, Gross unrealized losses	(619)		(619)		(529)
Total, Gross unrealized losses	$ (982)		$ (982)		$ (2,329)
Less than 12 months, number of positions | item	24		24		40
12 months or longer, number of positions | item	13		13		7
Total , number of positions | item	37		37		47
Fair value	$ 155,508		$ 155,508		$ 169,854
Number of securities in an unrealized loss position for less than twelve months | item	24		24		40
Number of securities in an unrealized loss position for more than twelve months | item	13		13		7
Credit related impairment losses on debt securities held for which a portion of OTTI was recognized in other comprehensive income [Roll Forward]
Reductions for securities sold during the period (realized)			$ 0	0
U.S. Government Agencies [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	$ 1,133		1,133		$ 966
12 months or longer, Fair value	0		0		0
Total, Fair value	1,133		1,133		966
Less than 12 months, Gross unrealized losses	(5)		(5)		(22)
12 months or longer, Gross unrealized losses	0		0		0
Total, Gross unrealized losses	$ (5)		$ (5)		$ (22)
Less than 12 months, number of positions | item	1		1		1
12 months or longer, number of positions | item	0		0		0
Total , number of positions | item	1		1		1
Fair value	$ 2,150		$ 2,150		$ 966
Number of securities in an unrealized loss position for less than twelve months | item	1		1		1
Number of securities in an unrealized loss position for more than twelve months | item	0		0		0
Mortgage-backed Securities-residential [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	$ 3,065		$ 3,065		$ 4,237
12 months or longer, Fair value	0		0		0
Total, Fair value	3,065		3,065		4,237
Less than 12 months, Gross unrealized losses	(15)		(15)		(17)
12 months or longer, Gross unrealized losses	0		0		0
Total, Gross unrealized losses	$ (15)		$ (15)		$ (17)
Less than 12 months, number of positions | item	1		1		2
12 months or longer, number of positions | item	0		0		0
Total , number of positions | item	1		1		2
Fair value	$ 15,850		$ 15,850		$ 7,038
Number of securities in an unrealized loss position for less than twelve months | item	1		1		2
Number of securities in an unrealized loss position for more than twelve months | item	0		0		0
Collateralized Mortgage Obligations, Issued or Guaranteed by U.S. Government Agencies [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	$ 52,109		$ 52,109		$ 73,864
12 months or longer, Fair value	24,974		24,974		12,045
Total, Fair value	77,083		77,083		85,909
Less than 12 months, Gross unrealized losses	(342)		(342)		(1,630)
12 months or longer, Gross unrealized losses	(578)		(578)		(499)
Total, Gross unrealized losses	$ (920)		$ (920)		$ (2,129)
Less than 12 months, number of positions | item	20		20		31
12 months or longer, number of positions | item	10		10		4
Total , number of positions | item	30		30		35
Fair value	$ 126,522		$ 126,522		$ 145,559
Number of securities in an unrealized loss position for less than twelve months | item	20		20		31
Number of securities in an unrealized loss position for more than twelve months | item	10		10		4
Mortgage-backed Securities, Issued by Private Enterprises [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value					$ 4,035
12 months or longer, Fair value					0
Total, Fair value					4,035
Less than 12 months, Gross unrealized losses					(86)
12 months or longer, Gross unrealized losses					0
Total, Gross unrealized losses					$ (86)
Less than 12 months, number of positions | item					3
12 months or longer, number of positions | item					0
Total , number of positions | item					3
Fair value					$ 4,035
Number of securities in an unrealized loss position for less than twelve months | item					3
Number of securities in an unrealized loss position for more than twelve months | item					0
Corporate Bonds [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	$ 1,713		$ 1,713		$ 1,701
Municipal Bonds [Member]
Available for sale securities, continuous unrealized loss position [Abstract]
Less than 12 months, Fair value	661		661		1,678
12 months or longer, Fair value	2,145		2,145		2,314
Total, Fair value	2,806		2,806		3,992
Less than 12 months, Gross unrealized losses	(1)		(1)		(45)
12 months or longer, Gross unrealized losses	(41)		(41)		(30)
Total, Gross unrealized losses	$ (42)		$ (42)		$ (75)
Less than 12 months, number of positions | item	2		2		3
12 months or longer, number of positions | item	3		3		3
Total , number of positions | item	5		5		6
Fair value	$ 7,541		$ 7,541		$ 8,708
Number of securities in an unrealized loss position for less than twelve months | item	2		2		3
Number of securities in an unrealized loss position for more than twelve months | item	3		3		3
Private Equity Funds [Member]
Gross realized gains and losses on the sale of securities recognized in earnings [Abstract]
Other than Temporary Impairment Losses, Investments, Portion Recognized in Earnings, Net, Available-for-sale Securities				$ 181
Available for sale securities, continuous unrealized loss position [Abstract]
12 months or longer, number of positions | security		2		2
Fair value	$ 1,706		$ 1,706		$ 1,821
Number of securities in an unrealized loss position for more than twelve months | security		2		2
Common Stock
Available for sale securities, continuous unrealized loss position [Abstract]
Fair value	$ 26		$ 26		$ 26