Investment Securities - Contractual Maturity - (Details) - USD ($) $ in Thousands	6 Months Ended
	Jun. 30, 2017	Jun. 30, 2016	Dec. 31, 2016
Amortized cost [Abstract]
Within 1 year	$ 1,798
After 1 but within 5 years	7,490
After 5 but within 10 years	1,007
After 10 years	1,000
Total available for sale debt securities	133,856
Available-for-sale Equity Securities, Amortized Cost Basis	1,732
Total available for sale securities	135,588		$ 170,998
Fair value [Abstract]
Within 1 year	1,797
After 1 but within 5 years	7,544
After 5 but within 10 years	1,039
After 10 years	1,028
Total available for sale debt securities	134,204
No contractual maturity	1,732
Fair value	135,936		169,854
Proceeds from sales of AFS investment securities	19,492	$ 10,705
U.S. Government Agencies [Member]
Amortized cost [Abstract]
Total available for sale securities	2,127		988
Fair value [Abstract]
Fair value	2,122		966
Mortgage-backed Securities-residential [Member]
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	7,029
Total available for sale securities	7,029		6,985
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	7,207
Fair value	7,207		7,038
Collateralized Mortgage Obligations, Issued or Guaranteed by U.S. Government Agencies [Member]
Amortized cost [Abstract]
Available for sale securities Debt maturities, amortized cost basis	115,532
Total available for sale securities	115,532		146,666
Fair value [Abstract]
Available-for-sale Securities, Debt Maturities, without Single Maturity Date, Fair Value	115,589
Fair value	115,589		145,559
Mortgage-backed Securities, Issued by Private Enterprises [Member]
Amortized cost [Abstract]
Total available for sale securities			4,121
Fair value [Abstract]
Fair value			4,035
Corporate Bonds [Member]
Amortized cost [Abstract]
Total available for sale securities	1,700		1,700
Fair value [Abstract]
Fair value	1,728		1,701
Municipal Bonds [Member]
Amortized cost [Abstract]
Total available for sale securities	7,468		8,691
Fair value [Abstract]
Fair value	7,558		8,708
Private Equity Funds [Member]
Amortized cost [Abstract]
Total available for sale securities	1,706		1,821
Fair value [Abstract]
Fair value	1,706		1,821
Common Stock
Amortized cost [Abstract]
Total available for sale securities	26		26
Fair value [Abstract]
Fair value	$ 26		$ 26