Investment Securities - By Contractual Maturity (Details) - USD ($) $ in Thousands	Mar. 31, 2016	Dec. 31, 2015	Mar. 31, 2015
Amortized Cost
One Year or Less	$ 1,422
More than One Year to Five Years	27,576
More than Five Years to Ten Years	72,213
More than Ten Years	175,607
Total	276,818
Fair Value
One Year or Less	1,423
More than One Year to Five Years	27,886
More than Five Years to Ten Years	73,594
More than Ten Years	176,420
Total	279,323
Amortized Cost
One Year or Less	1,422
More than One Year to Five Years	27,576
More than Five Years to Ten Years	72,213
More than Ten Years	166,017
Total	267,228	$ 279,711	$ 278,278
Fair Value
One Year or Less	1,423
More than One Year to Five Years	27,886
More than Five Years to Ten Years	73,594
More than Ten Years	166,808
Total	269,711	280,273	280,461
Amortized Cost
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	0
More than Ten Years	9,590
Amortized Cost	9,590	9,642	0
Fair Value
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	0
More than Ten Years	9,612
Total	9,612	9,572
Municipal bonds
Amortized Cost
One Year or Less	1,422
More than One Year to Five Years	27,576
More than Five Years to Ten Years	43,199
More than Ten Years	51,287
Total	123,484	128,546	103,997
Fair Value
One Year or Less	1,423
More than One Year to Five Years	27,886
More than Five Years to Ten Years	44,398
More than Ten Years	52,175
Total	125,882	130,245	105,523
Collateralized mortgage obligation
Amortized Cost
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	0
More than Ten Years	23,805
Total	23,805	24,722	11,204
Fair Value
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	0
More than Ten Years	23,866
Total	23,866	24,543	11,288
Mortgage-backed securities
Amortized Cost
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	29,014
More than Ten Years	90,925
Total	119,939	126,443	163,077
Fair Value
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	29,196
More than Ten Years	90,767
Total	119,963	125,485	$ 163,650
Amortized Cost
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	0
More than Ten Years	8,361
Amortized Cost	8,361	8,400
Fair Value
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	0
More than Ten Years	8,383
Total	8,383	$ 8,330
Other
Amortized Cost
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	0
More than Ten Years	1,229
Amortized Cost	1,229
Fair Value
One Year or Less	0
More than One Year to Five Years	0
More than Five Years to Ten Years	0
More than Ten Years	1,229
Total	$ 1,229