Premises and Equipment - Depreciation (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2017	Dec. 31, 2016	Dec. 31, 2015
Property, Plant and Equipment [Line Items]
Depreciation, Depletion and Amortization, Nonproduction	$ 3,761	$ 4,426	$ 3,614
Equipment [Member]
Property, Plant and Equipment [Line Items]
Depreciation	1,874	1,942	1,620
Building and Building Improvements [Member]
Property, Plant and Equipment [Line Items]
Depreciation	1,643	1,757	1,140
Software Development [Member]
Property, Plant and Equipment [Line Items]
Depreciation	244	275	337
SBM Financial, Inc. [Member] | Property, Plant and Equipment [Member]
Property, Plant and Equipment [Line Items]
Depreciation	$ 0	$ 452	$ 517