Income Taxes - Schedule of Net Deferred Tax liability (Details) - USD ($) $ in Thousands	Dec. 31, 2017	Dec. 31, 2016
Deferred tax assets:
Allowance for loan losses	$ 6,264	$ 8,516
Amortization of core deposit intangible	759	996
Compensation related	6,158	7,552
Unrealized loss on securities	988	2,462
Other	3,599	2,430
Subtotal	17,768	21,956
Deferred tax liabilities:
FHLB stock dividends	(550)	(827)
Depreciation	(4,115)	(6,548)
Mortgage servicing rights	(19,830)	(12,558)
Other	(5,131)	(6,203)
Subtotal	(29,626)	(26,136)
Net deferred tax liability	$ (11,858)	$ (4,180)