Income Taxes - Components of Deferred Tax Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015
Deferred tax assets:
Allowance for loan losses	$ 17,719	$ 19,773
Accumulated other comprehensive loss – Pension Plan	7,937	8,266
Accumulated other comprehensive loss – Unrealized losses on securities	1,618	157
Deferred compensation	4,140	3,908
OREO valuation adjustments	2,322	2,418
Net deferred loan fees	1,397	1,204
Deferred contract bonus	1,074	1,031
Nonvested PBRSUs	1,115	463
Fixed assets	781	413
Deferred Tax Assets, Tax Deferred Expense, Reserves and Accruals, Legal Settlements	793	482
Other	2,525	2,813
Total deferred tax assets	41,421	40,928
Deferred tax liabilities:
Deferred investment income	10,199	10,199
Pension plan	26,344	26,205
Mortgage servicing rights	3,243	3,153
Partnership adjustments	549	560
Other	596	872
Total deferred tax liabilities	40,931	40,989
Deferred Tax Assets, Net	$ 490
Net deferred tax asset (liability)		$ 61