Income Taxes - Components of Deferred Tax Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2015	Dec. 31, 2014
Deferred tax assets:
Allowance for loan losses	$ 19,773	$ 19,023
Accumulated other comprehensive loss – Pension Plan	8,266	8,005
Deferred Tax Assets, Unrealized Losses on Available-for-Sale Securities, Gross	157	0
Deferred compensation	3,908	3,820
OREO valuation adjustments	2,418	3,984
Deferred Tax Assets, Deferred Income	1,204	933
Deferred Tax Assets, Tax Deferred Expense, Reserves and Accruals, Other	1,031	0
Other	4,171	4,338
Total deferred tax assets	40,928	40,103
Deferred tax liabilities:
Accumulated other comprehensive income – Unrealized gains on securities	0	677
Deferred investment income	10,199	10,199
Pension plan	26,205	25,949
Mortgage servicing rights	3,153	3,015
Deferred Tax Liabilities, Partnership Adjustments	560	865
Other	872	804
Total deferred tax liabilities	40,989	41,509
Net deferred tax asset (liability)	$ (61)	$ (1,406)