Income Taxes - Components of Deferred Tax Assets and Liabilities (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2018	Dec. 31, 2017	Dec. 31, 2016	Dec. 31, 2015
Effective Income Tax Rate Reconciliation, at Federal Statutory Income Tax Rate, Percent		35.00%	35.00%	35.00%
Investment in qualified affordable housing tax credits amortization		$ 10,278	$ 7,300	$ 6,664
Current Federal Tax Expense (Benefit)		1,200
Deferred tax assets:
Allowance for loan losses		10,498	17,719
Accumulated other comprehensive loss – Pension plan		7,098	7,937
Accumulated other comprehensive loss – Unrealized losses on securities		946	1,618
Deferred Tax Assets, Tax Deferred Expense, Compensation and Benefits, Employee Compensation		2,744	4,140
Deferred compensation		1,327	2,322
Net deferred loan fees		1,115	1,397
Deferred contract bonus		645	1,074
Nonvested equity-based compensation		1,069	1,115
Fixed assets		877	781
Deferred Tax Assets, Tax Deferred Expense, Reserves and Accruals, Legal Settlements		4	793
Other		1,226	2,525
Total deferred tax assets		27,549	41,421
Deferred tax liabilities:
Deferred investment income		6,120	10,199
Pension plan		19,076	26,344
Mortgage servicing rights		2,034	3,243
Partnership adjustments		460	549
Other		364	596
Total deferred tax liabilities		28,054	40,931
Net deferred tax (liability) asset		(505)
Deferred Tax Assets, Net			$ 490
Change in Accounting Method Accounted for as Change in Estimate [Member]
Investment in qualified affordable housing tax credits amortization		$ 3,100
Subsequent Event [Member]
Effective Income Tax Rate Reconciliation, at Federal Statutory Income Tax Rate, Percent	21.00%