Loans, Impaired Loans Recorded Investment, Unpaid Principal Balance, Related Allowance, Average Recorded Investment, and Interest Income Recognized (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2019 USD ($)
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	$ 110,043
Unpaid Principal Balance	148,020
Related Allowance	16,165
Average Recorded Investment	113,697
Interest Income Recognized	$ 3,628
Net Recorded Investment of the Impaired Loans	63.00%
Bank Mutual
Loan and Lease Receivables, Impaired [Abstract]
Purchased credit-impaired loans	$ 2,000
Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	76,102
Unpaid Principal Balance	95,033
Related Allowance	16,165
Average Recorded Investment	77,235
Interest Income Recognized	3,012
Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	33,941
Unpaid Principal Balance	52,987
Related Allowance	0
Average Recorded Investment	36,462
Interest Income Recognized	616
Commercial and industrial
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	62,035
Unpaid Principal Balance	96,784
Related Allowance	12,010
Average Recorded Investment	65,792
Interest Income Recognized	1,895
Commercial and industrial | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	47,249
Unpaid Principal Balance	63,346
Related Allowance	12,010
Average Recorded Investment	45,290
Interest Income Recognized	1,832
Commercial and industrial | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	14,787
Unpaid Principal Balance	33,438
Related Allowance	0
Average Recorded Investment	20,502
Interest Income Recognized	63
Commercial real estate — owner occupied
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	1,676
Unpaid Principal Balance	1,682
Related Allowance	19
Average Recorded Investment	1,774
Interest Income Recognized	88
Commercial real estate — owner occupied | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	1,676
Unpaid Principal Balance	1,682
Related Allowance	19
Average Recorded Investment	1,774
Interest Income Recognized	88
Commercial real estate — owner occupied | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	0
Unpaid Principal Balance	0
Related Allowance	0
Average Recorded Investment	0
Interest Income Recognized	0
Commercial and business lending
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	63,711
Unpaid Principal Balance	98,466
Related Allowance	12,029
Average Recorded Investment	67,566
Interest Income Recognized	1,982
Commercial and business lending | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	48,924
Unpaid Principal Balance	65,028
Related Allowance	12,029
Average Recorded Investment	47,064
Interest Income Recognized	1,919
Commercial and business lending | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	14,787
Unpaid Principal Balance	33,438
Related Allowance	0
Average Recorded Investment	20,502
Interest Income Recognized	63
Commercial real estate — investor
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	4,633
Unpaid Principal Balance	5,808
Related Allowance	15
Average Recorded Investment	4,931
Interest Income Recognized	174
Commercial real estate — investor | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	928
Unpaid Principal Balance	2,104
Related Allowance	15
Average Recorded Investment	950
Interest Income Recognized	15
Commercial real estate — investor | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	3,705
Unpaid Principal Balance	3,705
Related Allowance	0
Average Recorded Investment	3,980
Interest Income Recognized	159
Real estate construction
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	477
Unpaid Principal Balance	559
Related Allowance	67
Average Recorded Investment	494
Interest Income Recognized	30
Real estate construction | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	477
Unpaid Principal Balance	559
Related Allowance	67
Average Recorded Investment	494
Interest Income Recognized	30
Real estate construction | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	0
Unpaid Principal Balance	0
Related Allowance	0
Average Recorded Investment	0
Interest Income Recognized	0
Commercial real estate lending
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	5,110
Unpaid Principal Balance	6,367
Related Allowance	82
Average Recorded Investment	5,425
Interest Income Recognized	204
Commercial real estate lending | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	1,405
Unpaid Principal Balance	2,663
Related Allowance	82
Average Recorded Investment	1,445
Interest Income Recognized	45
Commercial real estate lending | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	3,705
Unpaid Principal Balance	3,705
Related Allowance	0
Average Recorded Investment	3,980
Interest Income Recognized	159
Total commercial
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	68,820
Unpaid Principal Balance	104,833
Related Allowance	12,111
Average Recorded Investment	72,991
Interest Income Recognized	2,186
Total commercial | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	50,329
Unpaid Principal Balance	67,691
Related Allowance	12,111
Average Recorded Investment	48,509
Interest Income Recognized	1,965
Total commercial | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	18,491
Unpaid Principal Balance	37,142
Related Allowance	0
Average Recorded Investment	24,482
Interest Income Recognized	222
Residential mortgage
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	35,554
Unpaid Principal Balance	37,087
Related Allowance	2,740
Average Recorded Investment	34,683
Interest Income Recognized	1,229
Residential mortgage | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	21,450
Unpaid Principal Balance	22,625
Related Allowance	2,740
Average Recorded Investment	23,721
Interest Income Recognized	856
Residential mortgage | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	14,104
Unpaid Principal Balance	14,461
Related Allowance	0
Average Recorded Investment	10,962
Interest Income Recognized	373
Home equity
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	4,422
Unpaid Principal Balance	4,851
Related Allowance	1,190
Average Recorded Investment	4,773
Interest Income Recognized	211
Home equity | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	3,076
Unpaid Principal Balance	3,468
Related Allowance	1,190
Average Recorded Investment	3,756
Interest Income Recognized	191
Home equity | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	1,346
Unpaid Principal Balance	1,383
Related Allowance	0
Average Recorded Investment	1,017
Interest Income Recognized	21
Other consumer
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	1,247
Unpaid Principal Balance	1,249
Related Allowance	125
Average Recorded Investment	1,250
Interest Income Recognized	1
Other consumer | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	1,247
Unpaid Principal Balance	1,249
Related Allowance	125
Average Recorded Investment	1,250
Interest Income Recognized	1
Other consumer | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	0
Unpaid Principal Balance	0
Related Allowance	0
Average Recorded Investment	0
Interest Income Recognized	0
Total consumer
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	41,223
Unpaid Principal Balance	43,187
Related Allowance	4,055
Average Recorded Investment	40,706
Interest Income Recognized	1,441
Total consumer | Loans with a related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	25,773
Unpaid Principal Balance	27,342
Related Allowance	4,055
Average Recorded Investment	28,726
Interest Income Recognized	1,047
Total consumer | Loans with no related allowance
Loan and Lease Receivables, Impaired [Abstract]
Recorded Investment	15,450
Unpaid Principal Balance	15,845
Related Allowance	0
Average Recorded Investment	11,979
Interest Income Recognized	$ 394