Loans - Recorded Investment in Residential, Consumer and Credit Card Loans Based on Payment Activity (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	$ 2,828,745	$ 2,784,562
Residential real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	814,657	814,030
Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	94,425	124,785
Credit cards
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	8,683	7,569
Performing | Residential real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	809,994	809,247
Performing | Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	93,615	123,950
Performing | Credit cards
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	8,666	7,567
Nonperforming | Residential real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	4,663	4,783
Nonperforming | Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	810	835
Nonperforming | Credit cards
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	$ 17	$ 2