Loans - Recorded Investment in Residential, Consumer and Credit Card Loans Based on Payment Activity (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	$ 3,005,642	$ 2,784,562
Residential real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	805,413	814,030
Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	93,409	124,785
Credit cards
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	6,760	7,569
Performing | Residential real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	801,106	809,247
Performing | Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	92,924	123,950
Performing | Credit cards
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	6,748	7,567
Nonperforming | Residential real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	4,307	4,783
Nonperforming | Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	485	835
Nonperforming | Credit cards
Financing Receivable, Credit Quality Indicator [Line Items]
Financing receivable, net	$ 12	$ 2