Bank segment - Loans receivable (Details) - USD ($) $ in Thousands	6 Months Ended
	Jun. 30, 2020	Mar. 31, 2020	Dec. 31, 2019	Jun. 30, 2019	Mar. 31, 2019	Dec. 31, 2018
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	$ 5,450,802		$ 5,120,664
Deferred fees and discounts	(12,985)		512
Allowance for credit losses	(81,307)	$ (77,084)	(53,355)	$ (58,425)	$ (54,297)	$ (52,119)
Total financing receivables	$ 5,356,510		5,067,821
Minimum benchmark percentage of loan to appraisal ratio which mortgage insurance is required	80.00%
Real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	$ 4,160,943		4,192,069
Real estate | Residential 1-4 family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	2,123,226		2,178,135
Allowance for credit losses	(3,911)	(4,476)	(2,380)	(2,015)	(1,911)	(1,976)
Real estate | Commercial real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	855,566		824,830
Allowance for credit losses	(21,100)	(16,587)	(15,053)	(15,811)	(14,825)	(14,505)
Total financing receivables			824,830
Real estate | Home equity line of credit
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	1,065,264		1,092,125
Allowance for credit losses	(6,214)	(6,225)	(6,922)	(6,881)	(6,493)	(6,371)
Real estate | Residential land
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	13,224		14,704
Allowance for credit losses	(356)	(352)	(449)	(537)	(425)	(479)
Real estate | Commercial construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	92,904		70,605
Allowance for credit losses	(4,757)	(3,446)	(2,097)	(2,046)	(2,843)	(2,790)
Total financing receivables			70,605
Real estate | Residential construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	10,759		11,670
Allowance for credit losses	(14)	(14)	(3)	(2)	(3)	(4)
Commercial
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	1,073,829		670,674
Allowance for credit losses	(13,868)	(12,977)	(10,245)	(13,073)	(10,814)	(9,225)
Total financing receivables			670,674
Consumer
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans receivable	216,030		257,921
Allowance for credit losses	$ (31,087)	$ (33,007)	$ (16,206)	$ (18,060)	$ (16,983)	$ (16,769)