Bank segment - Loans receivable (Details) - USD ($) $ in Thousands	Mar. 31, 2021	Dec. 31, 2020	Mar. 31, 2020	Dec. 31, 2019
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	$ 5,325,770	$ 5,345,226
Deferred fees and discounts	(15,689)	(11,383)
Allowance for credit losses	(91,793)	(101,201)	$ (77,084)	$ (53,355)
Total loans, net	5,218,288	5,232,642
Real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	4,167,255	4,239,745
Real estate | Residential 1-4 family
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	2,107,537	2,144,239
Allowance for credit losses	(5,261)	(4,600)	(4,476)	(2,380)
Real estate | Commercial real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	1,012,968	983,865
Allowance for credit losses	(34,345)	(35,607)	(16,587)	(15,053)
Real estate | Home equity line of credit
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	901,462	963,578
Allowance for credit losses	(5,901)	(6,813)	(6,225)	(6,922)
Real estate | Residential land
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	17,468	15,617
Allowance for credit losses	(573)	(609)	(352)	(449)
Real estate | Commercial construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	114,455	121,424
Allowance for credit losses	(1,453)	(4,149)	(3,446)	(2,097)
Real estate | Residential construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	13,365	11,022
Allowance for credit losses	(16)	(11)	(14)	(3)
Commercial
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	1,010,004	936,748
Allowance for credit losses	(24,504)	(25,462)	(12,977)	(10,245)
Consumer
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans, gross	148,511	168,733
Allowance for credit losses	$ (19,740)	$ (23,950)	$ (33,007)	$ (16,206)