Loans and Allowance - Narrative (Details) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2020 USD ($)	Sep. 30, 2019 USD ($)	Sep. 30, 2020 USD ($) loan	Sep. 30, 2019 USD ($)	Jun. 30, 2020 USD ($)	Dec. 31, 2019 USD ($)	Jun. 30, 2019 USD ($)	Dec. 31, 2018 USD ($)
Receivables [Abstract]
Loans held for sale	$ 3,183		$ 3,183			$ 9,037
Accounts, Notes, Loans and Financing Receivable [Line Items]
Number of funded loans | loan			5,200
Loans	9,243,833		$ 9,243,833			8,459,310
Loans - allowance for loan losses	126,726	$ 80,571	126,726	$ 80,571	$ 121,119	80,284	$ 81,274	$ 80,552
Increase in past due loans			14,700
60-89 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Past Due	11,777		11,777			1,195
30-59 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Past Due	17,527		17,527			14,683
PPP, CARES Act
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans net of deferred fees	901,400		901,400
Deferred fees	21,500		21,500
University logo apparel sports
Accounts, Notes, Loans and Financing Receivable [Line Items]
Increase in non-accrual loans			2,800
Residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	1,673,864		1,673,864			1,732,201
Loans - allowance for loan losses	$ 22,047	$ 14,498	$ 22,047	$ 14,498	$ 21,078	14,569	$ 14,471	$ 14,322
Percentage of troubled debt restructured loans	88.00%	100.00%	63.00%	100.00%
Mortgage loans with formal foreclosure proceedings	$ 5,059		$ 5,059			1,033
Residential | 60-89 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Past Due	1,000		1,000			367
Residential | 30-59 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Past Due	2,743		2,743			4,290
Commercial and farmland | 60-89 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Increase in past due loans			9,900
Past Due	9,978		9,978			49
Commercial and farmland | 30-59 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Past Due	964		964			2,324
Commercial and industrial loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Increase in non-accrual loans			4,700
Commercial and industrial loans | 60-89 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Past Due	0		0			136
Commercial and industrial loans | 30-59 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Past Due	4,529		4,529			3,039
Construction | 60-89 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Past Due	0		0			0
Construction | 30-59 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Increase in past due loans			5,900
Past Due	7,575		7,575			1,630
Loans acquired with deteriorated credit quality
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	7,457		7,457			13,309
Loans acquired with deteriorated credit quality | Commercial and consumer loan
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	3,738		3,738			2,819
Loans - allowance for loan losses	116		116			124
Loans acquired with deteriorated credit quality | Residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	0		0			$ 1,715
Three relationships
Accounts, Notes, Loans and Financing Receivable [Line Items]
Increase in non-accrual loans			40,800
Three relationships | Senior living sector
Accounts, Notes, Loans and Financing Receivable [Line Items]
Increase in non-accrual loans			31,200
Three relationships | Commercial and farmland
Accounts, Notes, Loans and Financing Receivable [Line Items]
Increase in non-accrual loans			36,000
Three relationships | Self-storage
Accounts, Notes, Loans and Financing Receivable [Line Items]
Increase in non-accrual loans			3,400
One relationship | Construction | 30-59 Days Past Due
Accounts, Notes, Loans and Financing Receivable [Line Items]
Past Due	$ 7,500		$ 7,500