Loans and Allowance for Loan Losses - Allowance for Loan Losses (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2021	Sep. 30, 2020	Sep. 30, 2021	Sep. 30, 2020	Dec. 31, 2020
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	$ 24,889	$ 20,248	$ 26,510	$ 14,380
Charge-offs	(127)	(102)	(205)	(1,295)
Recoveries on loans previously charged off	98	74	636	384
Provision (reversal) for loan losses	(1,053)	4,326	(3,134)	11,077
Ending balance	23,807	24,546	23,807	24,546
Allowance for loan losses:
Individually evaluated for impairment	0		0		$ 5
Collectively evaluated for impairment	23,807		23,807		26,505
Total	23,807	24,546	23,807	24,546	26,510
Loans receivable:
Individually evaluated for impairment	1,669		1,669		3,731
Collectively evaluated for impairment	1,706,209		1,706,209		1,579,847
Total	1,707,878		1,707,878		1,583,578
Real Estate
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	18,672	15,545	18,894	9,854
Charge-offs	0	0	(18)	(138)
Recoveries on loans previously charged off	0	0	47	56
Provision (reversal) for loan losses	(733)	2,556	(984)	8,329
Ending balance	17,939	18,101	17,939	18,101
Allowance for loan losses:
Individually evaluated for impairment	0		0		3
Collectively evaluated for impairment	17,939		17,939		18,891
Total	17,939	18,101	17,939	18,101	18,894
Loans receivable:
Individually evaluated for impairment	1,344		1,344		2,200
Collectively evaluated for impairment	1,389,059		1,389,059		1,218,736
Total	1,390,403		1,390,403		1,220,936
Commercial and Industrial
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	5,866	4,337	7,222	4,354
Charge-offs	(84)	0	(100)	(916)
Recoveries on loans previously charged off	94	18	553	223
Provision (reversal) for loan losses	(336)	1,686	(2,135)	2,380
Ending balance	5,540	6,041	5,540	6,041
Allowance for loan losses:
Individually evaluated for impairment	0		0		2
Collectively evaluated for impairment	5,540		5,540		7,220
Total	5,540	6,041	5,540	6,041	7,222
Loans receivable:
Individually evaluated for impairment	325		325		1,531
Collectively evaluated for impairment	295,760		295,760		339,338
Total	296,085		296,085		340,869
Other Consumer
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Beginning balance	351	366	394	172
Charge-offs	(43)	(102)	(87)	(241)
Recoveries on loans previously charged off	4	56	36	105
Provision (reversal) for loan losses	16	84	(15)	368
Ending balance	328	404	328	404
Allowance for loan losses:
Individually evaluated for impairment	0		0		0
Collectively evaluated for impairment	328		328		394
Total	328	$ 404	328	$ 404	394
Loans receivable:
Individually evaluated for impairment	0		0		0
Collectively evaluated for impairment	21,390		21,390		21,773
Total	$ 21,390		$ 21,390		$ 21,773