Loans and Asset Quality - Total Loans Held for Investment by Category and Loans Held for Sale (Details) - USD ($) $ in Thousands	Sep. 30, 2022	Dec. 31, 2021
Financing Receivable, Allowance for Credit Loss [Line Items]
Total loans HFI	$ 1,879,669	$ 1,683,832
Total loans HFS	1,536	4,290
Real estate | Commercial real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Total loans HFI	787,464	670,293
Real estate | One-to-four family residential
Financing Receivable, Allowance for Credit Loss [Line Items]
Total loans HFI	532,034	474,420
Real estate | Construction and development
Financing Receivable, Allowance for Credit Loss [Line Items]
Total loans HFI	140,398	106,339
Commercial and industrial
Financing Receivable, Allowance for Credit Loss [Line Items]
Total loans HFI	307,159	311,373
SBA PPP, net of deferred income
Financing Receivable, Allowance for Credit Loss [Line Items]
Total loans HFI	1,350	17,550
Tax-exempt
Financing Receivable, Allowance for Credit Loss [Line Items]
Total loans HFI	84,947	80,726
Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Total loans HFI	$ 26,317	$ 23,131