Fair Value - Carrying Amount and Fair Values (Details) - USD ($) $ in Thousands	Mar. 31, 2021	Dec. 31, 2020
Carrying amount | Quoted Prices in Active Markets for Identical Assets (Level 1)
Financial assets:
Cash and cash equivalents	$ 239,577	$ 219,311
Carrying amount | Significant Other Observable Inputs (Level 2)
Financial assets:
Securities	34,551	47,571
SBA loans held for sale	8,809	9,335
Loans, net of allowance for loan losses	1,636,674	1,595,377
Federal Home Loan Bank stock	9,269	10,594
Accrued interest receivable	9,831	10,429
Financial liabilities:
Deposits	1,628,393	1,557,959
Borrowed funds and subordinated debentures	180,310	210
Accrued interest payable	255	248
Carrying amount | Significant Unobservable Inputs (Level 3)
Financial assets:
Servicing assets	1,738	1,857
Carrying amount | Significant Unobservable Inputs (Level 3) | Recurring | Corporate and other securities
Financial assets:
Securities	5,300
Estimated fair value | Quoted Prices in Active Markets for Identical Assets (Level 1)
Financial assets:
Cash and cash equivalents	239,577	219,311
Estimated fair value | Significant Other Observable Inputs (Level 2)
Financial assets:
Securities	34,551	47,571
SBA loans held for sale	10,392	10,712
Loans, net of allowance for loan losses	1,650,152	1,613,593
Federal Home Loan Bank stock	9,269	10,594
Accrued interest receivable	9,831	10,429
Financial liabilities:
Deposits	1,628,042	1,561,502
Borrowed funds and subordinated debentures	181,727	212,358
Accrued interest payable	255	248
Estimated fair value | Significant Unobservable Inputs (Level 3)
Financial assets:
Servicing assets	1,738	$ 1,857
Estimated fair value | Significant Unobservable Inputs (Level 3) | Recurring | Corporate and other securities
Financial assets:
Securities	$ 4,500