Income Taxes - Significant Components of Deferred Tax Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2020	Dec. 31, 2019
Deferred tax assets
Federal net operating loss carryforwards	$ 4,067	$ 5,034
State net operating loss carryforwards	459	552
Acquired loan basis	(184)	450
Other real estate owned	64	44
AMT credit carryforward	0	714
Allowance for loan losses	24,317	6,828
Accrued liabilities	2,675	1,744
Lease liability	4,598	4,994
Other	2,181	1,925
Total deferred tax assets	38,177	22,285
Deferred tax liabilities
Goodwill and intangible assets	2,452	2,143
Fair value adjustment on junior subordinated debentures	2,449	2,564
Premises and equipment	10,767	6,142
Installment gain on sale of subsidiary	1,049	1,816
Lease right-of-use asset	4,172	4,815
Unrealized gain on securities available for sale	1,601	339
Derivative financial instruments	193	0
Indemnification asset	8,575	0
Other	214	376
Total deferred tax liabilities	31,472	18,195
Net deferred tax asset before valuation allowance	6,705	4,090
Valuation allowance	(278)	(278)
Net deferred tax asset	$ 6,427	$ 3,812