Segment Information - Schedule of Segment Reporting Information (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2019
Segment Reporting Information [Line Items]
Net Interest Income	$ 12,704	$ 10,357
Provision for loan losses	1,956	131
Noninterest Income	4,561	2,334
Noninterest Expenses	13,378	9,026
Income taxes	328	699
Segment Profit (Loss)	1,603	2,835
Segment Assets	1,510,048	1,515,313	$ 1,515,313
Bank
Segment Reporting Information [Line Items]
Net Interest Income	12,656	10,357
Provision for loan losses	1,956	131
Noninterest Income	3,049	2,334
Noninterest Expenses	11,667	9,026
Income taxes	368	699
Segment Profit (Loss)	1,714	2,835
Segment Assets	1,497,788	1,503,284
Mortgage Banking
Segment Reporting Information [Line Items]
Net Interest Income	34	0
Provision for loan losses	0	0
Noninterest Income	1,253	0
Noninterest Expenses	1,195	0
Income taxes	11	0
Segment Profit (Loss)	81	0
Segment Assets	11,082	11,624
Small Business Specialty Lending Division
Segment Reporting Information [Line Items]
Net Interest Income	14	0
Provision for loan losses	0	0
Noninterest Income	259	0
Noninterest Expenses	516	0
Income taxes	(51)	0
Segment Profit (Loss)	(192)	0
Segment Assets	$ 1,178	$ 405