Segment Reporting (Details) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2020 USD ($)	Jun. 30, 2019 USD ($)	Jun. 30, 2020 USD ($) State Segment	Jun. 30, 2019 USD ($)	Dec. 31, 2019 USD ($)
Segment Reporting [Abstract]
Number of reportable segments | Segment			2
Number of states products and services are provided | State			21
Number of states the company can lend | State			48
Net Income (Loss) [Abstract]
Net interest income (loss)	$ 13,249	$ 12,981	$ 25,775	$ 25,917
Provision for loan losses	4,500	30	5,285	(650)
Net interest income (loss) after provision for loan losses	8,749	12,951	20,490	26,567
Noninterest income	66,904	35,190	98,368	59,447
Noninterest expenses [Abstract]
Compensation, payroll taxes, and other employee benefits	36,889	27,074	61,290	47,713
Occupancy, office furniture and equipment	2,534	2,680	5,275	5,456
FDIC insurance premiums	1,208	879	2,284	1,684
Advertising	864	963	1,764	1,921
Data processing	1,095	869	2,101	1,638
Communications	317	353	655	681
Professional fees	1,077	789	2,909	1,484
Real estate owned	33	19	44	51
Loan processing expense	1,208	879	2,284	1,684
Other	3,672	1,729	6,575	4,076
Total noninterest expenses	47,689	35,355	82,897	64,704
Income before income taxes	27,964	12,786	35,961	21,310
Income tax expense	7,016	3,143	8,944	5,125
Net income	20,948	9,643	27,017	16,185
Total assets	2,217,573	2,015,138	2,217,573	2,015,138	$ 1,996,347
Holding Company and Other [Member]
Net Income (Loss) [Abstract]
Net interest income (loss)	59	(20)	56	(8)
Provision for loan losses	0	0	0	0
Net interest income (loss) after provision for loan losses	59	(20)	56	(8)
Noninterest income	(250)	(253)	(612)	(448)
Noninterest expenses [Abstract]
Compensation, payroll taxes, and other employee benefits	(156)	(176)	(310)	(353)
Occupancy, office furniture and equipment	0	0	0	0
Advertising	0	0	0	0
Data processing	4	4	10	8
Communications	0	0	0	0
Professional fees	1	9	15	10
Real estate owned	0	0	0	0
Loan processing expense	0	0	0	0
Other	(99)	(92)	(328)	(146)
Total noninterest expenses	(250)	(255)	(613)	(481)
Income before income taxes	59	(18)	57	25
Income tax expense	2	4	8	13
Net income	57	(22)	49	12
Total assets	(363,286)	(234,178)	(363,286)	(234,178)
Operating Segments [Member] | Community Banking [Member]
Net Income (Loss) [Abstract]
Net interest income (loss)	13,701	13,530	26,609	26,662
Provision for loan losses	4,325	0	5,075	(700)
Net interest income (loss) after provision for loan losses	9,376	13,530	21,534	27,362
Noninterest income	2,936	1,079	3,964	1,960
Noninterest expenses [Abstract]
Compensation, payroll taxes, and other employee benefits	4,906	4,671	10,074	9,427
Occupancy, office furniture and equipment	866	944	1,880	1,916
Advertising	297	220	545	401
Data processing	678	493	1,283	950
Communications	91	93	188	175
Professional fees	226	160	424	428
Real estate owned	33	19	44	51
Loan processing expense	0	0	0	0
Other	532	635	1,112	1,124
Total noninterest expenses	7,629	7,235	15,550	14,472
Income before income taxes	4,683	7,374	9,948	14,850
Income tax expense	574	1,594	1,728	3,281
Net income	4,109	5,780	8,220	11,569
Total assets	2,141,415	1,958,339	2,141,415	1,958,339
Operating Segments [Member] | Mortgage Banking [Member]
Net Income (Loss) [Abstract]
Net interest income (loss)	(511)	(529)	(890)	(737)
Provision for loan losses	175	30	210	50
Net interest income (loss) after provision for loan losses	(686)	(559)	(1,100)	(787)
Noninterest income	64,218	34,364	95,016	57,935
Noninterest expenses [Abstract]
Compensation, payroll taxes, and other employee benefits	32,139	22,579	51,526	38,639
Occupancy, office furniture and equipment	1,668	1,736	3,395	3,540
Advertising	567	743	1,219	1,520
Data processing	413	372	808	680
Communications	226	260	467	506
Professional fees	850	620	2,470	1,046
Real estate owned	0	0	0	0
Loan processing expense	1,208	879	2,284	1,684
Other	3,239	1,186	5,791	3,098
Total noninterest expenses	40,310	28,375	67,960	50,713
Income before income taxes	23,222	5,430	25,956	6,435
Income tax expense	6,440	1,545	7,208	1,831
Net income	16,782	3,885	18,748	4,604
Total assets	$ 439,444	$ 290,977	$ 439,444	$ 290,977