Loans, Allowance for Loan Losses and Credit Quality - Summary of Allowance for Loan Losses and Recorded Investment in Loans by Portfolio Segment and Based on Impairment Method (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019	Mar. 31, 2020	Dec. 31, 2019	Mar. 31, 2019
Allowance for Loan Losses
Beginning Balance	$ 8,679	$ 6,577
Provision (credit) for loan losses	1,316	540
Loan charge-offs	(124)	(118)
Loan recoveries	62	129
Ending Balance	9,933	7,128
Ending Balance, Individually evaluated for impairment			$ 334		$ 879
Ending Balance, Collectively evaluated for impairment			9,599		6,249
Total	8,679	7,128	9,933	$ 8,679	7,128
Loans, Individually evaluated for impairment			4,069		5,423
Loans, Collectively evaluated for impairment			937,376		728,053
Loans, Acquired loans with deteriorated credit quality			1,592		565
Total Loans			943,037	909,856	734,041
Home Equity Lines of Credit
Allowance for Loan Losses
Total Loans			46,507	47,410
Residential
Allowance for Loan Losses
Total Loans			242,021	237,218
Commercial
Allowance for Loan Losses
Total Loans			313,899	310,019
Construction and Land Development
Allowance for Loan Losses
Total Loans			124,301	113,526
Commercial Loans
Allowance for Loan Losses
Beginning Balance	1,910	1,375
Provision (credit) for loan losses	(208)	177
Loan charge-offs	(62)	(75)
Loan recoveries	35	28
Ending Balance	1,675	1,505
Ending Balance, Individually evaluated for impairment			300		311
Ending Balance, Collectively evaluated for impairment			1,375		1,194
Total	1,675	1,505	1,675	1,910	1,505
Loans, Individually evaluated for impairment			407		427
Loans, Collectively evaluated for impairment			176,020		131,295
Loans, Acquired loans with deteriorated credit quality			77		62
Total Loans			176,504	161,286	131,784
Consumer Loans
Allowance for Loan Losses
Beginning Balance	425	326
Provision (credit) for loan losses	(74)	11
Loan charge-offs	(62)	(41)
Loan recoveries	16	5
Ending Balance	305	301
Ending Balance, Individually evaluated for impairment					12
Ending Balance, Collectively evaluated for impairment			305		289
Total	305	301	305	425	301
Loans, Individually evaluated for impairment					51
Loans, Collectively evaluated for impairment			39,775		32,490
Loans, Acquired loans with deteriorated credit quality			30		69
Total Loans			39,805	40,397	32,610
Real Estate Mortgage Loans
Allowance for Loan Losses
Total Loans			680,221	660,763
Real Estate Mortgage Loans | Home Equity Lines of Credit
Allowance for Loan Losses
Beginning Balance	344	394
Provision (credit) for loan losses	(125)	24
Loan recoveries	1
Ending Balance	220	418
Ending Balance, Collectively evaluated for impairment			220		418
Total	220	418	220	344	418
Loans, Individually evaluated for impairment			314		325
Loans, Collectively evaluated for impairment			46,193		41,339
Total Loans			46,507		41,664
Real Estate Mortgage Loans | Residential
Allowance for Loan Losses
Beginning Balance	1,412	1,579
Provision (credit) for loan losses	(405)	80
Loan charge-offs		(2)
Loan recoveries	1	1
Ending Balance	1,008	1,658
Ending Balance, Individually evaluated for impairment			34		507
Ending Balance, Collectively evaluated for impairment			974		1,151
Total	1,008	1,658	1,008	1,412	1,658
Loans, Individually evaluated for impairment			741		2,007
Loans, Collectively evaluated for impairment			240,801		190,177
Loans, Acquired loans with deteriorated credit quality			479		318
Total Loans			242,021	237,218	192,502
Real Estate Mortgage Loans | Commercial
Allowance for Loan Losses
Beginning Balance	3,601	1,961
Provision (credit) for loan losses	1,714	268
Loan recoveries	4	92
Ending Balance	5,319	2,321
Ending Balance, Individually evaluated for impairment					44
Ending Balance, Collectively evaluated for impairment			5,319		2,277
Total	5,319	2,321	5,319	3,601	2,321
Loans, Individually evaluated for impairment			2,464		2,458
Loans, Collectively evaluated for impairment			310,454		220,433
Loans, Acquired loans with deteriorated credit quality			981		51
Total Loans			313,899	310,019	222,942
Real Estate Mortgage Loans | Construction and Land Development
Allowance for Loan Losses
Beginning Balance	987	942
Provision (credit) for loan losses	414	(20)
Loan recoveries	5	3
Ending Balance	1,406	925
Ending Balance, Individually evaluated for impairment					5
Ending Balance, Collectively evaluated for impairment			1,406		920
Total	$ 1,406	$ 925	1,406	987	925
Loans, Individually evaluated for impairment			143		155
Loans, Collectively evaluated for impairment			124,133		112,319
Loans, Acquired loans with deteriorated credit quality			25		65
Total Loans			$ 124,301	$ 113,526	$ 112,539