Acquisitions Schedule of Consideration Paid, Assets Acquired and Liabilities Assumed in Acquisition of PBI (Details) - USD ($) $ in Thousands	Jan. 01, 2019	Dec. 31, 2019	Dec. 31, 2018
Business Acquisition [Line Items]
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Property Held-For-Sale	$ 0
Goodwill		$ 12,658	$ 15,272
As Recorded By Summit [Member]
Business Acquisition [Line Items]
Consideration paid in acquisition	21,737
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Cash and Equivalents	33,329
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Cash and Cash Equivalents	(93)
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Current Assets, Marketable Securities	55,113
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Securities Available for Sale	(93)
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Purchased Performing Loans	41,399
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Purchased Performing Loans	(977)
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Purchased Credit Impaired Loans	0
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Purchased Credit Impaired Loans	0
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Financial Assets, Allowance For Loan Losses	0
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Allowance For Loan Losses	410
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Property, Plant, and Equipment	815
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Property, Plant, And Equipment	(567)
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Property Held-For-Sale	0
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Finite-Lived Intangibles	2,129
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Intangibles	2,129
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Other Noncurrent Assets	1,010
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Other Assets	(100)
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Assets	133,795
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Assets	709
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Deposits	112,379
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Deposits	315
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Noncurrent Liabilities, Other	1,775
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Other Liabilities	353
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Liabilities	114,154
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Liabilities	668
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Net	19,641
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Fair Value Adjustment, Assets (Liabilities), Net	41
Goodwill	2,096
Peoples Bankshares Inc [Member]
Business Acquisition [Line Items]
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Cash and Equivalents	33,422
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Current Assets, Marketable Securities	55,206
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Purchased Performing Loans	42,376
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Purchased Credit Impaired Loans	0
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Financial Assets, Allowance For Loan Losses	(410)
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Property, Plant, and Equipment	1,382
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Property Held-For-Sale	0
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Finite-Lived Intangibles	0
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Other Noncurrent Assets	1,110
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Assets	133,086		133,000
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Deposits	112,064
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Noncurrent Liabilities, Other	1,422
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Liabilities	113,486		$ 113,000
Business Combination, Recognized Identifiable Assets Acquired and Liabilities Assumed, Net	19,600
Goodwill	2,100
Cash Consideration [Member] | As Recorded By Summit [Member]
Business Acquisition [Line Items]
Consideration paid in acquisition	12,740
Cash Consideration [Member] | Peoples Bankshares Inc [Member]
Business Acquisition [Line Items]
Consideration paid in acquisition	12,700
Stock Consideration [Member] | As Recorded By Summit [Member]
Business Acquisition [Line Items]
Consideration paid in acquisition	$ 8,997