Borrowings Narrative (Details) (USD $)	Dec. 31, 2013	Dec. 31, 2012
Line of Credit Facility [Line Items]
Remaining credit availability	$ 282,800,000
Long-term Debt	80,000,000
Federal Home Loan Bank advances, Interest rate, range from (in hundredths)	0.17%
Federal Home Loan Bank advances, Interest rate, range to (in hundredths)	1.98%
Federal Home Loan Bank advances, Weighted-average rate (in hundredths)	1.13%	1.17%
Federal Home Loan Bank borrowings	80,000,000	77,912,000
Federal Reserve Bank of Richmond [Member]
Line of Credit Facility [Line Items]
Revolving lines of credit, outstanding balance	0
Line of credit, maximum credit limit	43,100,000.0
Other Institutions [Member]
Line of Credit Facility [Line Items]
Line of credit, maximum credit limit	24,000,000
Southern Trust Mortgage [Member]
Line of Credit Facility [Line Items]
Loan guarantees by Middleburg Bank	10,000,000
Southern Trust Mortgage [Member] | Regional Bank [Member]
Line of Credit Facility [Line Items]
Revolving lines of credit, outstanding balance	0
Southern Trust Mortgage [Member] | Regional Bank [Member] | Fund Mortgage Held For Sale [Member]
Line of Credit Facility [Line Items]
Line of credit, maximum credit limit		24,000,000
Southern Trust Mortgage [Member] | Subsidiary Bank [Member] | Fund Mortgage Held For Sale [Member]
Line of Credit Facility [Line Items]
Revolving lines of credit, outstanding balance	32,300,000
Line of credit, maximum credit limit	75,000,000
Southern Trust Mortgage [Member] | Subsidiary Bank [Member] | Operating Purpose [Member]
Line of Credit Facility [Line Items]
Revolving lines of credit, outstanding balance	89,000
Line of credit, maximum credit limit	$ 5,000,000