Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	9 Months Ended		12 Months Ended
	Sep. 30, 2014		Dec. 31, 2013
Allowance for loan losses [Roll Forward]
Beginning balance	$ 13,320		$ 14,311
Adjustment for the sale of majority interest in consolidated subsidiary	(95)
Charge-offs	(4,114)		(1,381)
Recoveries	802		281
Provision	1,510		109
Ending balance	11,423		13,320
Ending allowance balance attributable to loans:
Individually evaluated for impairment	2,454		4,655
Collectively evaluated for impairment	8,969		8,665
Total ending allowance balance	11,423		13,320
Loans:
Individually evaluated for impairment	14,738		24,696
Collectively evaluated for impairment	714,014		703,784
Total Loans	728,752	[1]	728,480	[1]
Real Estate Construction [Member]
Allowance for loan losses [Roll Forward]
Beginning balance	847		1,258
Adjustment for the sale of majority interest in consolidated subsidiary	0
Charge-offs	(1,187)		(394)
Recoveries	181		68
Provision	582		(85)
Ending balance	423		847
Ending allowance balance attributable to loans:
Individually evaluated for impairment	69		486
Collectively evaluated for impairment	354		361
Total ending allowance balance	423		847
Loans:
Individually evaluated for impairment	259		2,636
Collectively evaluated for impairment	31,798		33,389
Total Loans	32,057		36,025
Real Estate Loans Secured by Farmland [Member]
Allowance for loan losses [Roll Forward]
Beginning balance	166		135
Adjustment for the sale of majority interest in consolidated subsidiary	0
Charge-offs	0		0
Recoveries	0		0
Provision	13		31
Ending balance	179		166
Ending allowance balance attributable to loans:
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	179		166
Total ending allowance balance	179		166
Loans:
Individually evaluated for impairment	0		0
Collectively evaluated for impairment	17,332		16,578
Total Loans	17,332		16,578
Real Estate Loans Secured by 1 - 4 Family Residential [Member]
Allowance for loan losses [Roll Forward]
Beginning balance	6,734		6,276
Adjustment for the sale of majority interest in consolidated subsidiary	(95)
Charge-offs	(1,186)		(785)
Recoveries	321		140
Provision	(1,279)		1,103
Ending balance	4,495		6,734
Ending allowance balance attributable to loans:
Individually evaluated for impairment	1,141		3,045
Collectively evaluated for impairment	3,354		3,689
Total ending allowance balance	4,495		6,734
Loans:
Individually evaluated for impairment	6,026		10,411
Collectively evaluated for impairment	261,190		262,973
Total Loans	267,216		273,384
Other Real Estate Loans [Member]
Allowance for loan losses [Roll Forward]
Beginning balance	3,506		4,348
Adjustment for the sale of majority interest in consolidated subsidiary	0
Charge-offs	(748)		(97)
Recoveries	109		37
Provision	526		(782)
Ending balance	3,393		3,506
Ending allowance balance attributable to loans:
Individually evaluated for impairment	305		812
Collectively evaluated for impairment	3,088		2,694
Total ending allowance balance	3,393		3,506
Loans:
Individually evaluated for impairment	4,578		9,142
Collectively evaluated for impairment	250,108		251,191
Total Loans	254,686		260,333
Commercial [Member]
Allowance for loan losses [Roll Forward]
Beginning balance	1,890		2,098
Adjustment for the sale of majority interest in consolidated subsidiary	0
Charge-offs	(958)		(75)
Recoveries	101		9
Provision	1,084		(142)
Ending balance	2,117		1,890
Ending allowance balance attributable to loans:
Individually evaluated for impairment	309		275
Collectively evaluated for impairment	1,808		1,615
Total ending allowance balance	2,117		1,890
Loans:
Individually evaluated for impairment	872		2,470
Collectively evaluated for impairment	139,683		127,084
Total Loans	140,555		129,554
Consumer [Member]
Allowance for loan losses [Roll Forward]
Beginning balance	177		196
Adjustment for the sale of majority interest in consolidated subsidiary	0
Charge-offs	(35)		(30)
Recoveries	90		27
Provision	584		(16)
Ending balance	816		177
Ending allowance balance attributable to loans:
Individually evaluated for impairment	630		37
Collectively evaluated for impairment	186		140
Total ending allowance balance	816		177
Loans:
Individually evaluated for impairment	3,003		37
Collectively evaluated for impairment	13,903		12,569
Total Loans	$ 16,906		$ 12,606

[1] Gross loan balances at September 30, 2014 and December 31, 2013 are net of deferred loan costs of $2.7 million and $2.4 million, respectively.