Loans - Credit Quality Indicators by Class of Loans (Details) - USD ($) $ in Thousands	Mar. 31, 2019	Dec. 31, 2018	Mar. 31, 2018
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	$ 893,650	$ 899,785	$ 1,137,255
Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	255,410	267,340	401,048
Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	89,723	87,506	94,679
Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	376,523	368,449	444,277
Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	130,700	132,435	145,671
Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	40,784	43,506	50,888
Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	510	549	$ 692
Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	814,316	822,634
Pass | Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	224,476	240,232
Pass | Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	84,869	83,240
Pass | Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	338,994	329,213
Pass | Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	124,742	125,984
Pass | Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	40,725	43,416
Pass | Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	510	549
Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	36,367	49,604
Special Mention | Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	10,871	20,259
Special Mention | Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	4,428	3,910
Special Mention | Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	19,156	23,475
Special Mention | Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	1,911	1,955
Special Mention | Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	1	5
Special Mention | Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	42,967	27,547
Substandard | Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	20,063	6,849
Substandard | Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	426	356
Substandard | Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	18,373	15,761
Substandard | Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	4,047	4,496
Substandard | Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	58	85
Substandard | Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful | Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful | Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful | Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful | Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful | Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful | Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	$ 0	$ 0