Fair Values - Qualitative Information About Level 3 Fair Value Measurements for Financial Instruments Measured at Fair Value on a Non-recurring Basis (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2015	Dec. 31, 2014
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	$ 9,496	$ 16,963
Other real estate owned	4,997,318	4,757,461
Nonrecurring basis
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	9,496	16,963
Other real estate owned	8,515	10,312
Commercial real estate | Nonrecurring basis
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	3,698	8,481
Other real estate owned	2,796	1,470
Commercial real estate | Nonrecurring basis | Loans receivable
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	3,698	8,481
Construction real estate | Nonrecurring basis
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Other real estate owned	3,387	6,473
SEPH commercial land and development | Nonrecurring basis
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	2,044
SEPH commercial land and development | Nonrecurring basis | Loans receivable
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	2,044	2,078
Remaining commercial | Nonrecurring basis
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	1,872	3,483
Remaining commercial | Nonrecurring basis | Loans receivable
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	1,872	3,483
Residential real estate | Nonrecurring basis
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	1,882	2,921
Other real estate owned	2,332	2,369
Residential real estate | Nonrecurring basis | Loans receivable
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Impaired loans	1,882	2,921
Commercial real estate | Nonrecurring basis | Other real estate owned
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Other real estate owned	2,796	1,470
Construction real estate | Nonrecurring basis | Other real estate owned
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Other real estate owned	3,387	6,473
Residential Real Estate | Nonrecurring basis | Other real estate owned
Fair Value Inputs, Assets, Quantitative Information [Line Items]
Other real estate owned	$ 2,332	$ 2,369
Sales comparison approach | Commercial real estate | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	0.00%	0.00%
Sales comparison approach | Commercial real estate | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	45.90%	84.00%
Sales comparison approach | Commercial real estate | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	20.30%	38.80%
Sales comparison approach | SEPH commercial land and development | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	5.00%	5.00%
Sales comparison approach | SEPH commercial land and development | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	40.00%	35.00%
Sales comparison approach | SEPH commercial land and development | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	22.10%	17.50%
Sales comparison approach | Remaining commercial | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	0.00%	0.20%
Sales comparison approach | Remaining commercial | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	25.30%	76.00%
Sales comparison approach | Remaining commercial | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	1.00%	45.40%
Sales comparison approach | Residential real estate | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	0.00%	0.00%
Sales comparison approach | Residential real estate | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	96.70%	120.60%
Sales comparison approach | Residential real estate | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Adj to comparables	12.50%	11.10%
Sales comparison approach | Commercial real estate | Low | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Adj to comparables	2.00%	0.00%
Sales comparison approach | Commercial real estate | High | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Adj to comparables	71.00%	87.00%
Sales comparison approach | Commercial real estate | Weighted Average [Member] | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Adj to comparables	26.90%	30.50%
Sales comparison approach | Construction real estate | Low | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Adj to comparables	0.00%	0.00%
Sales comparison approach | Construction real estate | High | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Adj to comparables	85.00%	82.90%
Sales comparison approach | Construction real estate | Weighted Average [Member] | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Adj to comparables	24.30%	27.10%
Sales comparison approach | Residential Real Estate | Low | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Adj to comparables	0.10%	0.00%
Sales comparison approach | Residential Real Estate | High | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Adj to comparables	61.80%	38.30%
Sales comparison approach | Residential Real Estate | Weighted Average [Member] | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Adj to comparables	23.00%	10.10%
Income approach | Commercial real estate | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Capitalization rate	7.00%	8.00%
Income approach | Commercial real estate | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Capitalization rate	13.30%	9.50%
Income approach | Commercial real estate | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Capitalization rate	9.50%	9.40%
Income approach | Residential real estate | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Capitalization rate	3.80%	7.90%
Income approach | Residential real estate | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Capitalization rate	10.10%	10.00%
Income approach | Residential real estate | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Capitalization rate	9.10%	8.00%
Income approach | Commercial real estate | Low | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Capitalization rate	9.50%	8.40%
Income approach | Commercial real estate | High | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Capitalization rate	9.50%	10.00%
Income approach | Commercial real estate | Weighted Average [Member] | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Capitalization rate	9.50%	9.40%
Income approach | Residential Real Estate | Low | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Capitalization rate		6.80%
Income approach | Residential Real Estate | High | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Capitalization rate		7.80%
Income approach | Residential Real Estate | Weighted Average [Member] | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Capitalization rate		7.60%
Cost approach | Commercial real estate | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Fair Value Inputs, Accumulated Depreciation	50.00%	23.00%
Cost approach | Commercial real estate | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Fair Value Inputs, Accumulated Depreciation	50.00%	23.00%
Cost approach | Commercial real estate | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Fair Value Inputs, Accumulated Depreciation	50.00%	23.00%
Cost approach | Residential real estate | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Fair Value Inputs, Accumulated Depreciation	33.30%
Cost approach | Residential real estate | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Fair Value Inputs, Accumulated Depreciation	50.00%
Cost approach | Residential real estate | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Fair Value Inputs, Accumulated Depreciation	43.40%
Cost approach | Commercial real estate | Low | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Fair Value Inputs, Accumulated Depreciation		60.00%
Cost approach | Commercial real estate | High | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Fair Value Inputs, Accumulated Depreciation		95.00%
Cost approach | Commercial real estate | Weighted Average [Member] | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Fair Value Inputs, Accumulated Depreciation		77.50%
Bulk sale approach | SEPH commercial land and development | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Discount rate	10.70%	10.80%
Bulk sale approach | SEPH commercial land and development | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Discount rate	10.70%	10.80%
Bulk sale approach | SEPH commercial land and development | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Discount rate	10.70%	10.80%
Bulk sale approach | Remaining commercial | Low | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Discount rate	10.00%	10.00%
Bulk sale approach | Remaining commercial | High | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Discount rate	10.70%	22.00%
Bulk sale approach | Remaining commercial | Weighted Average [Member] | Nonrecurring basis | Loans receivable
Fair Value Inputs [Abstract]
Discount rate	10.00%	16.50%
Bulk sale approach | Construction real estate | Low | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Discount rate	15.00%	15.00%
Bulk sale approach | Construction real estate | High | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Discount rate	15.00%	15.00%
Bulk sale approach | Construction real estate | Weighted Average [Member] | Nonrecurring basis | Other real estate owned
Fair Value Inputs [Abstract]
Discount rate	15.00%	15.00%