Portfolio Loans Not Covered by Loss Share ("Non-covered") - Summary of Non-covered Loans Individually Evaluated for Impairment by Category (Details) (USD $)	12 Months Ended
	Dec. 31, 2013 loan	Dec. 31, 2012 loan	Dec. 31, 2011
Financing Receivable, Impaired [Line Items]
Number of loans over 90 days past due and still accruing	0	1
Interest lost on impaired loans	$ 1,500,000	$ 2,700,000	$ 3,000,000
Cash collected and recognized as interest income	257,000	402,000	513,000
Interest income recognized on impaired loans continuing to accrue	16,000	461,000	429,000
Non-Covered Loans
Financing Receivable, Impaired [Line Items]
Unpaid Contractual Principal Balance	24,243,000	45,218,000
Recorded Investment With No Allowance	15,122,000	5,645,000
Recorded Investment With Allowance	6,445,000	33,082,000
Total Recorded Investment	21,567,000	38,727,000
Related Allowance	1,550,000	8,176,000
Average Recorded Investment	27,490,000	39,491,000
Non-Covered Loans | Commercial & Industrial
Financing Receivable, Impaired [Line Items]
Unpaid Contractual Principal Balance	4,377,000	9,005,000
Recorded Investment With No Allowance	0	96,000
Recorded Investment With Allowance	3,384,000	8,838,000
Total Recorded Investment	3,384,000	8,934,000
Related Allowance	736,000	3,446,000
Average Recorded Investment	6,574,000	6,379,000
Non-Covered Loans | Consumer & Other
Financing Receivable, Impaired [Line Items]
Unpaid Contractual Principal Balance	0	0
Recorded Investment With No Allowance	0	0
Recorded Investment With Allowance	0	0
Total Recorded Investment	0	0
Related Allowance	0	0
Average Recorded Investment	0	0
Non-Covered Loans | Commercial - Owner Occupied
Financing Receivable, Impaired [Line Items]
Unpaid Contractual Principal Balance	606,000	6,726,000
Recorded Investment With No Allowance	201,000	2,178,000
Recorded Investment With Allowance	421,000	3,594,000
Total Recorded Investment	622,000	5,772,000
Related Allowance	107,000	339,000
Average Recorded Investment	1,868,000	7,985,000
Non-Covered Loans | Commercial - Investor Owned
Financing Receivable, Impaired [Line Items]
Unpaid Contractual Principal Balance	8,033,000	19,864,000
Recorded Investment With No Allowance	7,190,000	185,000
Recorded Investment With Allowance	0	16,577,000
Total Recorded Investment	7,190,000	16,762,000
Related Allowance	0	3,400,000
Average Recorded Investment	11,348,000	10,500,000
Non-Covered Loans | Construction and Land Development
Financing Receivable, Impaired [Line Items]
Unpaid Contractual Principal Balance	10,668,000	6,491,000
Recorded Investment With No Allowance	7,383,000	1,560,000
Recorded Investment With Allowance	2,419,000	3,135,000
Total Recorded Investment	9,802,000	4,695,000
Related Allowance	703,000	732,000
Average Recorded Investment	5,770,000	10,259,000
Non-Covered Loans | Residential
Financing Receivable, Impaired [Line Items]
Unpaid Contractual Principal Balance	559,000	3,132,000
Recorded Investment With No Allowance	348,000	1,626,000
Recorded Investment With Allowance	221,000	938,000
Total Recorded Investment	569,000	2,564,000
Related Allowance	4,000	259,000
Average Recorded Investment	$ 1,930,000	$ 4,368,000