Loans and Allowance for Loan Losses (Details) (USD $)	12 Months Ended
	Dec. 31, 2012	Dec. 31, 2011	Dec. 31, 2010
Financing Receivable [Line Items]
Loan charge-offs	$ 30,968,000	$ 128,424,000
Recoveries of loan charge-offs	6,873,000	6,735,000
Net loan charge-offs	24,100,000
Ratio of Allowance for loan losses to loans held for investment	2.49%	3.23%	7.18%
Real estate - mortgage, 1-4 family residential [Member] | PC [Member]
Financing Receivable [Line Items]
Loans purchases	137,400,000
Loans purchases premium	3,700,000
Granite Corp. [Member] | Real estate - mortgage, 1-4 family residential [Member] | PC [Member]
Financing Receivable [Line Items]
Loans purchases	35,800,000
Loans purchases premium	$ 300,000