Fair Value Measurement (Details 4) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2012	Jun. 30, 2012	Dec. 31, 2011	Sep. 30, 2011	Jun. 30, 2011	Dec. 31, 2010
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Cash and Cash Equivalents, at Carrying Value	$ 22,051		$ 263,302	$ 138,429		$ 72,639
Assets:
Loans held for investment, net	1,268,703		1,300,873
Fair value of loans held for sale	525,926		130,546
Servicing Asset at Amortized Cost	7,725	7,655	7,112	6,612	6,608	6,035
Federal Home Loan Bank Stock	36,697		37,027
Deposits	1,981,814		2,009,755
Liabilities:
Advances from Federal Home Loan Banks	131,597		57,919
Long-term Debt	61,857		61,857
Carrying (Reported) Amount, Fair Value Disclosure [Member]
Assets:
Cash and cash equivalents			263,302
Loans held for investment			1,300,873
Fair value of loans held for sale			19,863
Servicing Asset at Amortized Cost			7,112
Federal Home Loan Bank Stock			37,027
Liabilities:
Deposits			2,009,755
Federal Home Loan Bank advances			57,919
Long-term debt			61,857
Estimate of Fair Value, Fair Value Disclosure [Member]
Assets:
Cash and cash equivalents	22,051		263,302
Loans held for investment	1,310,234		1,349,680
Fair value of loans held for sale	6,654		19,863
Servicing Asset at Amortized Cost	9,118		8,444
Federal Home Loan Bank Stock	36,697		37,027
Liabilities:
Deposits	1,986,794		2,012,708
Federal Home Loan Bank advances	135,933		63,243
Long-term debt	60,241		60,591
Estimate of Fair Value, Fair Value Disclosure [Member] | Level 1 [Member]
Assets:
Cash and cash equivalents	22,051		263,302
Loans held for investment	0		0
Fair value of loans held for sale	0		0
Servicing Asset at Amortized Cost	0		0
Federal Home Loan Bank Stock	0		0
Liabilities:
Deposits	0		0
Federal Home Loan Bank advances	0		0
Securities Loaned or Sold under Agreements to Repurchase, Fair Value Disclosure			0
Long-term debt	0
Estimate of Fair Value, Fair Value Disclosure [Member] | Level 2 [Member]
Assets:
Cash and cash equivalents	0		0
Loans held for investment	0		0
Fair value of loans held for sale	6,654		19,863
Servicing Asset at Amortized Cost	0
Federal Home Loan Bank Stock	36,697		37,027
Liabilities:
Deposits	1,986,794		2,012,708
Federal Home Loan Bank advances	135,933		63,243
Securities Loaned or Sold under Agreements to Repurchase, Fair Value Disclosure			60,591
Long-term debt	60,241
Estimate of Fair Value, Fair Value Disclosure [Member] | Level 3 [Member]
Assets:
Cash and cash equivalents	0		0
Loans held for investment	1,310,234		1,349,680
Fair value of loans held for sale	0		0
Servicing Asset at Amortized Cost	9,118		8,444
Federal Home Loan Bank Stock	0		0
Liabilities:
Deposits	0		0
Federal Home Loan Bank advances	0		0
Securities Loaned or Sold under Agreements to Repurchase, Fair Value Disclosure			0
Long-term debt	$ 0