Fair Value (Details) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2012	Dec. 31, 2011
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	$ 15,533	$ 21,518
Foreclosed assets	3,098	2,312
Recurring basis | Balance
Financial assets and liabilities measured on a recurring and non-recurring basis
I/O strip receivables	2,140	2,094
Recurring basis | Balance | Agency mortgage-backed securities
Financial assets and liabilities measured on a recurring and non-recurring basis
Available-for-sale securities	325,926	350,348
Recurring basis | Balance | Corporate bonds
Financial assets and liabilities measured on a recurring and non-recurring basis
Available-for-sale securities	23,221
Recurring basis | Balance | Trust preferred securities
Financial assets and liabilities measured on a recurring and non-recurring basis
Available-for-sale securities	40,673	30,107
Recurring basis | Significant Other Observable Inputs (Level 2)
Financial assets and liabilities measured on a recurring and non-recurring basis
I/O strip receivables	2,140	2,094
Recurring basis | Significant Other Observable Inputs (Level 2) | Agency mortgage-backed securities
Financial assets and liabilities measured on a recurring and non-recurring basis
Available-for-sale securities	325,926	350,348
Recurring basis | Significant Other Observable Inputs (Level 2) | Corporate bonds
Financial assets and liabilities measured on a recurring and non-recurring basis
Available-for-sale securities	23,221
Recurring basis | Significant Other Observable Inputs (Level 2) | Trust preferred securities
Financial assets and liabilities measured on a recurring and non-recurring basis
Available-for-sale securities	40,673	30,107
Non-recurring basis | Balance
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	8,131	9,952
Foreclosed assets	3,098	2,312
Non-recurring basis | Balance | Commercial and residential
Financial assets and liabilities measured on a recurring and non-recurring basis
Foreclosed assets	183	156
Non-recurring basis | Balance | Land and construction
Financial assets and liabilities measured on a recurring and non-recurring basis
Foreclosed assets	2,915	2,156
Non-recurring basis | Balance | Commercial
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	5,069	6,526
Non-recurring basis | Balance | Real estate | Commercial and residential
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	906	1,794
Non-recurring basis | Balance | Real estate | Land and construction
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans held-for-sale - other:	177	186
Impaired loans - held-for-investment:	1,697	1,590
Non-recurring basis | Balance | Real estate | Home equity
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	317	32
Non-recurring basis | Balance | Consumer
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	142	10
Non-recurring basis | Significant Other Observable Inputs (Level 2) | Real estate | Land and construction
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans held-for-sale - other:	177	186
Non-recurring basis | Significant Unobservable Inputs (Level 3)
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	8,131	9,952
Foreclosed assets	3,098	2,312
Non-recurring basis | Significant Unobservable Inputs (Level 3) | Commercial and residential
Financial assets and liabilities measured on a recurring and non-recurring basis
Foreclosed assets	183	156
Non-recurring basis | Significant Unobservable Inputs (Level 3) | Land and construction
Financial assets and liabilities measured on a recurring and non-recurring basis
Foreclosed assets	2,915	2,156
Non-recurring basis | Significant Unobservable Inputs (Level 3) | Commercial
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	5,069	6,526
Non-recurring basis | Significant Unobservable Inputs (Level 3) | Real estate | Commercial and residential
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	906	1,794
Non-recurring basis | Significant Unobservable Inputs (Level 3) | Real estate | Land and construction
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	1,697	1,590
Non-recurring basis | Significant Unobservable Inputs (Level 3) | Real estate | Home equity
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	317	32
Non-recurring basis | Significant Unobservable Inputs (Level 3) | Consumer
Financial assets and liabilities measured on a recurring and non-recurring basis
Impaired loans - held-for-investment:	$ 142	$ 10