Loans Receivable Held for Investment (Details 4) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2013	Dec. 31, 2012
Impaired loans
Unpaid principal balance	$ 49,838	$ 55,765
Recorded investment	38,550	44,376
Allowance for loan losses allocated, with an allowance recorded	2,450	2,732
One-to-four units
Impaired loans
Unpaid principal balance, with no related allowance recorded	2,348	1,986
Unpaid principal balance, with an allowance recorded	1,980	3,092
Recorded investment, with no related allowance recorded	1,694	1,484
Recorded investment, with an allowance recorded	1,980	3,092
Allowance for loan losses allocated, with an allowance recorded	508	719
Five or more units
Impaired loans
Unpaid principal balance, with no related allowance recorded	2,703	2,038
Unpaid principal balance, with an allowance recorded	1,586	1,947
Recorded investment, with no related allowance recorded	2,647	1,819
Recorded investment, with an allowance recorded	1,579	1,947
Allowance for loan losses allocated, with an allowance recorded	147	125
Commercial real estate
Impaired loans
Unpaid principal balance, with no related allowance recorded	4,869	10,184
Unpaid principal balance, with an allowance recorded	6,491	3,941
Recorded investment, with no related allowance recorded	1,451	6,423
Recorded investment, with an allowance recorded	6,485	3,941
Allowance for loan losses allocated, with an allowance recorded	293	543
Church
Impaired loans
Unpaid principal balance, with no related allowance recorded	12,938	18,664
Unpaid principal balance, with an allowance recorded	12,911	9,677
Recorded investment, with no related allowance recorded	9,695	15,689
Recorded investment, with an allowance recorded	12,857	9,639
Allowance for loan losses allocated, with an allowance recorded	1,486	1,276
Construction
Impaired loans
Unpaid principal balance, with no related allowance recorded		279
Recorded investment, with no related allowance recorded		273
Commercial Sports
Impaired loans
Unpaid principal balance, with no related allowance recorded	3,850	3,888
Commercial Other
Impaired loans
Unpaid principal balance, with an allowance recorded	162	69
Recorded investment, with an allowance recorded	162	69
Allowance for loan losses allocated, with an allowance recorded	$ 16	$ 69