Loans Receivable Held for Investment (Details 4) (USD $) In Thousands, unless otherwise specified	Dec. 31, 2013	Dec. 31, 2012
Impaired loans
Unpaid principal balance	$ 45,032	$ 55,765
Recorded investment	33,503	44,376
Allowance for loan losses allocated, with an allowance recorded	2,187	2,732
One-to-four units
Impaired loans
Unpaid principal balance, with no related allowance recorded	2,114	1,986
Unpaid principal balance, with an allowance recorded	1,612	3,092
Recorded investment, with no related allowance recorded	1,441	1,484
Recorded investment, with an allowance recorded	1,612	3,092
Allowance for loan losses allocated, with an allowance recorded	382	719
Five or more units
Impaired loans
Unpaid principal balance, with no related allowance recorded	2,690	2,038
Unpaid principal balance, with an allowance recorded	1,578	1,947
Recorded investment, with no related allowance recorded	2,598	1,819
Recorded investment, with an allowance recorded	1,565	1,947
Allowance for loan losses allocated, with an allowance recorded	143	125
Commercial real estate
Impaired loans
Unpaid principal balance, with no related allowance recorded	4,867	10,184
Unpaid principal balance, with an allowance recorded	3,503	3,941
Recorded investment, with no related allowance recorded	1,391	6,423
Recorded investment, with an allowance recorded	3,503	3,941
Allowance for loan losses allocated, with an allowance recorded	206	543
Church
Impaired loans
Unpaid principal balance, with no related allowance recorded	11,806	18,664
Unpaid principal balance, with an allowance recorded	12,862	9,677
Recorded investment, with no related allowance recorded	8,446	15,689
Recorded investment, with an allowance recorded	12,797	9,639
Allowance for loan losses allocated, with an allowance recorded	1,444	1,276
Construction
Impaired loans
Unpaid principal balance, with no related allowance recorded		279
Recorded investment, with no related allowance recorded		273
Commercial Sports
Impaired loans
Unpaid principal balance, with no related allowance recorded	3,850	3,888
Commercial Other
Impaired loans
Unpaid principal balance, with an allowance recorded	150	69
Recorded investment, with an allowance recorded	150	69
Allowance for loan losses allocated, with an allowance recorded	$ 12	$ 69