Loans Receivable Held for Investment (Details 4) (USD $) In Thousands, unless otherwise specified	Mar. 31, 2014	Dec. 31, 2013
Impaired loans
Unpaid principal balance	$ 36,438	$ 45,032
Recorded investment	28,856	33,503
Allowance for loan losses allocated, with an allowance recorded	1,884	2,187
One-to-four units
Impaired loans
Unpaid principal balance, with no related allowance recorded	1,962	2,114
Unpaid principal balance, with an allowance recorded	1,604	1,612
Recorded investment, with no related allowance recorded	1,276	1,441
Recorded investment, with an allowance recorded	1,604	1,612
Allowance for loan losses allocated, with an allowance recorded	377	382
Five or more units
Impaired loans
Unpaid principal balance, with no related allowance recorded	1,783	2,690
Unpaid principal balance, with an allowance recorded	1,897	1,578
Recorded investment, with no related allowance recorded	1,698	2,598
Recorded investment, with an allowance recorded	1,858	1,565
Allowance for loan losses allocated, with an allowance recorded	141	143
Commercial real estate
Impaired loans
Unpaid principal balance, with no related allowance recorded	4,864	4,867
Unpaid principal balance, with an allowance recorded	3,495	3,503
Recorded investment, with no related allowance recorded	1,368	1,391
Recorded investment, with an allowance recorded	3,495	3,503
Allowance for loan losses allocated, with an allowance recorded	194	206
Church
Impaired loans
Unpaid principal balance, with no related allowance recorded	10,613	11,806
Unpaid principal balance, with an allowance recorded	10,064	12,862
Recorded investment, with no related allowance recorded	7,432	8,446
Recorded investment, with an allowance recorded	9,987	12,797
Allowance for loan losses allocated, with an allowance recorded	1,160	1,444
Commercial Sports
Impaired loans
Unpaid principal balance, with no related allowance recorded		3,850
Commercial Other
Impaired loans
Unpaid principal balance, with no related allowance recorded	18
Unpaid principal balance, with an allowance recorded	138	150
Recorded investment, with an allowance recorded	138	150
Allowance for loan losses allocated, with an allowance recorded	$ 12	$ 12