Loans (Details 2) (USD $)	12 Months Ended
	Dec. 31, 2012	Dec. 31, 2011
Investment in loans by portfolio segment and based on impairment
Allowance, Beginning balance	$ 42,271,000	$ 50,883,000
Provision	39,325,000	24,658,000
Chargeoffs	(24,268,000)	(35,739,000)
Recoveries	1,960,000	2,469,000
Net (chargeoffs) recovery from asset sale	(38,158,000)
Net chargeoffs	(60,466,000)	(33,270,000)
Allowance, Ending balance	21,130,000	42,271,000
Period-end amount allocated to:
Loans individually evaluated for impairment	3,024,000	11,823,000
Loans collectively evaluated for impairment	18,106,000	30,448,000
Allowance, Ending balance	21,130,000	42,271,000
Period-end balances:
Loans individually evaluated for impairment	61,017,000	153,567,000
Loans collectively evaluated for impairment	1,025,139,000	1,266,629,000
Ending balance	1,086,156,000	1,420,196,000
Permanent Real Estate Loans [Member]
Investment in loans by portfolio segment and based on impairment
Allowance, Beginning balance	31,323,000	28,066,000
Provision	34,260,000	17,073,000
Chargeoffs	(16,790,000)	(14,734,000)
Recoveries	770,000	918,000
Net (chargeoffs) recovery from asset sale	(35,744,000)
Net chargeoffs	(51,764,000)	(13,816,000)
Allowance, Ending balance	13,819,000	31,323,000
Period-end amount allocated to:
Loans individually evaluated for impairment	2,380,000	8,275,000
Loans collectively evaluated for impairment	11,439,000	23,048,000
Allowance, Ending balance	13,819,000	31,323,000
Period-end balances:
Loans individually evaluated for impairment	43,013,000	115,290,000
Loans collectively evaluated for impairment	769,155,000	972,496,000
Ending balance	812,168,000	1,087,786,000
Construction Loans [Member]
Investment in loans by portfolio segment and based on impairment
Allowance, Beginning balance	4,493,000	8,533,000
Provision	2,310,000	8,126,000
Chargeoffs	(3,480,000)	(12,504,000)
Recoveries	215,000	338,000
Net (chargeoffs) recovery from asset sale	(2,134,000)
Net chargeoffs	(5,399,000)	(12,166,000)
Allowance, Ending balance	1,404,000	4,493,000
Period-end amount allocated to:
Loans individually evaluated for impairment	478,000	3,102,000
Loans collectively evaluated for impairment	926,000	1,391,000
Allowance, Ending balance	1,404,000	4,493,000
Period-end balances:
Loans individually evaluated for impairment	7,547,000	30,587,000
Loans collectively evaluated for impairment	25,305,000	33,280,000
Ending balance	32,852,000	63,867,000
Consumer Loans [Member]
Investment in loans by portfolio segment and based on impairment
Allowance, Beginning balance	4,576,000	5,260,000
Provision	2,721,000	2,171,000
Chargeoffs	(2,740,000)	(3,446,000)
Recoveries	724,000	591,000
Net (chargeoffs) recovery from asset sale	(822,000)
Net chargeoffs	(2,838,000)	(2,855,000)
Allowance, Ending balance	4,459,000	4,576,000
Period-end amount allocated to:
Loans individually evaluated for impairment		236,000
Loans collectively evaluated for impairment	4,459,000	4,340,000
Allowance, Ending balance	4,459,000	4,576,000
Period-end balances:
Loans individually evaluated for impairment	8,784,000	3,734,000
Loans collectively evaluated for impairment	205,809,000	234,663,000
Ending balance	214,593,000	238,397,000
Commercial Loans [Member]
Investment in loans by portfolio segment and based on impairment
Allowance, Beginning balance	1,879,000	9,024,000
Provision	34,000	(2,712,000)
Chargeoffs	(1,258,000)	(5,055,000)
Recoveries	251,000	622,000
Net (chargeoffs) recovery from asset sale	542,000
Net chargeoffs	(465,000)	(4,433,000)
Allowance, Ending balance	1,448,000	1,879,000
Period-end amount allocated to:
Loans individually evaluated for impairment	166,000	210,000
Loans collectively evaluated for impairment	1,282,000	1,669,000
Allowance, Ending balance	1,448,000	1,879,000
Period-end balances:
Loans individually evaluated for impairment	1,673,000	3,956,000
Loans collectively evaluated for impairment	24,870,000	26,190,000
Ending balance	$ 26,543,000	$ 30,146,000