Fair Value (Details 3) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2012	Dec. 31, 2011	Sep. 30, 2012 Non-recurring basis Decrease in value	Sep. 30, 2011 Non-recurring basis Decrease in value	Sep. 30, 2012 Non-recurring basis Level 2	Dec. 31, 2011 Non-recurring basis Level 2	Sep. 30, 2012 Non-recurring basis Level 3	Dec. 31, 2011 Non-recurring basis Level 3	Sep. 30, 2012 Non-recurring basis Total	Dec. 31, 2011 Non-recurring basis Total
Assets measured at fair value
Portfolio Assets - loans (1)	$ 54,741	$ 97,090	$ (1,545)	$ (279)	$ 5,704		$ 447	$ 1,923	$ 6,151	$ 1,923
Loans receivable - SBA held for investment (1)	19,563	19,151	(309)	(361)	248		137	559	385	559
Real estate held for sale (2)	11,953	26,856	(782)	(1,166)	2,673	6,297			2,673	6,297
Investments in unconsolidated subsidiaries	101,035	109,393						4,567		4,567
Assets held for sale, net of related liabilities						2,011				2,011
Total			$ (2,636)	$ (1,806)