Residential Whole Loans-Fair Value Components of Net Income (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2015	Sep. 30, 2014	Sep. 30, 2015	Sep. 30, 2014
Investments, Debt and Equity Securities [Abstract]
Coupon payments and other income received	$ 2,186		$ 5,018
Net unrealized gains	2,362		3,872
Net gain on payoff/liquidation of loans	431		1,286
Total	$ 4,979	$ 0	$ 10,176	$ 0