Residential Mortgage Loans Held in Securitization Trusts, Net and Real Estate Owned - Real Estate Owned (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2017	Dec. 31, 2016	Dec. 31, 2015
Reconciliation of Carrying Amount of Real Estate Investments [Roll Forward]
Balance at beginning of period	$ 150	$ 411	$ 965
Write downs	0	(9)	0
Transfer from mortgage loans held in securitization trusts	111	352	0
Disposal	(150)	(604)	(554)
Balance at the end of period	$ 111	$ 150	$ 411