Residential Whole Loans - Fair Value (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2017 USD ($) loan	Dec. 31, 2016 USD ($) loan
Residential Whole Loans [Abstract]
Outstanding principal balance	$ 1,562,373	$ 966,174
Aggregate fair value	$ 1,325,115	$ 814,682
Number of loans | loan	6,514	3,812