Residential Whole Loans (Fair Value) (Details) $ in Thousands	3 Months Ended		12 Months Ended
	Mar. 31, 2018 USD ($) loan	Mar. 31, 2017 USD ($)	Dec. 31, 2017 USD ($) loan
Investments, Debt and Equity Securities [Abstract]
Outstanding principal balance	$ 1,769,454		$ 1,562,373
Aggregate fair value	$ 1,555,620		$ 1,325,115
Number of loans | loan	7,085		6,514
Net gain on residential whole loans held at fair value	$ 38,498	$ 13,773