Use of Special Purpose Entities and Variable Interest Entities (Narrative) (Details) - USD ($) $ in Thousands		Mar. 31, 2019	Dec. 31, 2018	Mar. 31, 2018	Dec. 31, 2017
Variable Interest Entity [Line Items]
Residential whole loans, at fair value	[1]	$ 1,512,337	$ 1,665,978
Securitized debt		659,184	684,420
Total residential whole loans		5,200,000	4,700,000
Residential whole loans, at carrying value	[1]	3,724,146	3,016,715
Real estate owned		290,587	249,413	$ 182,940	$ 152,356
Other real estate, fair value		320,000	273,400
Asset-backed Securities, Securitized Loans and Receivables
Variable Interest Entity [Line Items]
Purchased credit impaired loans		202,700	209,400
Residential whole loans, at fair value		647,000	694,700
Securitized debt		659,184	684,420
Other Assets | Asset-backed Securities, Securitized Loans and Receivables
Variable Interest Entity [Line Items]
Real estate owned at fair value		$ 102,500	$ 79,000

[1] Includes approximately $202.7 million and $209.4 million of Residential whole loans, at carrying value and $647.0 million and $694.7 million of Residential whole loans, at fair value transferred to consolidated variable interest entities (“VIEs”) at March 31, 2019 and December 31, 2018, respectively. Such assets can be used only to settle the obligations of each respective VIE.