Use of Special Purpose Entities and Variable Interest Entities (Narrative) (Details) - USD ($) $ in Thousands		Dec. 31, 2019	Dec. 31, 2018
Variable Interest Entity [Line Items]
Residential whole loans, at carrying value	[1]	$ 6,066,345	$ 3,016,715
Residential whole loans, at fair value	[1]	1,381,583	1,665,978
Securitized debt		570,952	684,420
Residential whole loans		7,400,000	4,700,000
Asset-backed Securities, Securitized Loans and Receivables
Variable Interest Entity [Line Items]
Residential whole loans, at carrying value		186,400	209,400
Residential whole loans, at fair value		567,400	694,700
Securitized debt		570,952	684,420
Asset-backed Securities, Securitized Loans and Receivables | Other Assets
Variable Interest Entity [Line Items]
Real estate owned at fair value		$ 137,800	$ 79,000

[1] Includes approximately $186.4 million and $209.4 million of Residential whole loans, at carrying value and $567.4 million and $694.7 million of Residential whole loans, at fair value transferred to consolidated variable interest entities (“VIEs”) at December 31, 2019 and 2018, respectively. Such assets can be used only to settle the obligations of each respective VIE.