Residential Loans, at Fair Value - Components of Net Gain (Loss) on Residential Mortgages (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2022	Mar. 31, 2021
Consolidated SLST | VIE, Primary Beneficiary
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unrealized (losses) gains, net	$ (66,645)	$ (25,343)
Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unrealized (losses) gains, net	(25,524)	6,426
Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unrealized (losses) gains, net	$ (37,774)	$ 12,251