Residential Loans, at Fair Value - Concentration of Risk (Details) - Geographic Concentration Risk	6 Months Ended	12 Months Ended
	Jun. 30, 2021	Dec. 31, 2020
Residential loans | California
Concentration Risk [Line Items]
Geographic concentration, percent	22.00%	23.60%
Residential loans | Florida
Concentration Risk [Line Items]
Geographic concentration, percent	11.00%	13.10%
Residential loans | New York
Concentration Risk [Line Items]
Geographic concentration, percent	10.00%	9.20%
Residential loans | New Jersey
Concentration Risk [Line Items]
Geographic concentration, percent	7.50%	5.60%
Residential loans | Texas
Concentration Risk [Line Items]
Geographic concentration, percent	5.70%	5.60%
Residential loans | Maryland
Concentration Risk [Line Items]
Geographic concentration, percent	2.70%	2.80%
Residential loans | Illinois
Concentration Risk [Line Items]
Geographic concentration, percent	2.70%	2.50%
Consolidated SLST | Consolidated SLST | VIE, Primary Beneficiary | California
Concentration Risk [Line Items]
Geographic concentration, percent	10.70%	10.90%
Consolidated SLST | Consolidated SLST | VIE, Primary Beneficiary | Florida
Concentration Risk [Line Items]
Geographic concentration, percent	10.40%	10.50%
Consolidated SLST | Consolidated SLST | VIE, Primary Beneficiary | New York
Concentration Risk [Line Items]
Geographic concentration, percent	9.50%	9.30%
Consolidated SLST | Consolidated SLST | VIE, Primary Beneficiary | New Jersey
Concentration Risk [Line Items]
Geographic concentration, percent	7.20%	7.10%
Consolidated SLST | Consolidated SLST | VIE, Primary Beneficiary | Texas
Concentration Risk [Line Items]
Geographic concentration, percent	3.90%	4.00%
Consolidated SLST | Consolidated SLST | VIE, Primary Beneficiary | Maryland
Concentration Risk [Line Items]
Geographic concentration, percent	3.90%	3.80%
Consolidated SLST | Consolidated SLST | VIE, Primary Beneficiary | Illinois
Concentration Risk [Line Items]
Geographic concentration, percent	6.90%	6.80%
Residential loans held in securitization trusts | California
Concentration Risk [Line Items]
Geographic concentration, percent	22.90%	19.80%
Residential loans held in securitization trusts | Florida
Concentration Risk [Line Items]
Geographic concentration, percent	10.30%	8.10%
Residential loans held in securitization trusts | New York
Concentration Risk [Line Items]
Geographic concentration, percent	7.70%	8.90%
Residential loans held in securitization trusts | New Jersey
Concentration Risk [Line Items]
Geographic concentration, percent	5.00%	5.60%
Residential loans held in securitization trusts | Texas
Concentration Risk [Line Items]
Geographic concentration, percent	4.60%	4.30%
Residential loans held in securitization trusts | Maryland
Concentration Risk [Line Items]
Geographic concentration, percent	5.00%	6.30%
Residential loans held in securitization trusts | Illinois
Concentration Risk [Line Items]
Geographic concentration, percent	2.30%	2.70%