Financing Agreements - Financing Agreements (Details) - USD ($) $ in Thousands	Dec. 31, 2022	Dec. 31, 2021
Financing Agreements [Line Items]
Unpaid Principal Balance	$ 7,043,944	$ 6,377,370
Fair Value / Carrying Value	$ 6,812,086	$ 6,378,782
Weighted Average Cost of Funding	3.46%	2.58%
Agreements with mark-to-market collateral provisions
Financing Agreements [Line Items]
Unpaid Principal Balance	$ 2,223,298	$ 2,562,872
Fair Value / Carrying Value	$ 2,223,047	$ 2,562,299
Weighted Average Cost of Funding	3.62%	2.11%
Agreements with mark-to-market collateral provisions | 0-3 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	$ 940,455
Fair Value / Carrying Value	250,900
Agreements with mark-to-market collateral provisions | 3-6 Months (1)
Financing Agreements [Line Items]
Unpaid Principal Balance	53,247
Fair Value / Carrying Value	53,200
Agreements with mark-to-market collateral provisions | 6-12 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	939,434
Fair Value / Carrying Value	$ 290,200
Extension period	1 year
Agreements with mark-to-market collateral provisions | Greater than 12 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	$ 290,162
Fair Value / Carrying Value	304,100
Agreements with mark-to-market collateral provisions | Residential Whole Loans and REO
Financing Agreements [Line Items]
Unpaid Principal Balance	2,111,647	$ 2,403,724
Fair Value / Carrying Value	$ 2,111,396	$ 2,403,151
Weighted Average Cost of Funding	3.63%	2.15%
Weighted Average Term to Maturity (Months)	6 months 27 days	6 months 18 days
Agreements with mark-to-market collateral provisions | Residential Whole Loans and REO | 0-3 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	$ 828,804
Agreements with mark-to-market collateral provisions | Residential Whole Loans and REO | 3-6 Months (1)
Financing Agreements [Line Items]
Unpaid Principal Balance	53,247
Agreements with mark-to-market collateral provisions | Residential Whole Loans and REO | 6-12 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	939,434
Agreements with mark-to-market collateral provisions | Residential Whole Loans and REO | Greater than 12 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	290,162
Agreements with mark-to-market collateral provisions | Securitized debt
Financing Agreements [Line Items]
Unpaid Principal Balance	111,651	$ 159,148
Fair Value / Carrying Value	$ 111,651	$ 159,148
Weighted Average Cost of Funding	3.34%	1.78%
Weighted Average Term to Maturity (Months)	1 month 15 days	1 month 12 days
Agreements with mark-to-market collateral provisions | Securitized debt | 0-3 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	$ 111,651
Agreements with mark-to-market collateral provisions | Securitized debt | 3-6 Months (1)
Financing Agreements [Line Items]
Unpaid Principal Balance	0
Agreements with mark-to-market collateral provisions | Securitized debt | 6-12 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	0
Agreements with mark-to-market collateral provisions | Securitized debt | Greater than 12 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	0
Agreements with mark-to-market collateral provisions | Fair Value
Financing Agreements [Line Items]
Fair Value / Carrying Value	884,500	$ 1,300,000
Financing agreements, amortized cost	884,500	1,300,000
Agreements with non-mark-to-market collateral provisions | Residential Whole Loans and REO
Financing Agreements [Line Items]
Unpaid Principal Balance	1,004,260	939,003
Fair Value / Carrying Value	$ 1,003,604	$ 939,540
Weighted Average Cost of Funding	5.00%	3.57%
Weighted Average Term to Maturity (Months)	1 year 4 months 24 days	9 months 24 days
Agreements with non-mark-to-market collateral provisions | Residential Whole Loans and REO | 0-3 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	$ 9,268
Agreements with non-mark-to-market collateral provisions | Residential Whole Loans and REO | 3-6 Months (1)
Financing Agreements [Line Items]
Unpaid Principal Balance	184,576
Agreements with non-mark-to-market collateral provisions | Residential Whole Loans and REO | 6-12 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	415,041
Agreements with non-mark-to-market collateral provisions | Residential Whole Loans and REO | Greater than 12 Months
Financing Agreements [Line Items]
Unpaid Principal Balance	395,375
Agreements with non-mark-to-market collateral provisions | Fair Value
Financing Agreements [Line Items]
Fair Value / Carrying Value	578,900	$ 628,300
Financing agreements, amortized cost	578,900	627,000
Securitized debt | Residential Whole Loans and REO
Financing Agreements [Line Items]
Unpaid Principal Balance	3,586,397	2,645,495
Fair Value / Carrying Value	$ 3,357,590	$ 2,650,473
Weighted Average Cost of Funding	2.99%	1.94%
Securitized debt | Fair Value
Financing Agreements [Line Items]
Fair Value / Carrying Value	$ 2,400,000	$ 1,300,000
Weighted Average Cost of Funding	0.14%	0.02%
Financing agreements, amortized cost	$ 2,600,000	$ 1,300,000
Convertible senior notes | Unsecured Debt
Financing Agreements [Line Items]
Unpaid Principal Balance	229,989	230,000
Fair Value / Carrying Value	$ 227,845	$ 226,470
Weighted Average Cost of Funding	6.94%	6.94%