Financing Agreements - Pledged Collateral (Details) - USD ($) $ in Thousands	Sep. 30, 2023	Dec. 31, 2022
Debt Instrument [Line Items]
Accrued interest on assets	$ 34,900	$ 30,900
Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	9,180,009	8,057,175
Non-Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	1,390,220	1,331,386
Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	2,606,244	2,654,442
Securitized | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	5,183,545	4,071,347
Residential whole loans, at carrying value | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	1,598,579	1,814,780
Residential whole loans, at carrying value | Non-Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	58,487	215,993
Residential whole loans, at carrying value | Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	346,353	284,683
Residential whole loans, at carrying value | Securitized | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	1,193,739	1,314,104
Residential whole loans, at fair value | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	6,823,691	5,980,471
Residential whole loans, at fair value | Non-Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	1,331,733	1,095,556
Residential whole loans, at fair value | Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	1,534,207	2,164,158
Residential whole loans, at fair value | Securitized | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	3,957,751	2,720,757
Securities, at fair value | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	678,801	177,111
Securities, at fair value | Non-Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	0	0
Securities, at fair value | Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	678,801	177,111
Securities, at fair value | Securitized | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	0	0
Other assets: REO | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	78,938	84,813
Other assets: REO | Non-Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	0	19,837
Other assets: REO | Mark-to-Market | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	46,883	28,490
Other assets: REO | Securitized | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 32,055	$ 36,486