Financing Agreements - Pledged Collateral (Details) - USD ($) $ in Thousands	Dec. 31, 2023	Dec. 31, 2022
Debt Instrument [Line Items]
Accrued interest on assets	$ 36,400	$ 30,900
Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	9,710,754	8,057,175
Asset Pledged as Collateral with Right | Non-Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	1,513,904	1,331,386
Asset Pledged as Collateral with Right | Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	2,466,787	2,654,442
Asset Pledged as Collateral with Right | Securitized
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	5,730,063	4,071,347
Residential whole loans, at carrying value | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	1,544,086	1,814,780
Residential whole loans, at carrying value | Asset Pledged as Collateral with Right | Non-Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	55,056	215,993
Residential whole loans, at carrying value | Asset Pledged as Collateral with Right | Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	318,762	284,683
Residential whole loans, at carrying value | Asset Pledged as Collateral with Right | Securitized
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	1,170,268	1,314,104
Residential whole loans, at fair value | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	7,400,221	5,980,471
Residential whole loans, at fair value | Asset Pledged as Collateral with Right | Non-Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	1,458,848	1,095,556
Residential whole loans, at fair value | Asset Pledged as Collateral with Right | Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	1,414,912	2,164,158
Residential whole loans, at fair value | Asset Pledged as Collateral with Right | Securitized
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	4,526,461	2,720,757
Securities, at fair value | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	689,818	177,111
Securities, at fair value | Asset Pledged as Collateral with Right | Non-Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	0	0
Securities, at fair value | Asset Pledged as Collateral with Right | Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	689,818	177,111
Securities, at fair value | Asset Pledged as Collateral with Right | Securitized
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	0	0
Other assets: REO | Asset Pledged as Collateral with Right
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	76,629	84,813
Other assets: REO | Asset Pledged as Collateral with Right | Non-Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	0	19,837
Other assets: REO | Asset Pledged as Collateral with Right | Mark-to-Market
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	43,295	28,490
Other assets: REO | Asset Pledged as Collateral with Right | Securitized
Debt Instrument [Line Items]
Fair value of securities pledged against the repurchase agreements	$ 33,334	$ 36,486