Financing Agreements - Borrowings Under Repurchase Agreement and Associated Assets Pledged as Collateral (Details) - USD ($) $ in Thousands	Mar. 31, 2024	Dec. 31, 2023
Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 9,856,421	$ 9,710,754
Asset Pledged as Collateral with Right | Agreements with mark-to-market collateral provisions
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	2,657,267	2,466,787
Mark-to-market financing agreements secured by residential whole loans | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	1,476,667	1,544,086
Mark-to-market financing agreements secured by residential whole loans | Asset Pledged as Collateral with Right | Agreements with mark-to-market collateral provisions
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	302,540	318,762
Mark-to-market financing agreements secured by residential whole loans | Held at Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying value	$ 1,880,014	$ 1,712,489
Weighted average haircut (percent)	22.06%	20.90%
Mark-to-market financing agreements secured by residential whole loans | Held at Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 2,420,430	$ 2,204,239
Mark-to-market financing agreements secured by securities at fair value | Held at Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying value	$ 605,673	$ 622,603
Weighted average haircut (percent)	8.39%	8.07%
Mark-to-market financing agreements secured by securities at fair value | Held at Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 682,199	$ 689,818
Mark-to-market financing agreements secured by real estate owned | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	71,414	76,629
Mark-to-market financing agreements secured by real estate owned | Asset Pledged as Collateral with Right | Agreements with mark-to-market collateral provisions
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	39,546	43,295
Mark-to-market financing agreements secured by real estate owned | Held at Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying value	$ 23,411	$ 25,163
Weighted average haircut (percent)	48.55%	49.39%
Mark-to-market financing agreements secured by real estate owned | Held at Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 45,502	$ 50,365
Non-mark-to-market financing secured by residential whole loans | Held at Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying value	$ 1,102,114	$ 1,216,697
Weighted average haircut (percent)	18.99%	17.65%
Non-mark-to-market financing secured by residential whole loans | Held at Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 1,407,278	$ 1,510,146
Non-mark-to-market financing secured by real estate owned | Held at Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying value	$ 0	$ 0
Weighted average haircut (percent)	0.00%	0.00%
Non-mark-to-market financing secured by real estate owned | Held at Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 0	$ 0