Financing Agreements - Borrowings Under Repurchase Agreement And Associated Assets Pledged as Collateral (Details) - USD ($) $ in Thousands	Mar. 31, 2023	Dec. 31, 2022
Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 8,350,163	$ 8,057,175
Mark-to-market financing agreements secured by residential whole loans | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	1,744,490	1,814,780
Mark-to-market financing agreements secured by residential whole loans | Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying Value	$ 1,675,343	$ 2,095,002
Weighted average haircut (percent)	21.35%	18.33%
Mark-to-market financing agreements secured by residential whole loans | Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 2,172,874	$ 2,632,489
Mark-to-market financing agreements secured by securities at fair value | Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying Value	$ 405,027	$ 111,651
Weighted average haircut (percent)	13.33%	37.43%
Mark-to-market financing agreements secured by securities at fair value | Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 484,646	$ 177,111
Mark-to-market financing agreements secured by real estate owned | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	81,362	84,813
Mark-to-market financing agreements secured by real estate owned | Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying Value	$ 16,425	$ 16,394
Weighted average haircut (percent)	49.20%	48.07%
Mark-to-market financing agreements secured by real estate owned | Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 33,833	$ 33,367
Non-mark-to-market financing secured by residential whole loans | Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying Value	$ 937,882	$ 994,494
Weighted average haircut (percent)	21.12%	21.43%
Non-mark-to-market financing secured by residential whole loans | Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 1,228,569	$ 1,301,685
Non-mark-to-market financing secured by real estate owned | Fair Value
Assets Sold under Agreements to Repurchase [Line Items]
Carrying Value	$ 8,123	$ 9,109
Weighted average haircut (percent)	61.26%	60.23%
Non-mark-to-market financing secured by real estate owned | Fair Value | Asset Pledged as Collateral with Right
Assets Sold under Agreements to Repurchase [Line Items]
Fair value of securities pledged as collateral under financing agreements	$ 20,968	$ 22,902