Fair Value of Financial Instruments - Carrying Value vs Fair Value (Details) - USD ($) $ in Thousands		Dec. 31, 2023	Dec. 31, 2022
Financial Assets:
Securities, at fair value	[1]	$ 746,090	$ 333,364
Restricted cash		170,211	159,898
Carrying Value | Level 1
Financial Assets:
Cash and cash equivalents		318,000	334,183
Restricted cash		170,211	159,898
Carrying Value | Level 2
Financial Assets:
Securities, at fair value		746,090	333,364
Financial Liabilities:
Securitized debt		4,750,805	3,357,590
Convertible senior notes		208,989	227,845
Estimated Fair Value | Level 1
Financial Assets:
Cash and cash equivalents		318,000	334,183
Restricted cash		170,211	159,898
Estimated Fair Value | Level 2
Financial Assets:
Securities, at fair value		746,090	333,364
Financial Liabilities:
Securitized debt		4,655,195	3,217,905
Convertible senior notes		209,065	211,015
Residential whole loans, at fair value | Carrying Value | Level 2
Financial Assets:
Residential whole loans		55,779	51,094
Residential whole loans, at fair value | Carrying Value | Level 3
Financial Assets:
Residential whole loans		8,985,513	7,467,645
Residential whole loans, at fair value | Estimated Fair Value | Level 2
Financial Assets:
Residential whole loans		55,779	51,094
Residential whole loans, at fair value | Estimated Fair Value | Level 3
Financial Assets:
Residential whole loans		8,949,859	7,397,421
Agreements with non-mark-to-market collateral provisions | Carrying Value | Level 3
Financial Liabilities:
Financing agreements		1,216,697	1,003,604
Agreements with non-mark-to-market collateral provisions | Estimated Fair Value | Level 3
Financial Liabilities:
Financing agreements		1,217,671	1,004,260
Agreements with mark-to-market collateral provisions | Carrying Value | Level 3
Financial Liabilities:
Financing agreements		1,737,652	2,111,396
Agreements with mark-to-market collateral provisions | Estimated Fair Value | Level 3
Financial Liabilities:
Financing agreements		1,738,543	2,111,647
Agreements with mark-to-market collateral provisions | Carrying Value | Level 2
Financial Liabilities:
Financing agreements		622,603	111,651
Agreements with mark-to-market collateral provisions | Estimated Fair Value | Level 2
Financial Liabilities:
Financing agreements		$ 622,603	$ 111,651

[1] See Note 6 for information regarding the Company’s pledged assets.