Fair Value of Financial Instruments - Carrying Value vs Fair Value (Details) - USD ($) $ in Thousands		Jun. 30, 2023	Dec. 31, 2022
Financial Assets:
Securities, at fair value	[1]	$ 594,294	$ 333,364
Restricted cash		174,005	159,898
Carrying Value | Level 1
Financial Assets:
Cash and cash equivalents		329,391	334,183
Restricted cash		174,005	159,898
Carrying Value | Level 2
Financial Assets:
Securities, at fair value		594,294	333,364
Financial Liabilities:
Securitized debt		3,969,274	3,357,590
Convertible senior notes		228,575	227,845
Estimated Fair Value | Level 1
Financial Assets:
Cash and cash equivalents		329,391	334,183
Restricted cash		174,005	159,898
Estimated Fair Value | Level 2
Financial Assets:
Securities, at fair value		594,294	333,364
Financial Liabilities:
Securitized debt		3,839,491	3,217,905
Convertible senior notes		220,071	211,015
Residential whole loans, at fair value | Carrying Value | Level 2
Financial Assets:
Residential whole loans		58,068	51,094
Residential whole loans, at fair value | Carrying Value | Level 3
Financial Assets:
Residential whole loans		8,081,718	7,467,645
Residential whole loans, at fair value | Estimated Fair Value | Level 2
Financial Assets:
Residential whole loans		58,068	51,094
Residential whole loans, at fair value | Estimated Fair Value | Level 3
Financial Assets:
Residential whole loans		8,023,447	7,397,421
Financing agreements with non-mark-to-market collateral provisions | Carrying Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		967,884	1,003,604
Financing agreements with non-mark-to-market collateral provisions | Estimated Fair Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		968,546	1,004,260
Agreements with mark-to-market collateral provisions | Carrying Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		1,938,379	2,111,396
Agreements with mark-to-market collateral provisions | Estimated Fair Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		1,939,310	2,111,647
Financing agreements with mark-to-market collateral provisions | Carrying Value | Level 2
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		464,064	111,651
Financing agreements with mark-to-market collateral provisions | Estimated Fair Value | Level 2
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		$ 464,064	$ 111,651

[1] See Note 6 for information regarding the Company’s pledged assets.