Fair Value of Financial Instruments - Carrying Value vs Fair Value (Details) - USD ($) $ in Thousands		Sep. 30, 2022	Dec. 31, 2021
Financial Assets:
Securities, at fair value	[1]	$ 227,407	$ 256,685
Restricted cash		167,310	99,751
Carrying Value | Level 1
Financial Assets:
Cash and cash equivalents		434,086	304,696
Restricted cash		167,310	99,751
Carrying Value | Level 2
Financial Assets:
Securities, at fair value		227,407	256,685
Financial Liabilities:
Securitized debt		3,832,311	2,650,473
Convertible senior notes		227,489	226,470
Estimated Fair Value | Level 1
Financial Assets:
Cash and cash equivalents		434,086	304,696
Restricted cash		167,310	99,751
Estimated Fair Value | Level 2
Financial Assets:
Securities, at fair value		227,407	256,685
Financial Liabilities:
Securitized debt		3,722,703	2,646,203
Convertible senior notes		200,742	239,292
Residential whole loans, at fair value | Carrying Value | Level 2
Financial Assets:
Residential whole loans		852,996	1,082,765
Residential whole loans, at fair value | Carrying Value | Level 3
Financial Assets:
Residential whole loans		7,341,053	6,830,235
Residential whole loans, at fair value | Estimated Fair Value | Level 2
Financial Assets:
Residential whole loans		852,996	1,082,765
Residential whole loans, at fair value | Estimated Fair Value | Level 3
Financial Assets:
Residential whole loans		7,282,872	6,983,686
Financing agreements with non-mark-to-market collateral provisions | Carrying Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		1,104,849	939,540
Financing agreements with non-mark-to-market collateral provisions | Estimated Fair Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		1,105,441	940,257
Agreements with mark-to-market collateral provisions | Carrying Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		1,981,904	2,403,151
Agreements with mark-to-market collateral provisions | Estimated Fair Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		1,981,934	2,403,724
Financing agreements with mark-to-market collateral provisions | Carrying Value | Level 2
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		142,887	159,148
Financing agreements with mark-to-market collateral provisions | Estimated Fair Value | Level 2
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		$ 142,887	$ 159,148

[1] See Note 6 for information regarding the Company’s pledged assets.