Fair Value of Financial Instruments - Carrying Value vs Fair Value (Details) - USD ($) $ in Thousands		Jun. 30, 2022	Dec. 31, 2021
Financial Assets:
Securities, at fair value	[1]	$ 245,613	$ 256,685
Restricted cash		159,745	99,751
Carrying Value | Level 1
Financial Assets:
Cash and cash equivalents		385,598	304,696
Restricted cash		159,745	99,751
Carrying Value | Level 2
Financial Assets:
Securities, at fair value		245,613	256,685
Financial Liabilities:
Securitized debt		3,374,716	2,650,473
Convertible senior notes		227,140	226,470
Estimated Fair Value | Level 1
Financial Assets:
Cash and cash equivalents		385,598	304,696
Restricted cash		159,745	99,751
Estimated Fair Value | Level 2
Financial Assets:
Securities, at fair value		245,613	256,685
Financial Liabilities:
Securitized debt		3,299,290	2,646,203
Convertible senior notes		206,578	239,292
Residential whole loans, at fair value | Carrying Value | Level 2
Financial Assets:
Residential whole loans		912,245	1,082,765
Residential whole loans, at fair value | Carrying Value | Level 3
Financial Assets:
Residential whole loans		7,278,336	6,830,235
Residential whole loans, at fair value | Estimated Fair Value | Level 2
Financial Assets:
Residential whole loans		912,245	1,082,765
Residential whole loans, at fair value | Estimated Fair Value | Level 3
Financial Assets:
Residential whole loans		7,287,790	6,983,686
Financing agreements with non-mark-to-market collateral provisions | Carrying Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		905,087	939,540
Financing agreements with non-mark-to-market collateral provisions | Estimated Fair Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		905,761	940,257
Agreements with mark-to-market collateral provisions | Carrying Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		2,467,528	2,403,151
Agreements with mark-to-market collateral provisions | Estimated Fair Value | Level 3
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		2,467,670	2,403,724
Financing agreements with mark-to-market collateral provisions | Carrying Value | Level 2
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		157,895	159,148
Financing agreements with mark-to-market collateral provisions | Estimated Fair Value | Level 2
Financial Liabilities:
Financing agreements with mark-to-market collateral provisions		$ 157,895	$ 159,148

[1] See Note 6 for information regarding the Company’s pledged assets.