CONSOLIDATED STATEMENTS OF CASH FLOWS - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2023	Dec. 31, 2022	Dec. 31, 2021
Cash Flows From Operating Activities:
Net income/(loss)	$ 80,164	$ (231,581)	$ 328,870
Adjustments to reconcile net income/(loss) to net cash provided by operating activities:
Net (gain)/loss on residential whole loans	(87,379)	866,762	(12,931)
Impairment and other net (gain)/loss on securities and other portfolio investments, net	(7,058)	25,067	(74,602)
Net gain on real estate owned	(9,512)	(24,473)	(18,772)
Accretion of purchase discounts and amortization of purchase premiums on residential whole loans and securities	(18,968)	(27,314)	(59,424)
Provision/(reversal of provision) for credit losses on residential whole loans and other assets	(6,845)	25,933	(48,355)
Net (gain)/loss on derivatives used for risk management purposes	93,828	(247,898)	(2,095)
Net margin received/(paid) for derivatives used for risk management purposes	(35,643)	214,754	574
Net (gain)/loss on securitized debt measured at fair value through earnings	86,402	(290,639)	(14,391)
Net other non-cash losses included in net income	31,719	31,370	31,358
(Increase)/decrease in other assets	(80,930)	25,482	(6,475)
Increase/(decrease) in other liabilities	62,961	(12,114)	14,046
Net cash provided by operating activities	108,739	355,349	137,803
Cash Flows From Investing Activities:
Purchases and origination of residential whole loans, loan related investments and capitalized advances	(2,914,915)	(3,206,941)	(4,516,971)
Proceeds from sales of residential whole loans	345,656	0	0
Principal payments on residential whole loans and loan related investments	1,445,759	1,878,802	2,012,901
Increase in cash balances resulting from Lima One purchase transaction, net	0	0	6,121
Purchases of securities	(588,915)	0	0
Proceeds from sales of securities and other assets	23,294	15,660	0
Principal payments on securities	35,626	53,121	157,297
Proceeds from sales of real estate owned	115,026	133,980	187,010
Other investing activities	(11,121)	9,450	(30,896)
Net cash used in investing activities	(1,549,590)	(1,115,928)	(2,184,538)
Cash Flows From Financing Activities:
Principal payments on financing agreements with mark-to-market collateral provisions	(2,910,832)	(2,971,332)	(1,822,198)
Proceeds from borrowings under financing agreements with mark-to-market collateral provisions	3,049,030	2,631,606	3,022,279
Principal payments on other collateralized financing agreements	(1,997,811)	(2,152,143)	(1,883,068)
Proceeds from borrowings under other collateralized financing agreements	3,503,400	3,676,510	2,692,576
Payment made for other collateralized financing agreement related costs	(12,593)	(16,390)	(7,145)
Redemption of convertible senior notes and Senior Notes	(20,228)	0	(100,000)
Proceeds from issuances of common stock	(7)	1,183	1,825
Payments made for the repurchase of common stock through the stock repurchase program	0	(102,311)	(85,591)
Dividends paid on preferred stock	(32,875)	(32,875)	(32,875)
Dividends paid on common stock and dividend equivalents	(143,103)	(184,035)	(156,140)
Net cash provided by financing activities	1,434,981	850,213	1,629,663
Net increase/(decrease) in cash, cash equivalents and restricted cash	(5,870)	89,634	(417,072)
Cash, cash equivalents and restricted cash at beginning of period	494,081	404,447	821,519
Cash, cash equivalents and restricted cash at end of period	488,211	494,081	404,447
Supplemental Disclosure of Cash Flow Information
Interest paid	418,135	239,185	116,966
Non-cash Investing and Financing Activities:
Transfer from residential whole loans to real estate owned	84,662	82,911	72,304
Transfer from other interest earning assets (commercial loans) to REO	22,716	0	0
Dividends and dividend equivalents declared and unpaid	35,789	35,769	47,751
Right-of-use lease asset and lease liability	0	0	40,893
Repayment of Lima One preferred stock in connection with the Lima One transaction	0	0	22,030
Receivable for sale of unsettled residential whole loans	14,033	275,656	0
Payable for unsettled investment purchases	103,654	132,025	0
Deconsolidation of securitized Agency eligible investor loans and related debt	$ 0	$ 490,952	$ 0