CONSOLIDATED STATEMENTS OF CASH FLOWS $ in Thousands	12 Months Ended
	Dec. 31, 2020 USD ($)	Dec. 31, 2019 USD ($)	Dec. 31, 2018 USD ($)
Cash Flows From Operating Activities:
Net (loss)/income	$ (679,390)	$ 378,117	$ 301,801
Adjustments to reconcile net income to net cash provided by operating activities:
Losses/(gains) on residential whole loans and real estate owned, net	243,933	(79,948)	(70,579)
Gains on residential mortgage securities and MSR related assets, net	(74,515)	(69,082)	(24,492)
Impairment and other losses on securities available-for-sale and other assets	425,082	180	1,259
Loss on terminated swaps previously designed as hedges for accounting purposes	57,034	0	0
Accretion of purchase discounts on residential mortgage securities, residential whole loans and MSR-related assets	(35,103)	(70,383)	(82,904)
Amortization of purchase premiums on residential mortgage securities and residential whole loans, and amortization of terminated hedging instruments	46,052	45,216	29,270
Provision for credit and valuation losses on residential whole loans and other financial instruments	22,121	2,569	773
Net valuation and other non-cash losses included in net income	44,055	24,815	19,208
Decrease/(increase) in other assets	39,930	(34,262)	(26,487)
(Decrease)/increase in other liabilities	(50,803)	18,553	32
Net cash provided by operating activities	38,396	215,775	147,881
Cash Flows From Investing Activities:
Purchases of residential whole loans, loan related investments and capitalized advances	(1,477,320)	(4,591,422)	(3,055,434)
Proceeds from sales of residential whole loans, and residential whole loan repurchases	1,510,902	(6,769)	(3,405)
Principal payments on residential whole loans and loan related investments	1,825,606	1,378,529	531,909
Purchases of residential mortgage securities and MSR-related assets	(163,748)	(1,008,215)	(2,604,234)
Proceeds from sales of residential mortgage securities, MSR-related assets, and other assets	3,790,148	908,697	538,668
Principal payments on residential mortgage securities and MSR-related assets	633,194	2,098,416	2,327,817
Purchases of real estate owned and capital improvements	(10,198)	(20,110)	(13,367)
Proceeds from sales of real estate owned	279,786	108,012	121,304
Additions to leasehold improvements, furniture and fixtures	(4,862)	(1,879)	(1,133)
Net cash provided by/(used in) investing activities	6,383,508	(1,134,741)	(2,157,875)
Cash Flows From Financing Activities:
Principal payments on financing agreements with mark-to-market collateral provisions	(21,810,920)	(67,463,756)	(67,063,283)
Proceeds from borrowings under financing agreements with mark-to-market collateral provisions	14,008,042	68,724,021	68,327,462
Principal payments on other collateralized financing agreements	(1,733,345)	(114,386)	(97,969)
Proceeds from borrowings under other collateralized financing agreements	3,803,150	0	419,970
Payment made for other collateralized financing agreement related costs	(1,699)	0	(2,497)
Proceeds from issuance of convertible senior notes	0	223,311	0
Payments made for settlements and unwinds of Swaps	(60,022)	(40,029)	(61,502)
Proceeds from settlements on Swaps	0	0	65,393
Proceeds from issuance of series C preferred stock	275,000	0	0
Payments made for costs related to series C preferred stock issuance	(8,948)	0	0
Proceeds from issuances of common stock	7,441	12,325	392,474
Payments made for costs related to common stock issuances	0	0	(329)
Payments made for the repurchase of common stock through the share repurchase program	(50,835)	0	0
Proceeds from the issuance of warrants	14,041	0	0
Payments made for the repurchase of warrants	(33,650)	0	0
Dividends paid on preferred stock	(29,796)	(15,000)	(15,000)
Dividends paid on common stock and dividend equivalents	(113,508)	(361,565)	(329,759)
Net cash (used in)/provided by financing activities	(5,735,049)	964,921	1,634,960
Net increase/(decrease) in cash, cash equivalents and restricted cash	686,855	45,955	(375,034)
Cash, cash equivalents and restricted cash at beginning of period	134,664	88,709	463,743
Cash, cash equivalents and restricted cash at end of period	821,519	134,664	88,709
Supplemental Disclosure of Cash Flow Information
Interest Paid	254,270	330,398	232,657
Non-cash Investing and Financing Activities:
Net decrease in securities obtained as collateral/obligation to return securities obtained as collateral	0	0	(505,850)
Transfer from residential whole loans to real estate owned	96,766	257,701	215,038
Dividends and dividend equivalents declared and unpaid	34,016	90,749	90,198
Dividends and dividend equivalents declared and unpaid	$ 0	$ 0	$ 211,129