Segment Reporting (Details) - USD ($) $ in Thousands	3 Months Ended						9 Months Ended
	Sep. 30, 2022	Jun. 30, 2022	Mar. 31, 2022	Sep. 30, 2021	Jun. 30, 2021	Mar. 31, 2021	Sep. 30, 2022	Sep. 30, 2021	Dec. 31, 2021
Segment Reporting Information [Line Items]
Interest Income	$ 123,872			$ 90,881			$ 339,542	$ 256,460
Interest Expense	71,579			29,065			171,617	84,690
Net Interest Income	52,293			61,816			167,925	171,770
Provision for Credit Losses on Other Assets	0			0			(28,579)	0
Reversal of Provision/(Provision) for Credit Losses on Residential Whole Loans	(588)			9,709			1,106	41,326
Net Interest Income after (Provision)/Reversal of Provision for Credit Losses	51,705			71,525			140,452	213,096
Net mark-to-market and other net loss on residential whole loans measured at fair value	(291,875)			21,815			(796,664)	59,325
Net gains on derivatives used for risk management purposes	111,816			2,085			253,721	1,028
Net mark-to-market on Securitized debt measured at fair value	98,858			857			247,548	8,254
Net gain on real estate owned	3,860			6,829			19,777	13,725
Gain on investment in Lima One common equity	0			38,933			0	38,933
Lima One - origination, servicing and other fee income	12,372			9,638			37,539	9,638
Other income/(loss), net	778			14,289			(10,076)	19,510
Other (Loss)/Income, net	(64,191)			94,446			(248,155)	150,413
General and administrative expenses (including compensation)	29,875			24,869			87,695	56,871
Loan servicing, financing and other related costs	11,357			5,291			34,993	18,591
Amortization of intangible assets	1,300			3,300			7,900	3,300
Net (Loss)/Income	(55,018)	$ (100,367)	$ (82,906)	132,511	$ 66,714	$ 85,522	(238,291)	284,747
Less Preferred Stock Dividend Requirement	8,218			8,218			24,656	24,656
Net (Loss)/Income Available to Common Stock and Participating Securities	(63,236)			124,293			(262,947)	260,091
Total Assets	9,519,846						9,519,846		$ 9,139,688
Mortgage-Related Assets
Segment Reporting Information [Line Items]
Interest Income	90,406			85,702			263,960	251,168
Interest Expense	47,780			23,346			120,678	71,037
Net Interest Income	42,626			62,356			143,282	180,131
Provision for Credit Losses on Other Assets							0
Reversal of Provision/(Provision) for Credit Losses on Residential Whole Loans	(520)			9,783			1,211	41,400
Net Interest Income after (Provision)/Reversal of Provision for Credit Losses	42,106			72,139			144,493	221,531
Net mark-to-market and other net loss on residential whole loans measured at fair value	(227,046)			10,475			(655,737)	47,985
Net gains on derivatives used for risk management purposes	86,944			2,085			217,340	1,028
Net mark-to-market on Securitized debt measured at fair value	79,471			857			202,016	8,254
Net gain on real estate owned	3,860			6,806			19,745	13,702
Gain on investment in Lima One common equity				0				0
Lima One - origination, servicing and other fee income	0			0			0	0
Other income/(loss), net	2,054			1,441			(52)	1,078
Other (Loss)/Income, net	(54,717)			21,664			(216,688)	72,047
General and administrative expenses (including compensation)	0			0			0	0
Loan servicing, financing and other related costs	6,063			5,128			19,508	18,426
Amortization of intangible assets	0			0			0	0
Net (Loss)/Income	(18,674)			88,675			(91,703)	275,152
Less Preferred Stock Dividend Requirement	0			0			0	0
Net (Loss)/Income Available to Common Stock and Participating Securities	(18,674)			88,675			(91,703)	275,152
Total Assets	6,644,958						6,644,958		7,567,084
Lima One
Segment Reporting Information [Line Items]
Interest Income	31,883			5,130			73,736	5,130
Interest Expense	19,856			1,790			39,128	1,790
Net Interest Income	12,027			3,340			34,608	3,340
Provision for Credit Losses on Other Assets							0
Reversal of Provision/(Provision) for Credit Losses on Residential Whole Loans	(68)			(74)			(105)	(74)
Net Interest Income after (Provision)/Reversal of Provision for Credit Losses	11,959			3,266			34,503	3,266
Net mark-to-market and other net loss on residential whole loans measured at fair value	(64,829)			11,340			(140,927)	11,340
Net gains on derivatives used for risk management purposes	24,872			0			36,381	0
Net mark-to-market on Securitized debt measured at fair value	19,387			0			45,532	0
Net gain on real estate owned	0			23			32	23
Gain on investment in Lima One common equity				0				0
Lima One - origination, servicing and other fee income	12,372			9,638			37,539	9,638
Other income/(loss), net	(60)			5			280	5
Other (Loss)/Income, net	(8,258)			21,006			(21,163)	21,006
General and administrative expenses (including compensation)	14,926			10,395			40,158	10,395
Loan servicing, financing and other related costs	280			163			839	163
Amortization of intangible assets	1,300			3,300			7,900	3,300
Net (Loss)/Income	(12,805)			10,414			(35,557)	10,414
Less Preferred Stock Dividend Requirement	0			0			0	0
Net (Loss)/Income Available to Common Stock and Participating Securities	(12,805)			10,414			(35,557)	10,414
Total Assets	2,353,657						2,353,657		1,200,737
Corporate
Segment Reporting Information [Line Items]
Interest Income	1,583			49			1,846	162
Interest Expense	3,943			3,929			11,811	11,863
Net Interest Income	(2,360)			(3,880)			(9,965)	(11,701)
Provision for Credit Losses on Other Assets							(28,579)
Reversal of Provision/(Provision) for Credit Losses on Residential Whole Loans	0			0			0	0
Net Interest Income after (Provision)/Reversal of Provision for Credit Losses	(2,360)			(3,880)			(38,544)	(11,701)
Net mark-to-market and other net loss on residential whole loans measured at fair value	0			0			0	0
Net gains on derivatives used for risk management purposes	0			0			0	0
Net mark-to-market on Securitized debt measured at fair value	0			0			0	0
Net gain on real estate owned	0			0			0	0
Gain on investment in Lima One common equity				38,933				38,933
Lima One - origination, servicing and other fee income	0			0			0	0
Other income/(loss), net	(1,216)			12,843			(10,304)	18,427
Other (Loss)/Income, net	(1,216)			51,776			(10,304)	57,360
General and administrative expenses (including compensation)	14,949			14,474			47,537	46,476
Loan servicing, financing and other related costs	5,014			0			14,646	2
Amortization of intangible assets	0			0			0	0
Net (Loss)/Income	(23,539)			33,422			(111,031)	(819)
Less Preferred Stock Dividend Requirement	8,218			8,218			24,656	24,656
Net (Loss)/Income Available to Common Stock and Participating Securities	(31,757)			$ 25,204			(135,687)	$ (25,475)
Total Assets	$ 521,231						$ 521,231		$ 371,867