Valuation (Significant Unobservable Inputs Rollforward) (Details) - Level 3 $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2022 USD ($)	Jun. 30, 2021 USD ($)	Jun. 30, 2022 USD ($)	Jun. 30, 2021 USD ($)	Dec. 31, 2021 USD ($)
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	$ 3,348,826	$ 1,998,486	$ 2,881,539	$ 1,959,802
Assets, Accreted Discounts/Amortized Premiums	(6,957)	(6,514)	(15,351)	(10,478)
Assets, Realized Gain (Loss)	2,799	1,957	10,253	3,344
Assets, Change In Net Unrealized Gain/(Loss)	(152,622)	28,673	(238,400)	47,598
Assets, Purchases	1,057,895	526,832	2,229,118	891,873
Assets, Sales	(320,222)	(350,732)	(924,856)	(671,899)
Assets, Transfers into Level 3	26,331	13,806	38,365	12,005
Assets, Transfers out of Level 3	(5,937)	(31,396)	(30,555)	(51,133)
Assets, Ending Balance	3,950,113	2,181,112	3,950,113	2,181,112
Liabilities, Begining Balance	(1,426,542)	(911,594)	(984,168)	(755,405)
Liabilities, Accreted Discounts/Amortized Premiums	0	0	0	0
Liabilities, Realized Gain/(Loss)	0	(860)	0	(1,360)
Liabilities, Change In Net Unrealized Gain/(Loss)	74,608	2,479	130,249	3,805
Liabilities Purchases/Payments	99,198	231,521	213,953	325,014
Liabilities, Sales/Issuance	(398,096)	(324,639)	(1,010,866)	(575,147)
Liabilities, Transfers into Level 3	0	0	0	0
Liabilities, Transfers out of Level 3	0	0	0	0
Liabilities, Ending Balance	(1,650,832)	(1,003,093)	(1,650,832)	(1,003,093)
Non-Agency RMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	131,449		131,449		$ 134,888
Non-Agency RMBS | Valuation, Income Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	44,317		44,317		45,544
Non-Agency RMBS | Valuation, Market Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	87,132		87,132		89,344
CMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	9,380		9,380		13,134
CMBS | Valuation, Income Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	1,926		1,926		268
CMBS | Valuation, Market Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	7,454		7,454		12,866
CLOs
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	23,378		23,378		26,678
CLOs | Valuation, Income Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	8,128		8,128		8,014
CLOs | Valuation, Market Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	15,250		15,250		18,664
Asset-backed securities, backed by consumer loans | Valuation, Income Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	78,183		78,183		73,108
Residential mortgage loans | Performing Financial Instruments [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets					952,656
Residential mortgage loans | Valuation, Income Approach | Performing Financial Instruments [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	1,414,837		1,414,837		951,723
Residential mortgage loans | Measurement Input, Exercise Price | Valuation, Cost Approach | Performing Financial Instruments [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets					933
Commercial mortgage loans | Valuation, Income Approach | Performing Financial Instruments [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	445,913		445,913		310,735
Consumer loans | Valuation, Income Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	7,410		7,410		62,365
Corporate loan [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	10,448		10,448		10,531
Corporate loan [Member] | Valuation, Income Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	3,448		3,448		3,531
Corporate loan [Member] | Valuation, Market Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	7,000		7,000		7,000
Credit default swaps on asset-backed securities | Valuation, Income Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	202		202		303
Other secured borrowings, at fair value | Valuation, Market Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Fair value	$ (1,448,182)		$ (1,448,182)		$ (984,168)
Other secured borrowings, at fair value | Measurement Input, Quoted Price | Valuation, Market Approach | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt instrument, measurement input	57.86		57.86		93.34
Other secured borrowings, at fair value | Measurement Input, Quoted Price | Valuation, Market Approach | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt instrument, measurement input	98.79		98.79		102.14
Other secured borrowings, at fair value | Measurement Input, Quoted Price | Valuation, Market Approach | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt instrument, measurement input	91.80		91.80		99.94
Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	$ 192,269		$ 192,269		$ 195,643
Investment in unconsolidated entities | Measurement Input, Net Asset Value | Enterprise Value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	80,579		80,579		57,828
Investment in unconsolidated entities | Measurement Input, Exercise Price | Valuation, Cost Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Total assets	12,931		12,931		$ 14,036
Senior Notes | Valuation, Market Approach
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Fair value	$ (202,650)		$ (202,650)
Senior Notes | Measurement Input, Quoted Price | Valuation, Market Approach | Minimum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt instrument, measurement input	96.50		96.50
Senior Notes | Measurement Input, Quoted Price | Valuation, Market Approach | Maximum
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt instrument, measurement input	96.50		96.50
Senior Notes | Measurement Input, Quoted Price | Valuation, Market Approach | Weighted Average
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt instrument, measurement input	96.50		96.50
Other secured borrowings, at fair value
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Liabilities, Payments		230,661		323,654
Liabilities, Issuances		(324,639)		(575,147)
Liabilities, Begining Balance	$ (1,216,542)	(911,256)	$ (984,168)	(754,921)
Liabilities, Accreted Discounts/Amortized Premiums	0	0	0	0
Liabilities, Realized Gain/(Loss)	0	0	0	0
Liabilities, Change In Net Unrealized Gain/(Loss)	67,258	2,197	122,899	3,377
Liabilities Purchases/Payments	99,198		213,953
Liabilities, Sales/Issuance	(398,096)		(800,866)
Liabilities, Transfers into Level 3	0	0	0	0
Liabilities, Transfers out of Level 3	0	0	0	0
Liabilities, Ending Balance	(1,448,182)	(1,003,037)	(1,448,182)	(1,003,037)
Change in unrealized gains (losses), liabilities	67,300	2,200	122,900	3,200
Financial Derivatives - Liabilities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Change in unrealized gains (losses), liabilities		300		400
Senior notes
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Liabilities, Begining Balance	(210,000)		0
Liabilities, Accreted Discounts/Amortized Premiums	0		0
Liabilities, Realized Gain/(Loss)	0		0
Liabilities, Change In Net Unrealized Gain/(Loss)	7,350		7,350
Liabilities Purchases/Payments	0		0
Liabilities, Sales/Issuance	0		(210,000)
Liabilities, Transfers into Level 3	0		0
Liabilities, Transfers out of Level 3	0		0
Liabilities, Ending Balance	(202,650)		(202,650)
Senior Notes
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Change in unrealized gains (losses), liabilities	7,400		7,400
Securities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Change in unrealized gain (loss), assets	(17,900)	10,400	(22,600)	11,900
Securities | Agency RMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	8,621	17,402	9,710	11,663
Assets, Accreted Discounts/Amortized Premiums	(672)	(833)	(1,088)	(1,779)
Assets, Realized Gain (Loss)	(250)	1,510	133	959
Assets, Change In Net Unrealized Gain/(Loss)	206	(1,389)	(1,182)	(1,571)
Assets, Purchases	441	0	840	555
Assets, Sales	0	(4,528)	(514)	(1,533)
Assets, Transfers into Level 3	4,043	1,897	4,805	4,567
Assets, Transfers out of Level 3	0	(4,123)	(315)	(2,925)
Assets, Ending Balance	12,389	9,936	12,389	9,936
Securities | Non-Agency RMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	116,776	127,329	134,888	127,838
Assets, Accreted Discounts/Amortized Premiums	208	604	783	1,248
Assets, Realized Gain (Loss)	2,076	2,576	1,951	2,351
Assets, Change In Net Unrealized Gain/(Loss)	(6,407)	(137)	(9,458)	(740)
Assets, Purchases	15,577	1,782	18,614	22,392
Assets, Sales	(5,096)	(14,497)	(12,160)	(33,076)
Assets, Transfers into Level 3	12,048	4,299	17,710	5,834
Assets, Transfers out of Level 3	(3,733)	(7,492)	(20,879)	(11,383)
Assets, Ending Balance	131,449	114,464	131,449	114,464
Securities | CMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	9,526	17,294	13,134	63,148
Assets, Accreted Discounts/Amortized Premiums	32	(13)	61	319
Assets, Realized Gain (Loss)	(24)	0	1,118	1,931
Assets, Change In Net Unrealized Gain/(Loss)	(533)	1,249	(1,281)	5,029
Assets, Purchases	0	0	620	0
Assets, Sales	0	0	(2,234)	(41,088)
Assets, Transfers into Level 3	379	3,353	5,119	0
Assets, Transfers out of Level 3	0	4,858	7,157	12,314
Assets, Ending Balance	9,380	17,025	9,380	17,025
Securities | CLOs
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	22,824	37,585	26,678	111,100
Assets, Accreted Discounts/Amortized Premiums	(557)	272	(1,420)	1,240
Assets, Realized Gain (Loss)	2,067	(2,497)	3,020	(1,754)
Assets, Change In Net Unrealized Gain/(Loss)	(3,915)	9,155	(2,947)	16,208
Assets, Purchases	26	140	0	104
Assets, Sales	(4,724)	0	(10,480)	(70,002)
Assets, Transfers into Level 3	9,861	4,257	10,731	1,604
Assets, Transfers out of Level 3	(2,204)	(14,923)	(2,204)	(24,511)
Assets, Ending Balance	23,378	33,989	23,378	33,989
Securities | Asset-backed securities, backed by consumer loans
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	76,504	59,473	73,108	44,925
Assets, Accreted Discounts/Amortized Premiums	(1,040)	(1,326)	(2,153)	(1,983)
Assets, Realized Gain (Loss)	(26)	146	(299)	179
Assets, Change In Net Unrealized Gain/(Loss)	(3,608)	(1,098)	(5,632)	(1,583)
Assets, Purchases	17,444	21,478	36,237	45,812
Assets, Sales	(11,091)	(9,419)	(23,078)	(18,096)
Assets, Transfers into Level 3	0	0	0	0
Assets, Transfers out of Level 3	0	0	0	0
Assets, Ending Balance	78,183	69,254	78,183	69,254
Securities | Corporate debt securities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	500	4,761	5,198	4,082
Assets, Accreted Discounts/Amortized Premiums	0	0	0	0
Assets, Realized Gain (Loss)	(1)	47	1,533	227
Assets, Change In Net Unrealized Gain/(Loss)	(172)	177	(1,680)	183
Assets, Purchases	3,584	591	5,312	1,617
Assets, Sales	(2,144)	(132)	(8,596)	(665)
Assets, Transfers into Level 3	0	0	0	0
Assets, Transfers out of Level 3	0	0	0	0
Assets, Ending Balance	1,767	5,444	1,767	5,444
Securities | Corporate equity securities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	9,841	4,120	7,556	1,590
Assets, Accreted Discounts/Amortized Premiums	0	0	0	0
Assets, Realized Gain (Loss)	114	0	1,739	(385)
Assets, Change In Net Unrealized Gain/(Loss)	(535)	1,711	(1,365)	2,315
Assets, Purchases	856	2,666	4,983	4,977
Assets, Sales	0	0	(2,637)	0
Assets, Transfers into Level 3	0	0	0	0
Assets, Transfers out of Level 3	0	0	0	0
Assets, Ending Balance	10,276	8,497	10,276	8,497
Loans
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Change in unrealized gain (loss), assets	(114,000)	100	(184,400)	2,300
Loans | Residential mortgage loans
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	2,433,007	1,280,637	2,016,228	1,187,069
Assets, Accreted Discounts/Amortized Premiums	(4,544)	(3,240)	(9,011)	(5,711)
Assets, Realized Gain (Loss)	105	267	1,617	462
Assets, Change In Net Unrealized Gain/(Loss)	(112,735)	(528)	(183,248)	1,686
Assets, Purchases	896,384	327,697	1,619,479	549,778
Assets, Sales	(216,633)	(159,182)	(449,481)	(287,633)
Assets, Transfers into Level 3	0	0	0	0
Assets, Transfers out of Level 3	0	0	0	0
Assets, Ending Balance	2,995,584	1,445,651	2,995,584	1,445,651
Loans | Commercial mortgage loans
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	429,954	235,948	326,197	213,031
Assets, Accreted Discounts/Amortized Premiums	0	15	0	24
Assets, Realized Gain (Loss)	0	(130)	10	306
Assets, Change In Net Unrealized Gain/(Loss)	(1,388)	334	(1,224)	70
Assets, Purchases	66,164	90,332	333,806	147,499
Assets, Sales	(17,352)	(98,049)	(181,411)	(132,480)
Assets, Transfers into Level 3	0	0	0	0
Assets, Transfers out of Level 3	0	0	0	0
Assets, Ending Balance	477,378	228,450	477,378	228,450
Loans | Consumer loans
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	9,878	52,705	62,365	47,525
Assets, Accreted Discounts/Amortized Premiums	(384)	(1,993)	(2,523)	(3,836)
Assets, Realized Gain (Loss)	(518)	(11)	(698)	(1,276)
Assets, Change In Net Unrealized Gain/(Loss)	94	129	(371)	457
Assets, Purchases	37	9,319	10,982	26,068
Assets, Sales	(1,697)	(8,133)	(62,345)	(16,922)
Assets, Transfers into Level 3	0	0	0	0
Assets, Transfers out of Level 3	0	0	0	0
Assets, Ending Balance	7,410	52,016	7,410	52,016
Loans | Corporate loan [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	11,788	13,226	10,531	5,855
Assets, Accreted Discounts/Amortized Premiums	0	0	0	0
Assets, Realized Gain (Loss)	(1,000)	0	(1,000)	0
Assets, Change In Net Unrealized Gain/(Loss)	0	0	0	0
Assets, Purchases	675	3,360	2,325	10,731
Assets, Sales	(1,015)	(2)	(1,408)	(2)
Assets, Transfers into Level 3	0	0	0	0
Assets, Transfers out of Level 3	0	0	0	0
Assets, Ending Balance	10,448	16,584	10,448	16,584
Investment in unconsolidated entities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	219,303	147,684	195,643	141,620
Assets, Accreted Discounts/Amortized Premiums	0	0	0	0
Assets, Realized Gain (Loss)	262	33	1,140	161
Assets, Change In Net Unrealized Gain/(Loss)	(23,527)	18,569	(29,911)	25,076
Assets, Purchases	56,701	69,464	195,909	82,333
Assets, Sales	(60,470)	(56,771)	(170,512)	(70,211)
Assets, Transfers into Level 3	0	0	0	0
Assets, Transfers out of Level 3	0	0	0	0
Assets, Ending Balance	192,269	178,979	192,269	178,979
Change in unrealized gain (loss), assets	(26,200)	16,400	(34,000)	22,700
Financial Derivatives - Assets
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Change in unrealized gain (loss), assets	(100)	400	(100)	500
Financial Derivatives - Assets | Total return swaps
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance		2		9
Assets, Accreted Discounts/Amortized Premiums		0		0
Assets, Realized Gain (Loss)		0		141
Assets, Change In Net Unrealized Gain/(Loss)		518		512
Assets, Purchases		0		0
Assets, Sales		0		(142)
Assets, Transfers into Level 3		0		0
Assets, Transfers out of Level 3		0		0
Assets, Ending Balance		520		520
Financial Derivatives - Assets | Credit default swaps on asset-backed securities
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	304	320	303	347
Assets, Accreted Discounts/Amortized Premiums	0	0	0	0
Assets, Realized Gain (Loss)	(6)	16	(11)	42
Assets, Change In Net Unrealized Gain/(Loss)	(102)	(17)	(101)	(44)
Assets, Purchases	6	3	11	7
Assets, Sales	0	(19)	0	(49)
Assets, Transfers into Level 3	0	0	0	0
Assets, Transfers out of Level 3	0	0	0	0
Assets, Ending Balance	$ 202	303	$ 202	303
Financial Derivatives - Liabilities | Total return swaps
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Liabilities, Payments		860		1,360
Liabilities, Issuances		0		0
Liabilities, Begining Balance		(338)		(484)
Liabilities, Accreted Discounts/Amortized Premiums		0		0
Liabilities, Realized Gain/(Loss)		(860)		(1,360)
Liabilities, Change In Net Unrealized Gain/(Loss)		282		428
Liabilities, Transfers into Level 3		0		0
Liabilities, Transfers out of Level 3		0		0
Liabilities, Ending Balance		$ (56)		$ (56)