Fair Value Measurements - Changes in the Fair Value of Securities Classified as Level 3 (Details) - USD ($) $ in Thousands	3 Months Ended	12 Months Ended
	Dec. 31, 2022	Dec. 31, 2022	Dec. 31, 2021
Fair Value, Liabilities Measured on Recurring Basis [Roll Forward]
Fair Value, Asset, Recurring Basis, Unobservable Input Reconciliation, Asset, Gain (Loss), Statement of Other Comprehensive Income or Comprehensive Income [Extensible Enumeration]		Net unrealized gains (losses) on financial instruments at fair value
Securitized Debt | Variable Interest Entities, Primary Beneficiary | Level 3
Fair Value, Liabilities Measured on Recurring Basis [Roll Forward]
Securitized Debt, Beginning balance, Level 3		$ 7,726,043	$ 8,711,677
Securitized Debt, Transfers in to Level 3 assets		0	0
Securitized Debt, Transfers out of Level 3 assets		0	0
Securitized Debt, Transfer due to consolidation		774,514
Securitized Debt, Purchases of assets/ issuance of debt		1,122,982	5,521,953
Securitized Debt, Principal payments		(1,844,397)	(2,247,983)
Securitized Debt, Sales and Settlements		0	(4,192,295)
Securitized Debt, Net accretion (amortization)		5,021	12,010
Securitized Debt, Other than temporary credit impairment losses		0	0
Securitized Debt, Realized gains (losses) on sales and settlements		0	258,903
Securitized Debt, Net unrealized gains (losses) included in income		(683,421)	(338,222)
Securitized Debt, Gains (losses) included in other comprehensive income		0	0
Securitized Debt, Ending balance, Level 3	$ 7,100,742	7,100,742	7,726,043
Non-Agency RMBS | Level 3
Fair Value, Assets Measured on Recurring Basis [Roll Forward]
Beginning balance Level 3		1,810,208	2,150,714
Transfers into Level 3		0	0
Transfers out of Level 3		0	0
Transfer due to consolidation		(218,276)
Purchases of assets/ issuance of debt		23,589	34,656
Principal payments		(178,300)	(299,330)
Sales and settlements		(23,056)	(47,674)
Net accretion (amortization)		31,387	57,473
Gains (losses) included in net income
(Increase) decrease in provision for credit losses		(7,036)	(33)
Realized gains (losses) on sales and settlements		(15,594)	32,807
Net unrealized gains (losses) included in income		(104,631)	31,358
Gains (losses) included in other comprehensive income
Total unrealized gains (losses) for the period		(170,810)	(149,763)
Ending balance Level 3	1,147,481	1,147,481	1,810,208
Loans held for investment
Fair Value, Assets Measured on Recurring Basis [Roll Forward]
Transfers out of Level 3	0	0	(272,000)
Loans held for investment | Variable Interest Entities, Primary Beneficiary
Fair Value, Assets Measured on Recurring Basis [Roll Forward]
Sales and settlements		(5,369)	(1,679,280)
Gains (losses) included in net income
Net unrealized gains (losses) included in income		(1,347,475)	196,603
Loans held for investment | Variable Interest Entities, Primary Beneficiary | Level 3
Fair Value, Assets Measured on Recurring Basis [Roll Forward]
Beginning balance Level 3		12,032,299	13,112,129
Transfers into Level 3		0	0
Transfers out of Level 3		0	(272,198)
Transfer due to consolidation		1,047,838
Purchases of assets/ issuance of debt		1,429,503	3,248,683
Principal payments		(1,953,098)	(2,495,015)
Sales and settlements		(5,368)	(1,679,280)
Net accretion (amortization)		(52,767)	(79,223)
Gains (losses) included in net income
(Increase) decrease in provision for credit losses		0	0
Realized gains (losses) on sales and settlements		0	0
Net unrealized gains (losses) included in income		(1,343,807)	197,203
Gains (losses) included in other comprehensive income
Total unrealized gains (losses) for the period		0	0
Ending balance Level 3	$ 11,154,600	$ 11,154,600	$ 12,032,299