Fair Value of Financial Instruments - Fair Value Methodology (Details) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2024 USD ($)	Dec. 31, 2023 USD ($)
Residential whole loans, total or weighted average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans, fair value	$ 14,800	$ 103,700
Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased performing loans, fair value	6,902,683	6,646,651
Performing | Discounted cash flow
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased performing loans, fair value	6,751,137	6,522,457
Performing | Liquidation model
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased performing loans, fair value	$ 151,546	$ 124,194
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Discount rate | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.082	0.080
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Discount rate | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.062	0.065
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Discount rate | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.215	0.292
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Prepayment rate | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.119	0.101
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Prepayment rate | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Prepayment rate | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.553	0.464
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Default rate | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.005	0.005
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Default rate | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Default rate | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.311	0.273
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Loss severity | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.109	0.109
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Loss severity | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Performing | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Loss severity | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.990	0.990
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Discount rate | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.080	0.080
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Discount rate | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.080	0.080
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Discount rate | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.080	0.080
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Annual change in home prices | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Annual change in home prices	2.30%	2.60%
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Annual change in home prices | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Annual change in home prices	0.00%	0.00%
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Annual change in home prices | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Annual change in home prices	10.10%	10.10%
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Liquidation timeline (in years) | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	1 year 6 months	1 year 7 months 6 days
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Liquidation timeline (in years) | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	9 months 18 days	9 months 18 days
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Liquidation timeline (in years) | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	3 years 10 months 24 days	3 years 10 months 24 days
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Current value of underlying properties | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Current value of underlying properties	$ 1,647	$ 1,580
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Current value of underlying properties | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Current value of underlying properties	21	35
Performing | Residential whole loans, at fair value | Level 3 | Liquidation model | Current value of underlying properties | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Current value of underlying properties	7,400	5,500
Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased performing loans, fair value	681,188	704,852
Purchases excluded from level 2 fair value	601	572
Nonperforming | Discounted cash flow
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased performing loans, fair value	519,497	537,528
Nonperforming | Liquidation model
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased performing loans, fair value	161,691	167,324
Simple average amount	$ 512	$ 494
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Discount rate | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.070	0.068
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Discount rate | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.065	0.062
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Discount rate | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.109	0.102
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Prepayment rate | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.094	0.097
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Prepayment rate | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Prepayment rate | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.347	0.389
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Default rate | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.022	0.021
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Default rate | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Default rate | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.417	0.395
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Loss severity | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.096	0.097
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Loss severity | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Nonperforming | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Loss severity | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	1.000	1.000
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Discount rate | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.080	0.080
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Discount rate | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.080	0.080
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Discount rate | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.080	0.080
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Annual change in home prices | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Annual change in home prices	4.30%	4.60%
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Annual change in home prices | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Annual change in home prices	(0.30%)	(0.40%)
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Annual change in home prices | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Annual change in home prices	12.10%	12.70%
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Liquidation timeline (in years) | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	2 years 1 month 6 days	2 years 1 month 6 days
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Liquidation timeline (in years) | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	9 months 18 days	1 month 6 days
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Liquidation timeline (in years) | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	4 years 6 months	4 years 6 months
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Current value of underlying properties | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Current value of underlying properties	$ 863	$ 831
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Current value of underlying properties | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Current value of underlying properties	24	24
Nonperforming | Residential whole loans, at fair value | Level 3 | Liquidation model | Current value of underlying properties | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Current value of underlying properties	$ 4,720	$ 4,720