Fair Value of Financial Instruments - Fair Value Methodology (Details) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2023 USD ($)	Dec. 31, 2022 USD ($)
Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	$ 774,852	$ 795,894
Purchases excluded from level 2 fair value	515	215
Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	5,238,011	4,880,321
Discounted cash flow | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	538,661	546,675
Discounted cash flow | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	5,196,080	4,857,587
Liquidation model | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	236,191	249,219
Simple average amount	455	457
Liquidation model | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	$ 41,931	$ 22,734
Residential whole loans, at fair value | Level 3 | Liquidation model | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	(8.60%)	(5.40%)
Liquidation timeline (in years)	1 month 6 days	1 month 6 days
Fair value inputs, value of underlying property	$ 28	$ 28
Residential whole loans, at fair value | Level 3 | Liquidation model | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	(3.60%)	(1.00%)
Liquidation timeline (in years)	9 months 18 days	9 months 18 days
Fair value inputs, value of underlying property	$ 44	$ 50
Residential whole loans, at fair value | Level 3 | Liquidation model | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	39.90%	59.70%
Liquidation timeline (in years)	4 years 6 months	4 years 6 months
Fair value inputs, value of underlying property	$ 4,000	$ 4,000
Residential whole loans, at fair value | Level 3 | Liquidation model | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	9.50%	10.70%
Liquidation timeline (in years)	4 years 2 months 12 days	4 years 2 months 12 days
Fair value inputs, value of underlying property	$ 2,850	$ 2,850
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.060	0.063
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.062	0.056
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.103	0.100
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.191	0.227
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.067	0.070
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.074	0.076
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.078	0.078
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.078	0.078
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.078	0.078
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.078	0.078
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.078	0.078
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.078	0.078
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.326	0.335
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.425	0.448
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.094	0.089
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.089	0.079
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.511	0.524
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.277	0.194
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.033	0.037
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.008	0.008
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	1.000	1.000
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.204	1.000
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.113	0.113
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.072	0.073
Annual change in home prices | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	5.30%	6.90%
Annual change in home prices | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	2.30%	3.20%
Liquidation timeline (in years) | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	1 year 10 months 24 days	1 year 10 months 24 days
Liquidation timeline (in years) | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	1 year 10 months 24 days	1 year 10 months 24 days
Current value of underlying properties | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, value of underlying property	$ 738	$ 743
Current value of underlying properties | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, value of underlying property	$ 1,238	$ 1,319