Fair Value of Financial Instruments - Fair Value Methodology (Details) $ in Thousands	6 Months Ended	12 Months Ended
	Jun. 30, 2022 USD ($)	Dec. 31, 2021 USD ($)
Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	$ 921,690	$ 1,071,774
Purchases excluded from level 2 fair value	368	496
Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	4,329,990	3,150,314
Discounted cash flow | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	623,394	720,766
Discounted cash flow | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	4,311,462	3,142,366
Liquidation model | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	298,296	351,008
Simple average amount	441	421
Liquidation model | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased loans	$ 18,528	$ 7,948
Residential whole loans, at fair value | Level 3 | Liquidation model | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	3.40%	4.50%
Liquidation timeline (in years)	1 month 6 days	1 month 6 days
Fair value inputs, value of underlying property	$ 30	$ 10
Residential whole loans, at fair value | Level 3 | Liquidation model | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	0.00%	0.00%
Liquidation timeline (in years)	9 months 18 days	9 months 18 days
Fair value inputs, value of underlying property	$ 52	$ 60
Residential whole loans, at fair value | Level 3 | Liquidation model | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	19.60%	21.90%
Liquidation timeline (in years)	4 years 6 months	4 years 6 months
Fair value inputs, value of underlying property	$ 3,763	$ 3,995
Residential whole loans, at fair value | Level 3 | Liquidation model | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	16.50%	14.80%
Liquidation timeline (in years)	4 years 2 months 12 days	4 years 2 months 12 days
Fair value inputs, value of underlying property	$ 3,470	$ 1,750
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.037	0.015
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.047	0.014
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.097	0.098
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.271	0.259
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.057	0.036
Discount rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.063	0.039
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.067	0.067
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.070	0.070
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.500	0.500
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.070	0.070
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.079	0.080
Discount rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.070	0.070
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.384	0.440
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.576	0.472
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.101	0.144
Prepayment rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.125	0.190
Prepayment rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	9.20%	9.70%
Prepayment rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, annual change in home prices	11.30%	6.50%
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.352	0.508
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.184	0.178
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.035	0.039
Default rate | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.004	0.002
Default rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	1 year 10 months 24 days	1 year 8 months 12 days
Default rate | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	1 year 9 months 18 days	2 years
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.000	0.000
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	1.000	1.000
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.100	0.100
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.116	0.117
Loss severity | Residential whole loans, at fair value | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input	0.078	0.084
Loss severity | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, value of underlying property	$ 765	$ 770
Loss severity | Residential whole loans, at fair value | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair value inputs, value of underlying property	$ 1,487	$ 691