Fair Value of Financial Instruments (Fair Value Methodology) (Details) $ in Thousands	3 Months Ended	9 Months Ended
	Mar. 31, 2021 USD ($)	Sep. 30, 2021 USD ($)	Dec. 31, 2020 USD ($)
Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased Loans		$ 1,139,743	$ 1,216,637
Purchases excluded from level 2 fair value		1,100	265
Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased Loans		2,790,512
Purchases excluded from level 2 fair value		162,200
Discounted cash flow | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased Loans		774,666	789,576
Discounted cash flow | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased Loans		2,780,698
Liquidation model | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased Loans		365,077	427,061
Simple average amount		432	$ 380
Liquidation model | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Purchased Loans		$ 9,814
Residential whole loans | Level 3 | Liquidation model | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value Inputs, Annual Change in Home Prices	1.80%	4.60%
Liquidation timeline (in years)	9 months 18 days	1 month 6 days
Fair Value Inputs, Value of Underlying Property	$ 12	$ 10
Residential whole loans | Level 3 | Liquidation model | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value Inputs, Annual Change in Home Prices		0.00%
Liquidation timeline (in years)		1 year
Fair Value Inputs, Value of Underlying Property		$ 50
Residential whole loans | Level 3 | Liquidation model | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value Inputs, Annual Change in Home Prices	6.50%	27.30%
Liquidation timeline (in years)	4 years 9 months 18 days	4 years 9 months 18 days
Fair Value Inputs, Value of Underlying Property	$ 4,500	$ 3,995
Residential whole loans | Level 3 | Liquidation model | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value Inputs, Annual Change in Home Prices		12.40%
Liquidation timeline (in years)		4 years 4 months 24 days
Fair Value Inputs, Value of Underlying Property		$ 2,075
Discount rate | Residential whole loans | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.025	0.033
Discount rate | Residential whole loans | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.009
Discount rate | Residential whole loans | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.115	0.080
Discount rate | Residential whole loans | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.505
Discount rate | Residential whole loans | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.034	0.039
Discount rate | Residential whole loans | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.033
Discount rate | Residential whole loans | Level 3 | Liquidation model | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.067	0.067
Discount rate | Residential whole loans | Level 3 | Liquidation model | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.070
Discount rate | Residential whole loans | Level 3 | Liquidation model | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.500	0.500
Discount rate | Residential whole loans | Level 3 | Liquidation model | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.070
Discount rate | Residential whole loans | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.081	0.081
Discount rate | Residential whole loans | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.070
Prepayment rate | Residential whole loans | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.000	0.000
Prepayment rate | Residential whole loans | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.000
Prepayment rate | Residential whole loans | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.438	0.176
Prepayment rate | Residential whole loans | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.501
Prepayment rate | Residential whole loans | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.122	0.054
Prepayment rate | Residential whole loans | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.184
Prepayment rate | Residential whole loans | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value Inputs, Annual Change in Home Prices	3.60%	9.30%
Prepayment rate | Residential whole loans | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value Inputs, Annual Change in Home Prices		7.80%
Default rate | Residential whole loans | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.000	0.000
Default rate | Residential whole loans | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.000
Default rate | Residential whole loans | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.313	0.477
Default rate | Residential whole loans | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.218
Default rate | Residential whole loans | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.040	0.041
Default rate | Residential whole loans | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.003
Default rate | Residential whole loans | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)	1 year 9 months 18 days	1 year 8 months 12 days
Default rate | Residential whole loans | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Liquidation timeline (in years)		2 years 2 months 12 days
Loss severity | Residential whole loans | Level 3 | Discounted cash flow | Minimum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.000	0.000
Loss severity | Residential whole loans | Level 3 | Discounted cash flow | Minimum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.000
Loss severity | Residential whole loans | Level 3 | Discounted cash flow | Maximum | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		1.000	1.000
Loss severity | Residential whole loans | Level 3 | Discounted cash flow | Maximum | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.100
Loss severity | Residential whole loans | Level 3 | Discounted cash flow | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.118	0.127
Loss severity | Residential whole loans | Level 3 | Discounted cash flow | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Measurement input		0.089
Loss severity | Residential whole loans | Level 3 | Liquidation model | Weighted Average | Nonperforming
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value Inputs, Value of Underlying Property	$ 729	$ 788
Loss severity | Residential whole loans | Level 3 | Liquidation model | Weighted Average | Performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value Inputs, Value of Underlying Property		$ 884