Residential Whole Loans - Narrative (Details) - USD ($) $ in Millions	3 Months Ended
	Mar. 31, 2023	Dec. 31, 2022
Receivables [Abstract]
Total residential whole loans	$ 7,800.0	$ 7,500.0
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest income	3.6
Nonperforming
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loss on sale of financing receivable		72.3
Financing receivable, nonaccrual	389.1	413.1
Interest income	2.5
Agency eligible investor loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, sale		337.8
Agency eligible investor loans | Nonperforming
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, sale		598.0
Purchased Performing Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, nonaccrual	192.5	195.1
Purchased Credit Deteriorated Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, nonaccrual	79.4	80.5
Without Associated Credit Losses
Accounts, Notes, Loans and Financing Receivable [Line Items]
Financing receivable, nonaccrual	$ 70.2	$ 71.7