Borrowings (Schedule of Debt Repayments) (Details) $ in Thousands	Dec. 31, 2023 USD ($)
Debt Instrument [Line Items]
Expected principal repayments related to consolidated residential mortgage loan securitizations	$ 980,300
Repurchase Agreements
Debt Instrument [Line Items]
Long-term Debt, Maturities, Repayments of Principal in Next Rolling Twelve Months	2,908,806
Long-term Debt, Maturities, Repayments of Principal in Rolling Year Two	58,631
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Three	0
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Four	0
Long-term Debt, Maturities, Repayments of Principal in Rolling after Year Five	0
Total Scheduled Repayment of Principal	2,967,437
Other Secured Borrowings
Debt Instrument [Line Items]
Long-term Debt, Maturities, Repayments of Principal in Next Rolling Twelve Months	387,887
Long-term Debt, Maturities, Repayments of Principal in Rolling Year Two	312,325
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Three	238,353
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Four	163,168
Long-term Debt, Maturities, Repayments of Principal in Rolling after Year Five	124,385
Total Scheduled Repayment of Principal	1,226,118
Unsecured Borrowings
Debt Instrument [Line Items]
Long-term Debt, Maturities, Repayments of Principal in Next Rolling Twelve Months	0
Long-term Debt, Maturities, Repayments of Principal in Rolling Year Two	34,931
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Three	37,750
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Four	210,000
Long-term Debt, Maturities, Repayments of Principal in Rolling after Year Five	0
Total Scheduled Repayment of Principal	282,681
Total
Debt Instrument [Line Items]
Long-term Debt, Maturities, Repayments of Principal in Next Rolling Twelve Months	4,549,836
Long-term Debt, Maturities, Repayments of Principal in Rolling Year Two	1,320,452
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Three	1,141,741
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Four	1,194,426
Long-term Debt, Maturities, Repayments of Principal in Rolling after Year Five	312,775
Total Scheduled Repayment of Principal	8,519,230
HMBS-related obligations
Debt Instrument [Line Items]
Long-term Debt, Maturities, Repayments of Principal in Next Rolling Twelve Months	1,253,143
Long-term Debt, Maturities, Repayments of Principal in Rolling Year Two	914,565
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Three	865,638
Long-Term Debt, Maturities, Repayments of Principal in Rolling Year Four	821,258
Long-term Debt, Maturities, Repayments of Principal in Rolling after Year Five	188,390
Total Scheduled Repayment of Principal	$ 4,042,994