Residential Whole Loans - Residential Whole Loans, at Carrying and Fair Value (Details) $ in Thousands	3 Months Ended
	Dec. 31, 2023 USD ($) loan	Dec. 31, 2022 USD ($) loan
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	$ 1,550,234
Allowance for Credit Losses	(20,451)	$ (35,314)
Total Residential Whole Loans	$ 9,041,292	$ 7,518,739
Number of loans | loan	25,401	23,843
Held at Carrying Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance for Credit Losses	$ (20,451)	$ (35,314)
Total Residential Whole Loans	$ 1,529,784	$ 1,791,215
Number of loans | loan	6,326	7,126
Held at Fair Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance for Credit Losses	$ 0	$ 0
Total Residential Whole Loans	$ 7,511,508	$ 5,727,524
Number of loans | loan	19,075	16,717
Performing
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	$ 7,926,593	$ 6,287,650
Performing | Held at Carrying Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	1,120,509	1,356,235
Performing | Held at Fair Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	6,806,084	4,931,415
Nonperforming
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	705,424	796,109
Nonperforming | Held at Carrying Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	0	0
Nonperforming | Held at Fair Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	705,424	796,109
Non-QM loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	843,883
Non-QM loans | Performing
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	3,805,577	3,359,830
Non-QM loans | Performing | Held at Carrying Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	843,884	987,282
Non-QM loans | Performing | Held at Fair Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	2,961,693	2,372,548
Transitional loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	35,467
Transitional loans | Performing
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	2,361,496	1,417,220
Transitional loans | Performing | Held at Carrying Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	35,467	75,188
Transitional loans | Performing | Held at Fair Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	2,326,029	1,342,032
Single-family rental loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	172,213
Single-family rental loans | Performing
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	1,634,796	1,376,574
Single-family rental loans | Performing | Held at Carrying Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	172,213	210,833
Single-family rental loans | Performing | Held at Fair Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	1,462,583	1,165,741
Seasoned performing loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	68,945
Seasoned performing loans | Performing
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	68,945	82,932
Seasoned performing loans | Performing | Held at Carrying Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	68,945	82,932
Seasoned performing loans | Performing | Held at Fair Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	0	0
Agency eligible investor loans | Performing
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	55,779	51,094
Agency eligible investor loans | Performing | Held at Carrying Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	0	0
Agency eligible investor loans | Performing | Held at Fair Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	55,779	51,094
Purchased Credit Deteriorated Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	429,726	470,294
Purchased Credit Deteriorated Loans | Held at Carrying Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	429,726	470,294
Purchased Credit Deteriorated Loans | Held at Fair Value
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	0	0
Single Family Residential Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	1,200,000	784,900
Multi-Family Transitional Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	1,200,000	632,300
New Construction Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	471,100	$ 283,100
Residential Whole Loans | Level 3
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans, fair value	13,600
Loss from changes in fair value	$ 1,200