RESIDENTIAL MORTGAGE LOANS - Residential Mortgage Loans Outstanding by Loan Type, Excluding REO (Details) $ in Thousands	9 Months Ended
	Sep. 30, 2023 USD ($) loan	Dec. 31, 2022 USD ($)
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Weighted Average Life (Years)	1 year
Residential mortgage loans held-for-investment, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 462,187
Carrying Value	$ 370,957	$ 452,519
Loan Count | loan	8,610
Weighted Average Yield	8.90%
Weighted Average Life (Years)	4 years 6 months
Total residential mortgage loans, held-for-sale, at lower of cost or market
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 96,395
Carrying Value	$ 78,683	101,027
Loan Count | loan	2,271
Weighted Average Yield	9.00%
Weighted Average Life (Years)	4 years 3 months 18 days
Acquired performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 69,353
Carrying Value	$ 57,229	72,425
Loan Count | loan	1,947
Weighted Average Yield	8.90%
Weighted Average Life (Years)	4 years 8 months 12 days
Acquired performing loans | Ginnie Mae
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 252,100
Acquired non-performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	27,042
Carrying Value	$ 21,454	28,602
Loan Count | loan	324
Weighted Average Yield	9.30%
Weighted Average Life (Years)	3 years 2 months 12 days
Acquired non-performing loans | Ginnie Mae
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 186,700
Total residential mortgage loans, held-for-sale, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	2,787,891
Carrying Value	$ 2,740,599	3,297,271
Loan Count | loan	10,068
Weighted Average Yield	6.70%
Weighted Average Life (Years)	23 years
Acquired performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 997,217
Carrying Value	$ 963,554	890,131
Loan Count | loan	3,374
Weighted Average Yield	6.80%
Weighted Average Life (Years)	13 years 1 month 6 days
Acquired non-performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 211,662
Carrying Value	$ 191,078	340,342
Loan Count | loan	1,101
Weighted Average Yield	4.60%
Weighted Average Life (Years)	23 years 9 months 18 days
Originated loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 1,579,012
Carrying Value	$ 1,585,967	2,066,798
Loan Count | loan	5,593
Weighted Average Yield	6.90%
Weighted Average Life (Years)	29 years 2 months 12 days
Total residential mortgage loans, held-for-sale, at fair value/lower of cost or market
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 2,884,286
Carrying Value	$ 2,819,282	$ 3,398,298