INVESTMENTS IN RESIDENTIAL MORTGAGE LOANS - Loans Outstanding by Loan Type, Excluding REO (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2017 USD ($) loan	Dec. 31, 2016 USD ($) loan
Mortgage Loans on Real Estate [Line Items]
Weighted Average Life (Years)	1 year 2 months 2 days
Performing Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 557,381	$ 0
Carrying Value	$ 507,615	$ 0
Loan Count | loan	8,876	0
Weighted Average Yield	8.00%	0.00%
Weighted Average Life (Years)	5 years 6 months 1 day
Floating Rate Loans as a % of Face Amount	22.10%	0.00%
LTV Ratio	76.40%	0.00%
Weighted Average Delinquency	8.70%	0.00%
Weighted Average FICO	649	0
Performing Loans | Ginnie Mae EBO
Mortgage Loans on Real Estate [Line Items]
UPB of Underlying Mortgages	$ 33,700	$ 45,200
Purchased Credit Deteriorated Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	249,254	203,673
Carrying Value	$ 183,540	$ 190,761
Loan Count | loan	2,142	1,183
Weighted Average Yield	7.20%	5.50%
Weighted Average Life (Years)	3 years 1 month 13 days	2 years 8 months 29 days
Floating Rate Loans as a % of Face Amount	14.70%	8.70%
LTV Ratio	84.20%	71.50%
Weighted Average Delinquency	75.80%	94.90%
Weighted Average FICO	597	590
Total Residential Mortgage Loans, Held-for-Investment
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 806,635	$ 203,673
Carrying Value	$ 691,155	$ 190,761
Loan Count | loan	11,018	1,183
Weighted Average Yield	7.70%	5.50%
Weighted Average Life (Years)	4 years 9 months 6 days	2 years 8 months 29 days
Floating Rate Loans as a % of Face Amount	19.80%	8.70%
LTV Ratio	78.80%	71.50%
Weighted Average Delinquency	29.40%	94.90%
Weighted Average FICO	633	590
Reverse Mortgage Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 16,755	$ 22,645
Carrying Value	$ 6,870	$ 11,468
Loan Count | loan	48	69
Weighted Average Yield	7.50%	7.20%
Weighted Average Life (Years)	4 years 6 months 12 days	4 years 6 months 16 days
Floating Rate Loans as a % of Face Amount	15.90%	15.40%
LTV Ratio	141.20%	135.60%
Weighted Average Delinquency	77.80%	70.70%
Performing Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 1,044,116	$ 179,983
Carrying Value	$ 1,071,371	$ 175,194
Loan Count | loan	15,464	1,957
Weighted Average Yield	4.00%	4.30%
Weighted Average Life (Years)	4 years 10 months 2 days	5 years 10 months 30 days
Floating Rate Loans as a % of Face Amount	10.20%	22.40%
LTV Ratio	53.20%	102.90%
Weighted Average Delinquency	7.00%	6.40%
Weighted Average FICO	654	625
Non-Performing Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 846,181	$ 706,302
Carrying Value	$ 647,293	$ 510,003
Loan Count | loan	5,597	3,759
Weighted Average Yield	5.60%	7.10%
Weighted Average Life (Years)	4 years 3 months 12 days	2 years 10 months 8 days
Floating Rate Loans as a % of Face Amount	18.70%	20.60%
LTV Ratio	94.40%	105.00%
Weighted Average Delinquency	63.30%	75.90%
Weighted Average FICO	581	575
Total Residential Mortgage Loans, Held-for-Sale
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 1,907,052	$ 908,930
Carrying Value	$ 1,725,534	$ 696,665
Loan Count | loan	21,109	5,785
Weighted Average Yield	4.80%	6.50%
Weighted Average Life (Years)	4 years 7 months 2 days	3 years 6 months 1 day
Floating Rate Loans as a % of Face Amount	14.00%	20.80%
LTV Ratio	72.20%	105.40%
Weighted Average Delinquency	32.60%	62.00%
Weighted Average FICO	622	585
Real Estate Securities
Mortgage Loans on Real Estate [Line Items]
Residential mortgage loans, federal income tax basis	$ 2,414,400	$ 905,700
Reverse Mortgage Loans
Mortgage Loans on Real Estate [Line Items]
Interest in reverse mortgage loans	70.00%
Unpaid principal balance	$ 500	$ 500
Percentage of loans that have reached a termination event	54.30%	60.90%
Non-Performing Loans | Ginnie Mae EBO
Mortgage Loans on Real Estate [Line Items]
UPB of Underlying Mortgages	$ 66,500	$ 87,500
Nationstar | Reverse Mortgage Loans
Mortgage Loans on Real Estate [Line Items]
Interest in reverse mortgage loans	30.00%