VARIABLE INTEREST ENTITIES - Geographic Concentrations of Credit Risk Exceeding 5% of Total Loan Unpaid Principal Balances (Details) - Consolidated VIEs $ in Thousands	6 Months Ended
	Jun. 30, 2018 USD ($)
FREMF Trusts
Concentration Risk [Line Items]
Principal Balance	$ 2,515,597
FREMF Trusts | Maryland
Concentration Risk [Line Items]
Principal Balance	437,110
FREMF Trusts | Texas
Concentration Risk [Line Items]
Principal Balance	361,075
FREMF Trusts | Virginia
Concentration Risk [Line Items]
Principal Balance	329,250
FREMF Trusts | New York
Concentration Risk [Line Items]
Principal Balance	280,925
FREMF Trusts | North Carolina
Concentration Risk [Line Items]
Principal Balance	231,335
FREMF Trusts | Pennsylvania
Concentration Risk [Line Items]
Principal Balance	225,810
FREMF Trusts | Massachusetts
Concentration Risk [Line Items]
Principal Balance	179,440
FREMF Trusts | Ohio
Concentration Risk [Line Items]
Principal Balance	156,138
FREMF Trusts | Other
Concentration Risk [Line Items]
Principal Balance	$ 314,514
FREMF Trusts | Securitized Loans | Geographic Concentration Risk
Concentration Risk [Line Items]
Percentage of balance	100.00%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Maryland
Concentration Risk [Line Items]
Percentage of balance	17.30%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Texas
Concentration Risk [Line Items]
Percentage of balance	14.40%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Virginia
Concentration Risk [Line Items]
Percentage of balance	13.10%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | New York
Concentration Risk [Line Items]
Percentage of balance	11.20%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | North Carolina
Concentration Risk [Line Items]
Percentage of balance	9.20%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Pennsylvania
Concentration Risk [Line Items]
Percentage of balance	9.00%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Massachusetts
Concentration Risk [Line Items]
Percentage of balance	7.10%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Ohio
Concentration Risk [Line Items]
Percentage of balance	6.20%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Other
Concentration Risk [Line Items]
Percentage of balance	12.50%
Residential Mortgage Loan Trusts
Concentration Risk [Line Items]
Principal Balance	$ 217,497
Residential Mortgage Loan Trusts | California
Concentration Risk [Line Items]
Principal Balance	71,095
Residential Mortgage Loan Trusts | Florida
Concentration Risk [Line Items]
Principal Balance	18,596
Residential Mortgage Loan Trusts | Texas
Concentration Risk [Line Items]
Principal Balance	17,616
Residential Mortgage Loan Trusts | Illinois
Concentration Risk [Line Items]
Principal Balance	15,133
Residential Mortgage Loan Trusts | NEW JERSEY
Concentration Risk [Line Items]
Principal Balance	13,156
Residential Mortgage Loan Trusts | Other
Concentration Risk [Line Items]
Principal Balance	$ 81,901
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk
Concentration Risk [Line Items]
Percentage of balance	100.00%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | California
Concentration Risk [Line Items]
Percentage of balance	32.70%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Florida
Concentration Risk [Line Items]
Percentage of balance	8.60%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Texas
Concentration Risk [Line Items]
Percentage of balance	8.10%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Illinois
Concentration Risk [Line Items]
Percentage of balance	7.00%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | NEW JERSEY
Concentration Risk [Line Items]
Percentage of balance	6.00%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Other
Concentration Risk [Line Items]
Percentage of balance	37.60%