FAIR VALUE MEASUREMENT - Carrying Values and Fair Values of Financial Assets Recorded at Fair Value on a Recurring Basis (Details) - USD ($) $ in Thousands	Sep. 30, 2017	Dec. 31, 2016
Investments in:
Excess mortgage servicing rights, at fair value	$ 1,178,308	$ 1,399,455
Mortgage servicing rights, at fair value	1,702,749	659,483
Mortgage servicing rights financing receivable, at fair value	607,396	0
Real estate securities, available-for-sale	6,714,846	5,073,858
Derivative assets	10,444	6,762
Restricted cash	152,047	163,095
Liabilities
Derivative liabilities	82	$ 3,021
Recurring Basis
Investments in:
Excess mortgage servicing rights, principal balance	228,173,728
Excess mortgage servicing rights, equity method investees, principal balance	53,675,234
Mortgage servicing rights, principal balance	177,285,425
Mortgage servicing rights financing receivable, principal balance	67,052,949
Servicer advances, principal balance	3,651,705
Real estate securities, available-for-sale, principal balance	13,030,161
Residential mortgage loans, held-for-investment, principal balance	820,544
Residential mortgage loans, held-for-sale, principal balance	1,619,737
Consumer loans, held-for-investment, principal balance	1,472,973
Derivative assets, notional amount	5,358,500
Cash and cash equivalents, principal balance	279,760
Restricted cash, principal balance	152,047
Other assets, principal balance	1,427,891
Liabilities
Repurchase agreements, principal balance	7,850,214
Notes and bonds payable, principal balance	7,248,905
Derivative liabilities, notional amount	3,713,500
Recurring Basis | Carrying Value
Investments in:
Excess mortgage servicing rights, at fair value	1,178,308
Excess mortgage servicing rights, equity method investees, at fair value	175,633
Mortgage servicing rights, at fair value	1,702,749
Mortgage servicing rights financing receivable, at fair value	607,396
Servicer advance investments, at fair value	4,044,802
Real estate securities, available-for-sale	6,714,846
Residential mortgage loans, held-for-investment	702,227
Residential mortgage loans, held-for-sale	1,426,751
Consumer loans, held-for-investment	1,467,933
Derivative assets	10,444
Cash and cash equivalents	279,760
Restricted cash	152,047
Other assets	31,150
Assets, fair value	18,494,046
Liabilities
Repurchase agreements	7,848,028
Notes and bonds payable	7,236,967
Derivative liabilities	82
Liabilities, fair value	15,085,077
Recurring Basis | Fair Value
Investments in:
Excess mortgage servicing rights, at fair value	1,178,308
Excess mortgage servicing rights, equity method investees, at fair value	175,633
Mortgage servicing rights, at fair value	1,702,749
Mortgage servicing rights financing receivable, at fair value	607,396
Servicer advance investments, at fair value	4,044,802
Real estate securities, available-for-sale	6,714,846
Residential mortgage loans, held-for-investment	713,665
Residential mortgage loans, held-for-sale	1,474,430
Consumer loans, held-for-investment	1,483,223
Derivative assets	10,444
Cash and cash equivalents	279,760
Restricted cash	152,047
Other assets	31,150
Assets, fair value	18,568,453
Liabilities
Repurchase agreements	7,850,214
Notes and bonds payable	7,248,265
Derivative liabilities	82
Liabilities, fair value	15,098,561
Recurring Basis | Fair Value | Level 1
Investments in:
Excess mortgage servicing rights, at fair value	0
Excess mortgage servicing rights, equity method investees, at fair value	0
Mortgage servicing rights, at fair value	0
Mortgage servicing rights financing receivable, at fair value	0
Servicer advance investments, at fair value	0
Real estate securities, available-for-sale	0
Residential mortgage loans, held-for-investment	0
Residential mortgage loans, held-for-sale	0
Consumer loans, held-for-investment	0
Derivative assets	0
Cash and cash equivalents	279,760
Restricted cash	152,047
Other assets	20,900
Assets, fair value	452,707
Liabilities
Repurchase agreements	0
Notes and bonds payable	0
Derivative liabilities	0
Liabilities, fair value	0
Recurring Basis | Fair Value | Level 2
Investments in:
Excess mortgage servicing rights, at fair value	0
Excess mortgage servicing rights, equity method investees, at fair value	0
Mortgage servicing rights, at fair value	0
Mortgage servicing rights financing receivable, at fair value	0
Servicer advance investments, at fair value	0
Real estate securities, available-for-sale	1,190,656
Residential mortgage loans, held-for-investment	0
Residential mortgage loans, held-for-sale	0
Consumer loans, held-for-investment	0
Derivative assets	10,444
Cash and cash equivalents	0
Restricted cash	0
Other assets	0
Assets, fair value	1,201,100
Liabilities
Repurchase agreements	7,850,214
Notes and bonds payable	0
Derivative liabilities	82
Liabilities, fair value	7,850,296
Recurring Basis | Fair Value | Level 3
Investments in:
Excess mortgage servicing rights, at fair value	1,178,308
Excess mortgage servicing rights, equity method investees, at fair value	175,633
Mortgage servicing rights, at fair value	1,702,749
Mortgage servicing rights financing receivable, at fair value	607,396
Servicer advance investments, at fair value	4,044,802
Real estate securities, available-for-sale	5,524,190
Residential mortgage loans, held-for-investment	713,665
Residential mortgage loans, held-for-sale	1,474,430
Consumer loans, held-for-investment	1,483,223
Derivative assets	0
Cash and cash equivalents	0
Restricted cash	0
Other assets	10,250
Assets, fair value	16,914,646
Liabilities
Repurchase agreements	0
Notes and bonds payable	7,248,265
Derivative liabilities	0
Liabilities, fair value	$ 7,248,265