FAIR VALUE OF FINANCIAL INSTRUMENTS - Summary of Financial Assets and Liabilities (Details) - USD ($) $ in Thousands	Sep. 30, 2015	Dec. 31, 2014
Assets:
Fair Value	$ 2,241,525	$ 2,034,719
Liabilities:
Fair Value	4,313,165	4,198,379
Repurchase agreements - short-term
Liabilities:
Outstanding Face Amount	793,933	1,331,603
Fair Value	793,933	1,331,603
Repurchase agreements - long-term
Liabilities:
Outstanding Face Amount	397,393	100,062
Fair Value	397,393	100,062
Borrowings under credit agreement
Liabilities:
Outstanding Face Amount	20,400	11,000
Fair Value	20,400	11,000
Borrowings under credit and security agreement
Liabilities:
Outstanding Face Amount		46,750
Fair Value		46,750
Revolving credit facility
Liabilities:
Outstanding Face Amount	50,000	25,000
Fair Value	50,000	25,000
Mortgage loan financing
Liabilities:
Outstanding Face Amount	562,035	442,753
Fair Value	590,265	455,846
Borrowings from the FHLB
Liabilities:
Outstanding Face Amount	1,786,000	1,611,000
Fair Value	1,857,923	1,616,373
Senior unsecured notes
Liabilities:
Outstanding Face Amount	619,555	619,555
Fair Value	603,251	611,745
Mortgage loan receivables held for investment
Assets:
Outstanding Face Amount	1,811,216	1,536,923
Fair Value	1,820,999	1,540,388
Mortgage loan receivables held for sale
Assets:
Outstanding Face Amount	333,488	417,955
Fair Value	342,611	421,991
FHLB stock
Assets:
Outstanding Face Amount	77,915	72,340
Fair Value	77,915	72,340
Level 2
Assets:
Fair Value		0
Liabilities:
Fair Value	37,421	68,357
Level 2 | Repurchase agreements - short-term
Liabilities:
Fair Value	37,421	68,357
Level 2 | Repurchase agreements - long-term
Liabilities:
Fair Value		0
Level 2 | Senior unsecured notes
Liabilities:
Fair Value		0
Level 3
Assets:
Fair Value	2,241,525	2,034,719
Liabilities:
Fair Value	4,275,744	4,130,022
Level 3 | Repurchase agreements - short-term
Liabilities:
Fair Value	756,512	1,263,246
Level 3 | Repurchase agreements - long-term
Liabilities:
Fair Value	397,393	100,062
Level 3 | Borrowings under credit agreement
Liabilities:
Fair Value	20,400	11,000
Level 3 | Borrowings under credit and security agreement
Liabilities:
Fair Value		46,750
Level 3 | Revolving credit facility
Liabilities:
Fair Value	50,000	25,000
Level 3 | Mortgage loan financing
Liabilities:
Fair Value	590,265	455,846
Level 3 | Borrowings from the FHLB
Liabilities:
Fair Value	1,857,923	1,616,373
Level 3 | Senior unsecured notes
Liabilities:
Fair Value	603,251	611,745
Level 3 | Mortgage loan receivables held for investment
Assets:
Fair Value	1,820,999	1,540,388
Level 3 | Mortgage loan receivables held for sale
Assets:
Fair Value	342,611	421,991
Level 3 | FHLB stock
Assets:
Fair Value	77,915	72,340
Recurring
Assets:
Fair Value	2,415,681	2,815,989
Recurring | Nonhedge derivatives
Liabilities:
Derivative liability face amount	130,051	1,428,700
Fair Value	21,942	13,446
Recurring | CMBS
Assets:
Outstanding Face Amount	1,936,316	2,247,565
Fair Value	1,985,195	2,305,409
Recurring | CMBS interest-only
Assets:
Outstanding Face Amount	7,246,777	7,239,503
Fair Value	356,928	378,335
Recurring | GNMA interest-only
Assets:
Outstanding Face Amount	662,161	1,400,141
Fair Value	28,603	66,642
Recurring | GN construction securities
Assets:
Outstanding Face Amount	26,730	27,538
Fair Value	27,849	28,406
Recurring | GN permanent securities
Assets:
Outstanding Face Amount	16,435	36,232
Fair Value	16,807	36,773
Recurring | Nonhedge derivatives
Assets:
Derivative asset face amount	10,009	125,050
Fair Value	299	424
Recurring | Level 2
Assets:
Fair Value	299	132,245
Recurring | Level 2 | Nonhedge derivatives
Liabilities:
Fair Value	21,942	13,446
Recurring | Level 2 | CMBS
Assets:
Fair Value		0
Recurring | Level 2 | CMBS interest-only
Assets:
Fair Value		0
Recurring | Level 2 | GNMA interest-only
Assets:
Fair Value		66,642
Recurring | Level 2 | GN construction securities
Assets:
Fair Value		28,406
Recurring | Level 2 | GN permanent securities
Assets:
Fair Value		36,773
Recurring | Level 2 | Nonhedge derivatives
Assets:
Fair Value	299	424
Recurring | Level 3
Assets:
Fair Value	2,415,382	2,683,744
Recurring | Level 3 | Nonhedge derivatives
Liabilities:
Fair Value	0	0
Recurring | Level 3 | CMBS
Assets:
Fair Value	1,985,195	2,305,409
Recurring | Level 3 | CMBS interest-only
Assets:
Fair Value	356,928	$ 378,335
Recurring | Level 3 | GNMA interest-only
Assets:
Fair Value	28,603
Recurring | Level 3 | GN construction securities
Assets:
Fair Value	27,849
Recurring | Level 3 | GN permanent securities
Assets:
Fair Value	$ 16,807