QUARTERLY RESULTS (UNAUDITED) - Summarized Quarterly Results of Operations (Details) - USD ($) $ / shares in Units, $ in Thousands		3 Months Ended																12 Months Ended
		Dec. 31, 2019		Sep. 30, 2019		Jun. 30, 2019		Mar. 31, 2019		Dec. 31, 2018		Sep. 30, 2018		Jun. 30, 2018		Mar. 31, 2018		Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Quarterly Financial Information Disclosure [Abstract]
Interest income on residential mortgage investments		$ 73,617		$ 77,693		$ 85,100		$ 83,807		$ 72,902		$ 67,649		$ 65,202		$ 69,138		$ 320,217	$ 274,891	$ 232,435
Secured borrowings		(51,688)		(62,800)		(67,945)		(63,779)		(59,321)		(54,393)		(48,241)		(45,021)		(246,212)	(206,976)	(138,757)
Net interest income on residential mortgage investments		21,929		14,893		17,155		20,028		13,581		13,256		16,961		24,117
Other interest income (expense)	[1]	(1,244)		(845)		(1,300)		(1,469)		(1,284)		(1,560)		(1,595)		(1,483)
Other (expense) income		11,987	[2]	(10,852)	[2]	(79,315)	[2]	(26,305)	[2]	(3,313)	[2]	(3,016)	[2]	(2,378)	[2]	(3,214)	[2]	$ (104,485)	$ (11,921)	$ (7,443)
Net (loss) income		$ 32,672		$ 3,196		$ (63,460)		$ (7,746)		$ 8,984		$ 8,680		$ 12,988		$ 19,420
Basic net (loss) income per common share		$ 0.30		$ (0.02)		$ (0.80)		$ (0.15)										$ (0.62)	$ 0.34	$ 0.65
Diluted net (loss) income per common share		$ 0.29		$ (0.02)		$ (0.80)		$ (0.15)										(0.62)	0.34	0.65
Basic and diluted net income per common share										$ 0.05		$ 0.04		$ 0.09		$ 0.16		$ (0.62)	$ 0.34	$ 0.65

[1] Consists principally of interest on unsecured borrowings, overnight investments and cash collateral receivable with interest rate swap counterparties.

[2] Consists of losses on derivative instruments (net), losses on sales of investments, general and administrative expenses, including compensation-related costs, and miscellaneous other revenue (expense).