Investment in Securities (Summary of Investment Income) (Details) - USD ($) $ in Thousands	3 Months Ended			9 Months Ended
	Sep. 30, 2019	Sep. 30, 2018	[1]	Sep. 30, 2019	Sep. 30, 2018
Securities, Available-for-sale [Line Items]
Net Amortization					$ (35,130)
Interest Income	$ 39,985	$ 35,300		$ 114,548	$ 95,333	[1]
Agency RMBS
Securities, Available-for-sale [Line Items]
Coupon Interest	16,026			42,681
Net Amortization	(6,290)			(15,883)
Interest Income	9,736			26,798
Non-Agency RMBS and CMBS
Securities, Available-for-sale [Line Items]
Coupon Interest	3,383			10,514
Net Amortization	430			1,666
Interest Income	3,813			12,180
CLOs
Securities, Available-for-sale [Line Items]
Coupon Interest	3,480			11,391
Net Amortization	(627)			(1,138)
Interest Income	2,853			10,253
Other Securities
Securities, Available-for-sale [Line Items]
Coupon Interest	1,670			4,706
Net Amortization	(611)			(1,528)
Interest Income	1,059			3,178
Securities Investment [Member]
Securities, Available-for-sale [Line Items]
Coupon Interest	24,559			69,292
Net Amortization	(7,098)			(16,883)
Interest Income	$ 17,461			$ 52,409

[1] Includes interest income and interest expense of a consolidated securitization trust of $1.3 million and $0.7 million, respectively, for the three-month period ended September 30, 2018. Includes interest income and interest expense of a consolidated securitization trust of $3.9 million and $2.4 million, respectively, for the nine-month period ended September 30, 2018. See Note 6 for further details on the Company's consolidated securitization trust.