Investment in Securities (Summary of Investment Income) (Details) - USD ($) $ in Thousands	3 Months Ended			6 Months Ended
	Jun. 30, 2019	Jun. 30, 2018	[1]	Jun. 30, 2019	Jun. 30, 2018
Securities, Available-for-sale [Line Items]
Net Amortization					$ (22,949)
Interest Income	$ 38,547	$ 31,941		$ 74,563	$ 60,033	[1]
Agency RMBS
Securities, Available-for-sale [Line Items]
Coupon Interest	14,466			26,656
Net Amortization	(4,965)			(9,593)
Interest Income	9,501			17,063
Non-Agency RMBS and CMBS
Securities, Available-for-sale [Line Items]
Coupon Interest	3,282			7,131
Net Amortization	689			1,236
Interest Income	3,971			8,367
CLOs
Securities, Available-for-sale [Line Items]
Coupon Interest	3,666			7,910
Net Amortization	(576)			(511)
Interest Income	3,090			7,399
Other Securities
Securities, Available-for-sale [Line Items]
Coupon Interest	1,443			3,036
Net Amortization	(355)			(917)
Interest Income	1,088			2,119
Securities Investment [Member]
Securities, Available-for-sale [Line Items]
Coupon Interest	22,857			44,733
Net Amortization	(5,207)			(9,785)
Interest Income	$ 17,650			$ 34,948

[1] Includes interest income and interest expense of a consolidated securitization trust of $1.3 million and $0.8 million, respectively, for the three-month period ended June 30, 2018. Includes interest income and interest expense of a consolidated securitization trust of $2.6 million and $1.7 million, respectively, for the six-month period ended June 30, 2018. See Note 6 for further details on the Company's consolidated securitization trust.