Investment in Securities (Summary of Investment Income) (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2020	Jun. 30, 2019	Jun. 30, 2020	Jun. 30, 2019
Securities, Available-for-sale [Line Items]
Interest Income	$ 39,281	$ 38,547	$ 91,389	$ 74,563
Agency RMBS
Securities, Available-for-sale [Line Items]
Coupon Interest	12,603	14,466	33,515	26,656
Net Amortization	(9,218)	(4,965)	(18,062)	(9,593)
Interest Income	3,385	9,501	15,453	17,063
Non-Agency RMBS and CMBS
Securities, Available-for-sale [Line Items]
Coupon Interest	3,631	3,282	7,684	7,131
Net Amortization	1,377	689	2,090	1,236
Interest Income	5,008	3,971	9,774	8,367
CLOs
Securities, Available-for-sale [Line Items]
Coupon Interest	3,987	3,666	9,406	7,910
Net Amortization	376	(576)	(635)	(511)
Interest Income	4,363	3,090	8,771	7,399
Other Securities
Securities, Available-for-sale [Line Items]
Coupon Interest	3,098	1,443	6,023	3,036
Net Amortization	(1,104)	(355)	(2,149)	(917)
Interest Income	1,994	1,088	3,874	2,119
Securities Investment [Member]
Securities, Available-for-sale [Line Items]
Coupon Interest	23,319	22,857	56,628	44,733
Net Amortization	(8,569)	(5,207)	(18,756)	(9,785)
Interest Income	$ 14,750	$ 17,650	$ 37,872	$ 34,948