Investment in Securities (Summary of Investment Income) (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2020	Sep. 30, 2019	Sep. 30, 2020	Sep. 30, 2019
Securities, Available-for-sale [Line Items]
Interest Income	$ 43,075	$ 39,985	$ 134,463	$ 114,548
Agency RMBS
Securities, Available-for-sale [Line Items]
Coupon Interest	14,117	16,026	47,632	42,681
Net Amortization	(7,454)	(6,290)	(25,516)	(15,883)
Interest Income	6,663	9,736	22,116	26,798
Non-Agency RMBS and CMBS
Securities, Available-for-sale [Line Items]
Coupon Interest	3,311	3,383	10,995	10,514
Net Amortization	1,911	430	4,001	1,666
Interest Income	5,222	3,813	14,996	12,180
CLOs
Securities, Available-for-sale [Line Items]
Coupon Interest	2,256	3,480	11,662	11,391
Net Amortization	2,179	(627)	1,543	(1,138)
Interest Income	4,435	2,853	13,205	10,253
Other Securities
Securities, Available-for-sale [Line Items]
Coupon Interest	2,970	1,670	8,992	4,706
Net Amortization	(1,435)	(611)	(3,584)	(1,528)
Interest Income	1,535	1,059	5,408	3,178
Securities Investment [Member]
Securities, Available-for-sale [Line Items]
Coupon Interest	22,654	24,559	79,281	69,292
Net Amortization	(4,799)	(7,098)	(23,556)	(16,883)
Interest Income	$ 17,855	$ 17,461	$ 55,725	$ 52,409