Investment in Securities (Summary of Investment in Securities) (Details) - USD ($) $ in Thousands	6 Months Ended
	Jun. 30, 2019	Dec. 31, 2018
Securities, Available-for-sale [Line Items]
Current Principal	$ 1,607,712
Unamortized Premium (Discount)	(127,647)
Amortized Cost	1,589,291
Gross Unrealized Gains	41,681
Gross Unrealized Losses	(11,878)
Investment Owned At Fair Value, Net	1,619,094
Securities, at fair value	1,668,677
Securities sold short, at fair value	$ 49,583	$ 850,577
Fair Value		$ 2,939,311
Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.21%
Weighted Average Yield	4.47%
Weighted average life	6 years 18 days
Long
Securities, Available-for-sale [Line Items]
Current Principal	$ 1,655,957
Unamortized Premium (Discount)	(127,388)
Amortized Cost	1,637,795
Gross Unrealized Gains	41,681
Gross Unrealized Losses	(10,799)
Securities, at fair value	$ 1,668,677
Long | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.14%
Weighted Average Yield	4.55%
Weighted average life	6 years 26 days
Long | 15-year fixed-rate mortgages
Securities, Available-for-sale [Line Items]
Current Principal	$ 133,432
Unamortized Premium (Discount)	3,326
Amortized Cost	136,758
Gross Unrealized Gains	1,158
Gross Unrealized Losses	(503)
Securities, at fair value	$ 137,413
Long | 15-year fixed-rate mortgages | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.29%
Weighted Average Yield	2.56%
Weighted average life	3 years 318 days
Long | 20-year fixed-rate mortgages
Securities, Available-for-sale [Line Items]
Current Principal	$ 839
Unamortized Premium (Discount)	57
Amortized Cost	896
Gross Unrealized Gains	0
Gross Unrealized Losses	0
Securities, at fair value	$ 896
Long | 20-year fixed-rate mortgages | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.50%
Weighted Average Yield	3.03%
Weighted average life	5 years
Long | 30-year fixed-rate mortgages
Securities, Available-for-sale [Line Items]
Current Principal	$ 997,663
Unamortized Premium (Discount)	44,210
Amortized Cost	1,041,873
Gross Unrealized Gains	11,186
Gross Unrealized Losses	(5,057)
Securities, at fair value	$ 1,048,002
Long | 30-year fixed-rate mortgages | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.14%
Weighted Average Yield	3.29%
Weighted average life	6 years 263 days
Long | Adjustable rate mortgages
Securities, Available-for-sale [Line Items]
Current Principal	$ 8,887
Unamortized Premium (Discount)	334
Amortized Cost	9,221
Gross Unrealized Gains	81
Gross Unrealized Losses	(111)
Securities, at fair value	$ 9,191
Long | Adjustable rate mortgages | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.02%
Weighted Average Yield	3.06%
Weighted average life	2 years 310 days
Long | Reverse mortgages
Securities, Available-for-sale [Line Items]
Current Principal	$ 109,314
Unamortized Premium (Discount)	8,157
Amortized Cost	117,471
Gross Unrealized Gains	1,334
Gross Unrealized Losses	(311)
Securities, at fair value	$ 118,494
Long | Reverse mortgages | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.50%
Weighted Average Yield	3.08%
Weighted average life	6 years 270 days
Long | Interest only securities
Securities, Available-for-sale [Line Items]
Amortized Cost	$ 24,346
Gross Unrealized Gains	1,125
Gross Unrealized Losses	(566)
Securities, at fair value	$ 24,905
Long | Interest only securities | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.36%
Weighted Average Yield	7.90%
Weighted average life	3 years 168 days
Long | Non-Agency RMBS
Securities, Available-for-sale [Line Items]
Current Principal	$ 281,461
Unamortized Premium (Discount)	(118,422)
Amortized Cost	163,039
Gross Unrealized Gains	19,690
Gross Unrealized Losses	(1,398)
Securities, at fair value	$ 181,331
Long | Non-Agency RMBS | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.78%
Weighted Average Yield	6.57%
Weighted average life	4 years 307 days
Long | CMBS
Securities, Available-for-sale [Line Items]
Current Principal	$ 68,412
Unamortized Premium (Discount)	(37,571)
Amortized Cost	30,841
Gross Unrealized Gains	3,089
Gross Unrealized Losses	0
Securities, at fair value	$ 33,930
Long | CMBS | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.21%
Weighted Average Yield	9.59%
Weighted average life	8 years 37 days
Long | Non-Agency IOs
Securities, Available-for-sale [Line Items]
Amortized Cost	$ 5,443
Gross Unrealized Gains	1,970
Gross Unrealized Losses	0
Securities, at fair value	$ 7,413
Long | Non-Agency IOs | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	1.02%
Weighted Average Yield	21.44%
Weighted average life	3 years 102 days
Long | CLOs
Securities, Available-for-sale [Line Items]
Amortized Cost	$ 77,992
Gross Unrealized Gains	697
Gross Unrealized Losses	(2,478)
Securities, at fair value	$ 76,211
Long | CLOs | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.14%
Weighted Average Yield	15.27%
Weighted average life	5 years 223 days
Long | Asset-backed securities, backed by consumer loans
Securities, Available-for-sale [Line Items]
Current Principal	$ 30,892
Unamortized Premium (Discount)	(6,541)
Amortized Cost	24,351
Gross Unrealized Gains	978
Gross Unrealized Losses	(310)
Securities, at fair value	$ 25,019
Long | Asset-backed securities, backed by consumer loans | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	14.61%
Weighted Average Yield	14.37%
Weighted average life	1 year 55 days
Long | Corporate debt securities
Securities, Available-for-sale [Line Items]
Current Principal	$ 25,057
Unamortized Premium (Discount)	(20,938)
Amortized Cost	4,119
Gross Unrealized Gains	26
Gross Unrealized Losses	(64)
Securities, at fair value	$ 4,081
Long | Corporate debt securities | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	12.51%
Weighted Average Yield	17.63%
Weighted average life	329 days
Long | Corporate equity securities
Securities, Available-for-sale [Line Items]
Amortized Cost	$ 1,445
Securities, at fair value	1,791
Corporate equity, gross unrealized gains	347
Corporate equity, gross unrealized losses	(1)
Short
Securities, Available-for-sale [Line Items]
Current Principal	(48,245)
Unamortized Premium (Discount)	259
Amortized Cost	48,504
Gross Unrealized Gains	0
Gross Unrealized Losses	(1,079)
Securities sold short, at fair value	$ 49,583
Short | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	2.06%
Weighted Average Yield	1.80%
Weighted average life	6 years 318 days
Short | Corporate debt securities
Securities, Available-for-sale [Line Items]
Current Principal	$ (450)
Unamortized Premium (Discount)	6
Amortized Cost	456
Gross Unrealized Gains	0
Gross Unrealized Losses	(14)
Securities sold short, at fair value	$ 470
Short | Corporate debt securities | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	5.42%
Weighted Average Yield	5.21%
Weighted average life	5 years 142 days
Short | U.S. Treasury securities
Securities, Available-for-sale [Line Items]
Current Principal	$ (37,900)
Unamortized Premium (Discount)	492
Amortized Cost	38,392
Gross Unrealized Gains	0
Gross Unrealized Losses	(582)
Securities sold short, at fair value	$ 38,974
Short | U.S. Treasury securities | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	2.34%
Weighted Average Yield	2.19%
Weighted average life	8 years 47 days
Short | European sovereign bonds
Securities, Available-for-sale [Line Items]
Current Principal	$ (9,895)
Unamortized Premium (Discount)	(239)
Amortized Cost	9,656
Gross Unrealized Gains	0
Gross Unrealized Losses	(483)
Securities sold short, at fair value	$ 10,139
Short | European sovereign bonds | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	0.75%
Weighted Average Yield	0.12%
Weighted average life	2 years 29 days