Investment in Securities (Maturities of Securities) (Details) $ in Thousands	9 Months Ended
	Sep. 30, 2019 USD ($)
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 1,875,929
Amortized Cost	$ 1,808,453
Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.25%
Long
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 1,875,929
Amortized Cost	$ 1,844,840
Long | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.20%
Long | Agency RMBS
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 1,527,959
Amortized Cost	$ 1,513,535
Long | Agency RMBS | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.12%
Long | Agency RMBS | Less than three years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 305,530
Amortized Cost	$ 303,273
Long | Agency RMBS | Less than three years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.46%
Long | Agency RMBS | Greater than three years less than seven years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 721,685
Amortized Cost	$ 714,126
Long | Agency RMBS | Greater than three years less than seven years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.12%
Long | Agency RMBS | Greater than seven years less than eleven [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 495,138
Amortized Cost	$ 490,615
Long | Agency RMBS | Greater than seven years less than eleven [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.91%
Long | Agency RMBS | Greater than eleven year [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 5,606
Amortized Cost	$ 5,521
Long | Agency RMBS | Greater than eleven year [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.55%
Long | Agency RMBS | Interest only securities
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 37,048
Amortized Cost	$ 35,775
Long | Agency RMBS | Interest only securities | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	2.40%
Long | Agency RMBS | Interest only securities | Less than three years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 12,616
Amortized Cost	$ 12,400
Long | Agency RMBS | Interest only securities | Less than three years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.02%
Long | Agency RMBS | Interest only securities | Greater than three years less than seven years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 24,432
Amortized Cost	$ 23,375
Long | Agency RMBS | Interest only securities | Greater than three years less than seven years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	2.07%
Long | Agency RMBS | Interest only securities | Greater than seven years less than eleven [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 0
Amortized Cost	$ 0
Long | Agency RMBS | Interest only securities | Greater than seven years less than eleven [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	0.00%
Long | Agency RMBS | Interest only securities | Greater than eleven year [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 0
Amortized Cost	$ 0
Long | Agency RMBS | Interest only securities | Greater than eleven year [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	0.00%
Long | Non-Agency RMBS and CMBS
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 189,706
Amortized Cost	$ 176,348
Long | Non-Agency RMBS and CMBS | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.60%
Long | Non-Agency RMBS and CMBS | Less than three years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 45,363
Amortized Cost	$ 42,229
Long | Non-Agency RMBS and CMBS | Less than three years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	2.48%
Long | Non-Agency RMBS and CMBS | Greater than three years less than seven years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 73,889
Amortized Cost	$ 66,788
Long | Non-Agency RMBS and CMBS | Greater than three years less than seven years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	4.92%
Long | Non-Agency RMBS and CMBS | Greater than seven years less than eleven [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 55,218
Amortized Cost	$ 51,072
Long | Non-Agency RMBS and CMBS | Greater than seven years less than eleven [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.57%
Long | Non-Agency RMBS and CMBS | Greater than eleven year [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 15,236
Amortized Cost	$ 16,259
Long | Non-Agency RMBS and CMBS | Greater than eleven year [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	1.13%
Long | Non-Agency RMBS
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 153,614
Amortized Cost	$ 143,345
Long | Non-Agency RMBS | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.63%
Long | Non-Agency RMBS | Interest only securities
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 7,082
Amortized Cost	$ 5,146
Long | Non-Agency RMBS | Interest only securities | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	0.98%
Long | Non-Agency RMBS | Interest only securities | Less than three years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 3,290
Amortized Cost	$ 1,983
Long | Non-Agency RMBS | Interest only securities | Less than three years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	0.00%
Long | Non-Agency RMBS | Interest only securities | Greater than three years less than seven years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 3,792
Amortized Cost	$ 3,163
Long | Non-Agency RMBS | Interest only securities | Greater than three years less than seven years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	1.59%
Long | Non-Agency RMBS | Interest only securities | Greater than seven years less than eleven [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 0
Amortized Cost	$ 0
Long | Non-Agency RMBS | Interest only securities | Greater than seven years less than eleven [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	0.00%
Long | Non-Agency RMBS | Interest only securities | Greater than eleven year [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 0
Amortized Cost	$ 0
Long | Non-Agency RMBS | Interest only securities | Greater than eleven year [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	0.00%
Long | CLOs And Other Securities [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 109,898
Amortized Cost	$ 110,232
Long | CLOs And Other Securities [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	7.25%
Long | CLOs And Other Securities [Member] | Less than three years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 45,890
Amortized Cost	$ 45,239
Long | CLOs And Other Securities [Member] | Less than three years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	12.96%
Long | CLOs And Other Securities [Member] | Greater than three years less than seven years [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 62,386
Amortized Cost	$ 63,584
Long | CLOs And Other Securities [Member] | Greater than three years less than seven years [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	3.35%
Long | CLOs And Other Securities [Member] | Greater than seven years less than eleven [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 1,622
Amortized Cost	$ 1,409
Long | CLOs And Other Securities [Member] | Greater than seven years less than eleven [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	0.00%
Long | CLOs And Other Securities [Member] | Greater than eleven year [Member]
Securities, Available-for-sale [Line Items]
Securities, at fair value	$ 0
Amortized Cost	$ 0
Long | CLOs And Other Securities [Member] | Greater than eleven year [Member] | Weighted Average
Securities, Available-for-sale [Line Items]
Weighted Average Coupon	0.00%