Investment in Securities (Maturities of Securities) (Details) $ in Thousands	Mar. 31, 2019 USD ($)
Agency RMBS
Fair Value
Less than three years	$ 38,461
Greater than three years less than seven years	486,247
Greater than seven years less then eleven years	581,828
Greater than eleven years	12,251
Total	1,118,787
Amortized Cost
Less than three years	38,339
Greater than three years less than seven years	489,019
Greater than seven years less then eleven years	585,867
Greater than eleven years	12,030
Total	$ 1,125,255
Weighted Average Coupon(2)
Less than three years	4.10%
Greater than three years less than seven years	4.04%
Greater than seven years less then eleven years	4.29%
Greater than eleven years	4.64%
Total	4.17%
Agency Interest Only Securities
Fair Value
Less than three years	$ 7,582
Greater than three years less than seven years	17,582
Greater than seven years less then eleven years	264
Greater than eleven years	0
Total	25,428
Amortized Cost
Less than three years	7,763
Greater than three years less than seven years	17,460
Greater than seven years less then eleven years	250
Greater than eleven years	0
Total	$ 25,473
Weighted Average Coupon(2)
Less than three years	0.00%
Greater than three years less than seven years	0.66%
Greater than seven years less then eleven years	3.42%
Greater than eleven years	3.14%
Total	3.31%
Non-Agency RMBS and CMBS
Fair Value
Less than three years	$ 60,447
Greater than three years less than seven years	65,214
Greater than seven years less then eleven years	53,571
Greater than eleven years	52,484
Total	231,716
Amortized Cost
Less than three years	52,618
Greater than three years less than seven years	60,490
Greater than seven years less then eleven years	49,960
Greater than eleven years	52,409
Total	$ 215,477
Weighted Average Coupon(2)
Less than three years	2.15%
Greater than three years less than seven years	3.61%
Greater than seven years less then eleven years	5.32%
Greater than eleven years	2.07%
Total	3.36%
Non-Agency IOs
Fair Value
Less than three years	$ 176
Greater than three years less than seven years	3,829
Greater than seven years less then eleven years	306
Greater than eleven years	3,335
Total	7,646
Amortized Cost
Less than three years	30
Greater than three years less than seven years	3,450
Greater than seven years less then eleven years	0
Greater than eleven years	2,213
Total	$ 5,693
Weighted Average Coupon(2)
Less than three years	0.00%
Greater than three years less than seven years	0.50%
Greater than seven years less then eleven years	1.40%
Greater than eleven years	2.00%
Total	0.77%
CLOs and Other Securities
Fair Value
Less than three years	$ 34,601
Greater than three years less than seven years	90,984
Greater than seven years less then eleven years	17,235
Greater than eleven years	1,623
Total	144,443
Amortized Cost
Less than three years	34,877
Greater than three years less than seven years	91,770
Greater than seven years less then eleven years	16,487
Greater than eleven years	1,451
Total	$ 144,585
Weighted Average Coupon(2)
Less than three years	0.00%
Greater than three years less than seven years	0.68%
Greater than seven years less then eleven years	4.33%
Greater than eleven years	12.03%
Total	5.69%