Investment in Securities (Unrealized Loss Positions Securities) (Details) - USD ($) $ in Thousands	Mar. 31, 2021	Dec. 31, 2020
Securities, Available-for-sale [Line Items]
Fair Value, Less than 12 Months	$ 809,221	$ 153,786
Unrealized Losses, Less than 12 Months	(15,673)	(4,358)
Fair Value, Greater than 12 Months	18,202	4,627
Unrealized Losses, Greater than 12 Months	(2,482)	(1,523)
Fair Value, Total	827,423	158,413
Unrealized Loss, Total	(18,155)	(5,881)
Agency RMBS
Securities, Available-for-sale [Line Items]
Fair Value, Less than 12 Months	806,309	126,926
Unrealized Losses, Less than 12 Months	(15,549)	(981)
Fair Value, Greater than 12 Months	629	832
Unrealized Losses, Greater than 12 Months	(33)	(44)
Fair Value, Total	806,938	127,758
Unrealized Loss, Total	(15,582)	(1,025)
Non-Agency RMBS
Securities, Available-for-sale [Line Items]
Fair Value, Less than 12 Months	1,048	20,474
Unrealized Losses, Less than 12 Months	(103)	(2,616)
Fair Value, Greater than 12 Months	16,209	2,599
Unrealized Losses, Greater than 12 Months	(1,563)	(348)
Fair Value, Total	17,257	23,073
Unrealized Loss, Total	(1,666)	(2,964)
CLOs
Securities, Available-for-sale [Line Items]
Fair Value, Less than 12 Months	1,864	5,279
Unrealized Losses, Less than 12 Months	(16)	(753)
Fair Value, Greater than 12 Months	1,364	1,196
Unrealized Losses, Greater than 12 Months	(886)	(1,131)
Fair Value, Total	3,228	6,475
Unrealized Loss, Total	(902)	(1,884)
Other Securities
Securities, Available-for-sale [Line Items]
Fair Value, Less than 12 Months	0	1,107
Unrealized Losses, Less than 12 Months	(5)	(8)
Fair Value, Greater than 12 Months	0	0
Unrealized Losses, Greater than 12 Months	0	0
Fair Value, Total	0	1,107
Unrealized Loss, Total	$ (5)	$ (8)