Mortgage-Backed Securities - Unrealized Loss Positions (Details) $ in Thousands	Sep. 30, 2021 USD ($) security	Dec. 31, 2020 USD ($) security
Investment Holdings [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 118,355	$ 339,522
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (11,790)	$ (22,925)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	50	53
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 93,216	$ 120,192
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (105,386)	$ (99,385)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	79	75
Total, Estimated Fair Value	$ 211,571	$ 459,714
Total, Unrealized Losses	$ (117,176)	$ (122,310)
Total, Number of Positions | security	129	128
Residential Mortgage-Backed Securities | Non-Agency RMBS - Senior
Investment Holdings [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 9,596	$ 11,985
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (190)	$ (248)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	1	2
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 0	$ 0
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ 0	$ 0
Unrealized Loss Position for 12 Months or More, Number of Positions | security	0	0
Total, Estimated Fair Value	$ 9,596	$ 11,985
Total, Unrealized Losses	$ (190)	$ (248)
Total, Number of Positions | security	1	2
Residential Mortgage-Backed Securities | Non-Agency RMBS - Subordinated
Investment Holdings [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 42,385	$ 253,822
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (478)	$ (8,711)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	7	5
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 21,600	$ 34,697
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (21,106)	$ (29,318)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	10	15
Total, Estimated Fair Value	$ 63,985	$ 288,519
Total, Unrealized Losses	$ (21,584)	$ (38,029)
Total, Number of Positions | security	17	20
Residential Mortgage-Backed Securities | Non-Agency RMBS - Senior interest-only
Investment Holdings [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 25,352	$ 38,604
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (4,992)	$ (8,682)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	34	34
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 24,475	$ 22,761
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (49,566)	$ (46,537)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	51	43
Total, Estimated Fair Value	$ 49,827	$ 61,365
Total, Unrealized Losses	$ (54,558)	$ (55,219)
Total, Number of Positions | security	85	77
Residential Mortgage-Backed Securities | Agency RMBS - Interest-only
Investment Holdings [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 16,904	$ 31,059
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (5,980)	$ (4,938)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	5	6
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 47,141	$ 54,153
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (34,714)	$ (23,461)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	18	16
Total, Estimated Fair Value	$ 64,045	$ 85,212
Total, Unrealized Losses	$ (40,694)	$ (28,399)
Total, Number of Positions | security	23	22
Commercial Mortgage Backed Securities | Agency MBS - Commercial
Investment Holdings [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 0	$ 0
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ 0	$ 0
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	0	0
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 0	$ 8,581
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ 0	$ (69)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	0	1
Total, Estimated Fair Value	$ 0	$ 8,581
Total, Unrealized Losses	$ 0	$ (69)
Total, Number of Positions | security	0	1
Commercial Mortgage Backed Securities | Agency CMBS - Interest-only
Investment Holdings [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 24,118	$ 4,052
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (150)	$ (346)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	3	6
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 0	$ 0
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ 0	$ 0
Unrealized Loss Position for 12 Months or More, Number of Positions | security	0	0
Total, Estimated Fair Value	$ 24,118	$ 4,052
Total, Unrealized Losses	$ (150)	$ (346)
Total, Number of Positions | security	3	6