Mortgage-Backed Securities - Unrealized Loss Positions (Details) $ in Thousands	Sep. 30, 2023 USD ($) security	Dec. 31, 2022 USD ($) security
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 184,655	$ 656,243
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (23,228)	$ (87,584)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	101	260
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 290,170	$ 97,874
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (152,706)	$ (74,935)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	184	66
Total, Estimated Fair Value	$ 474,825	$ 754,117
Total, Unrealized Losses	$ (175,934)	$ (162,519)
Total, Number of Positions | security	285	326
Residential Mortgage-Backed Securities | Non-Agency RMBS - Senior
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 76,971	$ 83,553
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (833)	$ (6,170)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	15	13
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 38,809	$ 7,577
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (7,423)	$ (763)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	5	1
Total, Estimated Fair Value	$ 115,780	$ 91,130
Total, Unrealized Losses	$ (8,256)	$ (6,933)
Total, Number of Positions | security	20	14
Residential Mortgage-Backed Securities | Non-Agency RMBS - Subordinated
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 20,937	$ 161,959
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (1,749)	$ (27,120)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	6	28
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 175,017	$ 37,025
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (46,317)	$ (9,557)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	34	8
Total, Estimated Fair Value	$ 195,954	$ 198,984
Total, Unrealized Losses	$ (48,066)	$ (36,677)
Total, Number of Positions | security	40	36
Residential Mortgage-Backed Securities | Non-Agency RMBS - Senior Interest-only
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 18,428	$ 41,890
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (12,469)	$ (24,411)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	39	79
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 26,260	$ 15,213
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (86,827)	$ (55,613)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	102	50
Total, Estimated Fair Value	$ 44,688	$ 57,103
Total, Unrealized Losses	$ (99,296)	$ (80,024)
Total, Number of Positions | security	141	129
Residential Mortgage-Backed Securities | Agency RMBS - Interest-only
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 0	$ 6,062
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ 0	$ (500)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	0	4
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 8,589	$ 2,825
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (5,558)	$ (4,491)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	8	4
Total, Estimated Fair Value	$ 8,589	$ 8,887
Total, Unrealized Losses	$ (5,558)	$ (4,991)
Total, Number of Positions | security	8	8
Commercial Mortgage Backed Securities | Agency MBS - Project Loans
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 64,005	$ 281,307
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (7,559)	$ (18,880)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	39	131
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 41,335	$ 0
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (6,305)	$ 0
Unrealized Loss Position for 12 Months or More, Number of Positions | security	33	0
Total, Estimated Fair Value	$ 105,340	$ 281,307
Total, Unrealized Losses	$ (13,864)	$ (18,880)
Total, Number of Positions | security	72	131
Commercial Mortgage Backed Securities | Agency CMBS - Interest-only
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 4,314	$ 81,472
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (618)	$ (10,503)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	2	5
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 160	$ 35,234
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (276)	$ (4,511)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	2	3
Total, Estimated Fair Value	$ 4,474	$ 116,706
Total, Unrealized Losses	$ (894)	$ (15,014)
Total, Number of Positions | security	4	8