Mortgage-Backed Securities - Unrealized Loss Positions (Details) $ in Thousands	Mar. 31, 2024 USD ($) security	Dec. 31, 2023 USD ($) security
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 103,849	$ 136,549
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (9,729)	$ (10,352)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	54	52
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 311,892	$ 306,074
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (130,372)	$ (129,176)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	190	186
Total, Estimated Fair Value	$ 415,741	$ 442,623
Total, Unrealized Losses	$ (140,101)	$ (139,528)
Total, Number of Positions | security	244	238
Residential Mortgage-Backed Securities | Non-Agency RMBS - Senior
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 46,803	$ 51,476
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (283)	$ (281)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	13	13
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 45,625	$ 42,859
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (6,335)	$ (6,700)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	8	6
Total, Estimated Fair Value	$ 92,429	$ 94,335
Total, Unrealized Losses	$ (6,618)	$ (6,981)
Total, Number of Positions | security	21	19
Residential Mortgage-Backed Securities | Non-Agency RMBS - Subordinated
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 25,720	$ 19,382
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (630)	$ (1,467)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	4	5
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 189,006	$ 181,822
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (31,289)	$ (35,081)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	35	33
Total, Estimated Fair Value	$ 214,726	$ 201,204
Total, Unrealized Losses	$ (31,919)	$ (36,548)
Total, Number of Positions | security	39	38
Residential Mortgage-Backed Securities | Non-Agency RMBS - Interest-only
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 20,615	$ 19,263
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (6,983)	$ (4,035)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	30	27
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 29,047	$ 36,731
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (83,153)	$ (77,455)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	105	105
Total, Estimated Fair Value	$ 49,662	$ 55,994
Total, Unrealized Losses	$ (90,136)	$ (81,490)
Total, Number of Positions | security	135	132
Residential Mortgage-Backed Securities | Agency RMBS - Interest-only
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 1,278	$ 0
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (8)	$ 0
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	1	0
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 9,526	$ 9,151
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (4,813)	$ (5,077)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	8	8
Total, Estimated Fair Value	$ 10,803	$ 9,151
Total, Unrealized Losses	$ (4,821)	$ (5,077)
Total, Number of Positions | security	9	8
Commercial Mortgage Backed Securities | Agency MBS - Project loans
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 5,149	$ 43,827
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (1,065)	$ (4,040)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	3	5
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 38,557	$ 35,353
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (4,502)	$ (4,601)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	32	32
Total, Estimated Fair Value	$ 43,706	$ 79,180
Total, Unrealized Losses	$ (5,567)	$ (8,641)
Total, Number of Positions | security	35	37
Commercial Mortgage Backed Securities | Agency CMBS - Interest-only
Debt Securities, Available-for-sale [Line Items]
Unrealized Loss Position for Less than 12 Months, Estimated Fair Value	$ 4,284	$ 2,601
Unrealized Loss Position for Less than 12 Months, Unrealized Losses	$ (760)	$ (529)
Unrealized Loss Position for Less than 12 Months, Number of Positions | security	3	2
Unrealized Loss Position for 12 Months or More, Estimated Fair Value	$ 131	$ 158
Unrealized Loss Position for 12 Months or More, Unrealized Losses	$ (280)	$ (262)
Unrealized Loss Position for 12 Months or More, Number of Positions | security	2	2
Total, Estimated Fair Value	$ 4,415	$ 2,759
Total, Unrealized Losses	$ (1,040)	$ (791)
Total, Number of Positions | security	5	4