Mortgage-Backed Securities - Schedule of Components of Carrying Value of MBS and GSE CRT Portfolio (Details) - USD ($) $ in Thousands	Dec. 31, 2023	Dec. 31, 2022	Dec. 31, 2021	Dec. 31, 2020
Debt Securities, Available-for-sale [Line Items]
Commercial loan, held-for-investment	$ 26,604	$ 22,744
Principal/notional balance	5,864,813	5,687,505
Unamortized premium	5,061	5,195
Unamortized discount	(931,456)	(957,420)
Allowance for credit losses	(320)	0	$ 0	$ (1,768)
Gross unrealized gains	111,422	66,850
Gross unrealized losses	(4,214)	(10,237)
Fair Value	5,045,306	4,791,893
MBS
Debt Securities, Available-for-sale [Line Items]
Commercial loan, held-for-investment	22,300	21,300
Principal/notional balance	5,025,062	4,770,175
Unamortized premium	5,061	5,195
Unamortized discount	(169,342)	(126,112)
Allowance for credit losses	(320)	0
Gross unrealized gains	107,899	62,245
Gross unrealized losses	(393)	(7,535)
Fair Value	4,967,967	4,703,968
Interest-Only Securities
Debt Securities, Available-for-sale [Line Items]
Principal/notional balance	839,751	917,330
Unamortized premium	0	0
Unamortized discount	(762,114)	(831,308)
Allowance for credit losses	0	0
Gross unrealized gains	3,523	4,605
Gross unrealized losses	(3,821)	(2,702)
Fair Value	$ 77,339	$ 87,925