Securities, at Fair Value - Narrative (Details)		3 Months Ended	12 Months Ended
		Mar. 31, 2024 USD ($)	Dec. 31, 2023 USD ($) option	Oct. 31, 2023	Mar. 31, 2023 USD ($)	Dec. 31, 2022 USD ($)
Debt Securities, Available-for-sale [Line Items]
Securities, at fair value	[1]	$ 736,950,000	$ 746,090,000
Allowance for loan loss		19,612,000	20,451,000		$ 33,062,000	$ 35,314,000
Securities, at fair value
Debt Securities, Available-for-sale [Line Items]
Securities, at fair value		82,000,000	79,900,000
Amortized costs		75,400,000	74,200,000
Gross unrealized gains		$ 6,600,000	$ 5,700,000
Weighted average yield (in percent)		17.00%	17.00%
Weighted average to maturity		1 year 7 months 2 days	1 year 10 months 2 days
Optional extension period		2 years
Term Notes Backed By MSR Related Collateral
Debt Securities, Available-for-sale [Line Items]
Number of options to extend maturity | option			1
Step up in coupon rate (in percent)				0.75%
CRT securities
Debt Securities, Available-for-sale [Line Items]
CRT securities		$ 84,400,000	$ 83,200,000
Non-Agency MBS
Debt Securities, Available-for-sale [Line Items]
Securities, at fair value		23,600,000	23,800,000
Gross unrealized losses		300,000	100,000
CRT, Fair Value Option
Debt Securities, Available-for-sale [Line Items]
Securities, at fair value		51,700,000	51,200,000
Debt securities, sold		0
Gross unrealized losses		0	$ 0
AFS securities
Debt Securities, Available-for-sale [Line Items]
Gross unrealized losses		0
Allowance for loan loss		$ 0			$ 0

[1] See Note 6 for information regarding the Company’s pledged assets.