Other Assets - Real Estate Owned - Activity (Details) $ in Thousands	3 Months Ended
	Mar. 31, 2024 USD ($) property	Mar. 31, 2023 USD ($) property	Dec. 31, 2023 property
Real Estate Owned [Roll Forward]
Balance at beginning of period	$ 110,174	$ 130,605
Adjustments to record at lower of cost or fair value	(1,267)	(1,052)
Transfer from residential whole loans	19,342	20,925
Purchases and capital improvements, net	151	137
Disposals and other	(22,215)	(29,400)
Balance at end of period	$ 106,185	$ 121,215
Number of properties | property	288	362	300
Gain (loss) recorded on transfer from residential whole loans to real estate owned	$ (1,100)	$ 41
Properties sold during period | property	73	93
Proceeds from sale of real estate	$ 24,200	$ 33,800
Gain on sales of real estate owned	$ 2,000	$ 5,000