Other Assets - Real Estate Owned - Activity (Details) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2024 USD ($) property	Jun. 30, 2023 USD ($) property	Jun. 30, 2024 USD ($) property	Jun. 30, 2023 USD ($) property
Real Estate Owned [Roll Forward]
Balance at beginning of period	$ 106,185	$ 121,215	$ 110,174	$ 130,605
Adjustments to record at lower of cost or fair value	(2,690)	(1,800)	(3,957)	(2,853)
Transfer from residential whole loans	25,609	28,256	44,951	49,181
Purchases and capital improvements, net	52	118	203	256
Disposals and other	(21,143)	(27,793)	(43,358)	(57,193)
Balance at end of period	$ 108,013	$ 119,996	$ 108,013	$ 119,996
Number of properties | property	352	338	352	338
Gain (loss) recorded on transfer from residential whole loans to real estate owned	$ (1,200)	$ 1,200	$ (2,400)	$ 1,300
Properties sold during period | property	63	95	136	194
Proceeds from sale of real estate	$ 25,600	$ 31,700	$ 49,800	$ 65,500
Gain on sales of real estate owned	$ 4,500	$ 4,000	$ 6,500	$ 9,000