Other Assets - Real Estate Owned - Activity (Details) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2023 USD ($) property	Jun. 30, 2022 USD ($) property	Jun. 30, 2023 USD ($) property	Jun. 30, 2022 USD ($) property	Dec. 31, 2022 property
Real Estate Owned [Roll Forward]
Balance at beginning of period	$ 121,215	$ 145,568	$ 130,605	$ 156,223
Adjustments to record at lower of cost or fair value	(1,800)	(324)	(2,853)	(772)
Transfer from residential whole loans	28,256	23,138	49,181	45,217
Purchases and capital improvements, net	118	277	256	629
Disposals and other	(27,793)	(32,812)	(57,193)	(65,450)
Balance at end of period	$ 119,996	$ 135,847	$ 119,996	$ 135,847
Number of properties | property	338	438	338	438	388
Properties sold during period | property	95	124	194	259
Proceeds from sale of real estate	$ 31,700	$ 39,600	$ 65,500	$ 81,200
Gain on sales of real estate owned	$ 4,000	$ 3,900	$ 9,000	$ 6,100