Other Assets (Details) - USD ($) $ in Thousands		Jun. 30, 2023	Mar. 31, 2023	Dec. 31, 2022	Jun. 30, 2022	Mar. 31, 2022	Dec. 31, 2021
Deferred Costs, Capitalized, Prepaid, and Other Assets Disclosure [Abstract]
Receivable for sale of unsettled residential whole loans		$ 0		$ 275,656
REO		119,996	$ 121,215	130,605	$ 135,847	$ 145,568	$ 156,223
Goodwill		61,076		61,076
Intangibles, net		9,600		12,200
Capital contributions made to loan origination partners		27,437		28,308
Other interest-earning assets		70,473		63,964
Interest receivable		79,490		68,704
Other loan related receivables		25,745		23,463
Lease right-of-use asset		38,340		39,459
Other		66,598		62,786
Total Other Assets	[1]	$ 498,755		$ 766,221
Operating Lease, Right-of-Use Asset, Statement of Financial Position [Extensible Enumeration]		Total Other Assets		Total Other Assets
Accumulated amortization		$ 18,400		$ 15,800
Borrowing rate		7.50%

[1] See Note 6 for information regarding the Company’s pledged assets.