Other Assets and Other Liabilities and Accrued Expenses - Schedule of Other Liabilities and Accrued Expenses (Details) - USD ($) $ in Thousands	Mar. 31, 2021	Dec. 31, 2020
Deferred Costs, Capitalized, Prepaid, and Other Assets Disclosure [Abstract]
Accounts payable and accrued expenses	$ 71,556	$ 106,142
Operating lease liability	32,700	32,914
Deferred tax liabilities, net	30,692	24,183
Securities sold, not yet purchased	39,625	46,637
Due to brokers	52,151	45,047
Loans eligible for repurchase liability	75,952	70,593
Commissions payable	11,371	18,678
Other	21,476	18,671
Total other liabilities and accrued expenses	$ 335,523	$ 362,865