Loans Held for Investment - Changes in Carrying Value of Securitized Loans Held For Investment Carried at Fair Value (Details) - Loans Held for Investment at Fair Value - Variable Interest Entities, Primary Beneficiary - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2020	Dec. 31, 2019
SEC Schedule, 12-29, Real Estate Companies, Investment in Movement in Mortgage Loans on Real Estate [Roll Forward]
Balance, beginning of period	$ 14,292,815	$ 12,572,581
Purchases	1,860,998	5,086,491
Principal paydowns	(1,966,590)	(1,717,745)
Sales and settlements	(1,053,943)	(1,812,760)
Net periodic accretion (amortization)	(85,794)	(77,491)
Realized gains (losses) on sales and settlements	0	1,590
Change in fair value	64,643	240,149
Balance, end of period	$ 13,112,129	$ 14,292,815