Real Estate Securities, Available for Sale - Narrative (Details)	1 Months Ended	3 Months Ended	12 Months Ended
	Jul. 31, 2019 USD ($) securitization_trust	Dec. 31, 2020 USD ($) securitization_trust loan security	Dec. 31, 2020 USD ($) securitization_trust security loan	Dec. 31, 2019 USD ($) security	Dec. 31, 2018 USD ($)
Debt Securities, Available-for-sale [Line Items]
Gross sales price of securities sold			$ 149,629,000	$ 0	$ 0
Loss from sale of securities			42,153,000	0	0
Unrealized gains (losses) in OCI		$ 16,300,000		17,800,000
Realized loss on securities from write-down to fair value			32,606,000	0	0
Securities in unrealized loss position		$ 0	$ 0
Interest receivable				700,000
Number of securitization trusts held | securitization_trust		2	2
Proceeds from sale of mortgage loans held in securitization trusts			$ 0	39,848,000	$ 0
Gross assets		$ 6,211,937,000	6,211,937,000	7,414,306,000
Gross liabilities		4,253,259,000	4,253,259,000	5,212,956,000
Primary beneficiary, securitization trust
Debt Securities, Available-for-sale [Line Items]
Number of securitization trusts sold | securitization_trust	1
Proceeds from sale of mortgage loans held in securitization trusts	$ 33,400,000
Gross assets	1,200,000,000	1,774,713,000	1,774,713,000	1,879,990,000
Gross liabilities	$ 1,200,000,000	1,714,653,000	1,714,653,000	1,769,181,000
Primary beneficiary
Debt Securities, Available-for-sale [Line Items]
Gross assets		3,487,187,000	3,487,187,000	3,502,341,000
Gross liabilities		3,052,848,000	3,052,848,000	3,117,364,000
Mortgage loans held in trust, unpaid principal balance		1,700,000,000	1,700,000,000	1,800,000,000
Mortgage obligations held in trust, unpaid principal balance		$ 1,600,000,000	$ 1,600,000,000	1,600,000,000
Number of underlying mortgage loans | loan		114	114
Weighted average coupon (in percentage)		4.50%	4.50%
Weighted average loan to value ratio (in percentage)		60.00%	60.00%
Difference between held and issued mortgage loans in securitization trusts		$ 59,500,000	$ 59,500,000	$ 110,100,000
Credit facilities | CMBS credit facilities
Debt Securities, Available-for-sale [Line Items]
Debt repaid			$ 104,000,000.0
Real estate securities, available for sale
Debt Securities, Available-for-sale [Line Items]
Number of securities sold | security			41
Gross sales price of securities sold			$ 149,600,000
Loss from sale of securities			$ 42,200,000
Number of securities held | security		2	2	43
Carrying value of securities		$ 10,189,000	$ 10,189,000	$ 236,303,000
Weighted average contractual maturity			28 years 4 months 24 days
Expected maturity			5 years 6 months
Real estate securities, available for sale | CRE Securities
Debt Securities, Available-for-sale [Line Items]
Loss from sale of securities			$ 32,600,000