Commercial Mortgage and Subordinate Loans, Net - Schedule of Mortgage Loans by Property Type and Geographic Distribution (Details) - USD ($) $ in Thousands	6 Months Ended	12 Months Ended
	Jun. 30, 2019	Dec. 31, 2018
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 5,443,744	$ 4,927,593
% of Portfolio	100.00%	100.00%
Hotel
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 1,322,167	$ 1,286,590
% of Portfolio	24.30%	26.10%
Residential-for-sale: construction
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 706,260	$ 528,510
% of Portfolio	12.90%	10.70%
Residential-for-sale: inventory
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 395,375	$ 577,053
% of Portfolio	7.30%	11.70%
Office
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 1,202,311	$ 832,620
% of Portfolio	22.20%	16.90%
Urban Predevelopment
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 595,263	$ 683,886
% of Portfolio	10.80%	13.90%
Multifamily
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 472,785	$ 448,899
% of Portfolio	8.70%	9.10%
Industrial
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 227,451	$ 32,000
% of Portfolio	4.20%	0.60%
Retail Center
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 158,128	$ 156,067
% of Portfolio	2.90%	3.20%
Healthcare
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 142,933	$ 156,814
% of Portfolio	2.60%	3.20%
Other
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 120,041	$ 151,197
% of Portfolio	2.20%	3.10%
Mixed Use
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 101,030	$ 73,957
% of Portfolio	1.90%	1.50%
Manhattan, NY
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 1,883,438	$ 1,669,145
% of Portfolio	34.60%	33.90%
Brooklyn, NY
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 553,859	$ 346,056
% of Portfolio	10.20%	7.00%
Northeast
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 18,738	$ 23,479
% of Portfolio	0.30%	0.50%
West
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 696,804	$ 614,160
% of Portfolio	12.80%	12.50%
Midwest
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 638,981	$ 631,710
% of Portfolio	11.80%	12.80%
Southeast
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 501,878	$ 559,043
% of Portfolio	9.20%	11.30%
Southwest
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 119,910	$ 96,345
% of Portfolio	2.20%	2.00%
Mid Atlantic
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 108,056	$ 211,775
% of Portfolio	2.00%	4.30%
United Kingdom
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 654,952	$ 700,460
% of Portfolio	12.00%	14.20%
Other International
Loans and Leases Receivable Disclosure [Line Items]
Carrying value	$ 267,128	$ 75,420
% of Portfolio	4.90%	1.50%