Stockholders' Equity - Components of Accumulated Other Comprehensive Income (Details) - USD ($) $ in Thousands	3 Months Ended				6 Months Ended
	Jun. 30, 2022	Mar. 31, 2022	Jun. 30, 2021	Mar. 31, 2021	Jun. 30, 2022	Jun. 30, 2021
Accumulated Other Comprehensive Income (Loss) [Line Items]
Unrealized gain (loss) on mortgage-backed securities, net	$ (1,825)		$ 1,155		$ (4,246)	$ 2,136
Reclassification of amortization of net deferred (gain) loss on de-designated interest rate swaps to repurchase agreements interest expense	(4,802)		(5,429)		(9,998)	(10,797)
Currency translation adjustments on investment in unconsolidated venture	(93)		(632)		(293)	(23)
Total other comprehensive income (loss)	(6,720)		(4,906)		(14,537)	(8,684)
AOCI Including Portion Attributable to Noncontrolling Interest, Net of Tax [Roll Forward]
Beginning balance	1,127,947	$ 1,402,135	1,481,932	$ 1,367,158	1,402,135	1,367,158
Total other comprehensive income (loss)	(6,720)	(7,817)	(4,906)	(3,778)	(14,537)	(8,684)
Ending balance	959,463	1,127,947	1,373,379	1,481,932	959,463	1,373,379
Equity method investments
Accumulated Other Comprehensive Income (Loss) [Line Items]
Currency translation adjustments on investment in unconsolidated venture	(93)		(632)		(293)	(23)
Total other comprehensive income (loss)	(93)		(632)		(293)	(23)
AOCI Including Portion Attributable to Noncontrolling Interest, Net of Tax [Roll Forward]
Beginning balance	224	424	1,108	499	424	499
Total other comprehensive income (loss)	(93)		(632)		(293)	(23)
Ending balance	131	224	476	1,108	131	476
Available-for-sale securities
Accumulated Other Comprehensive Income (Loss) [Line Items]
Unrealized gain (loss) on mortgage-backed securities, net	(1,825)		1,155		(4,246)	2,136
Total other comprehensive income (loss)	(1,825)		1,155		(4,246)	2,136
AOCI Including Portion Attributable to Noncontrolling Interest, Net of Tax [Roll Forward]
Beginning balance	4,328	6,749	6,974	5,993	6,749	5,993
Total other comprehensive income (loss)	(1,825)		1,155		(4,246)	2,136
Ending balance	2,503	4,328	8,129	6,974	2,503	8,129
Derivatives and hedging
Accumulated Other Comprehensive Income (Loss) [Line Items]
Reclassification of amortization of net deferred (gain) loss on de-designated interest rate swaps to repurchase agreements interest expense	(4,802)		(5,429)		(9,998)	(10,797)
Total other comprehensive income (loss)	(4,802)		(5,429)		(9,998)	(10,797)
AOCI Including Portion Attributable to Noncontrolling Interest, Net of Tax [Roll Forward]
Beginning balance	24,917	30,113	46,745	52,113	30,113	52,113
Total other comprehensive income (loss)	(4,802)		(5,429)		(9,998)	(10,797)
Ending balance	20,115	24,917	41,316	46,745	20,115	41,316
Accumulated other comprehensive income
AOCI Including Portion Attributable to Noncontrolling Interest, Net of Tax [Roll Forward]
Beginning balance	29,469	37,286	54,827	58,605	37,286	58,605
Total other comprehensive income (loss)	(6,720)	(7,817)	(4,906)	(3,778)
Ending balance	$ 22,749	$ 29,469	$ 49,921	$ 54,827	$ 22,749	$ 49,921