Other Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Other Liabilities [Abstract]
Securitized debt	$ 570,952	$ 684,420
Convertible Debt	223,971	0
Senior Notes	96,862	96,816
Dividends and dividend equivalents payable	90,749	90,198
Accrued interest payable	18,238	16,280
Payable for unsettled residential whole loans purchases	0	211,129
Accrued expenses and other	42,819	26,296
Total Other Liabilities	$ 1,043,591	$ 1,125,139