Other Liabilities (Details) - USD ($) $ in Thousands	Jun. 30, 2018	Dec. 31, 2017	Jun. 30, 2017
Other Liabilities [Abstract]
Securitized debt	$ 518,655	$ 363,944
Obligation to return securities held as collateral, at fair value	253,721	504,062
Senior Notes	96,794	96,773
Dividends and dividend equivalents payable	79,948	79,771	$ 79,559
Accrued interest payable	11,018	12,263
Accrued expenses and other liabilities	17,871	21,584
Total Other Liabilities	$ 978,007	$ 1,078,397