Other Liabilities (Details) - USD ($) $ in Thousands	Jun. 30, 2021	Dec. 31, 2020
Other Liabilities [Abstract]
Payable for unsettled residential whole loans purchases	$ 131,275	$ 0
Dividends and dividend equivalents payable	44,236	34,016
Lease liability	41,680	636
Accrued interest payable	6,406	11,116
Accrued expenses and other	29,484	24,754
Total Other Liabilities	$ 253,081	$ 70,522