Other Assets (Details) - USD ($) $ in Thousands	Mar. 31, 2017	Dec. 31, 2016	Mar. 31, 2016	Dec. 31, 2015
Other Assets [Abstract]
REO	$ 98,708	$ 80,503	$ 43,291	$ 28,026
Corporate loan	93,002	85,800
Interest receivable	27,465	27,795
Swaps, at fair value	119	233
Goodwill	7,189	7,189
Prepaid and other assets	60,012	78,775
Total Other Assets	$ 286,495	$ 280,295