Other Assets (Details) - USD ($) $ in Thousands	Sep. 30, 2016	Jun. 30, 2016	Dec. 31, 2015	Sep. 30, 2015	Jun. 30, 2015	Dec. 31, 2014
Other Assets [Abstract]
REO	$ 73,189	$ 56,784	$ 28,026	$ 14,525	$ 8,857	$ 5,492
Interest receivable	27,842		29,002
Swaps, at fair value	121		1,127
Goodwill	7,189		7,189
Prepaid and other assets	99,708		101,455
Total Other Assets	$ 208,049		$ 166,799