Other Assets (Details) - USD ($) $ in Thousands	Jun. 30, 2016	Mar. 31, 2016	Dec. 31, 2015	Jun. 30, 2015	Mar. 31, 2015	Dec. 31, 2014
Other Assets [Abstract]
REO	$ 56,784	$ 43,291	$ 28,026	$ 8,857	$ 8,333	$ 5,492
Interest receivable	28,034		29,002
Swaps, at fair value	0		1,127
Goodwill	7,189		7,189
Prepaid and other assets	135,548		101,455
Total Other Assets	$ 227,555		$ 166,799