Other Assets (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Other Assets [Abstract]
REO	$ 411,659	$ 249,413	$ 152,356
Capital contributions made to loan origination partners	147,992	23,210
Other interest-earning assets	70,468	92,022
MBS and loan related receivables	114,828	130,964
Other	39,304	32,176
Total Other Assets	784,251	$ 527,785
REO held for investment	$ 27,300