Other Assets and Liabilities - Additional Information (Details) $ in Thousands		3 Months Ended	9 Months Ended
	May 01, 2018 USD ($)	Sep. 30, 2018 USD ($) property	Sep. 30, 2018 USD ($) property	Jun. 30, 2018 USD ($)	Dec. 31, 2017 USD ($) property	Sep. 30, 2017 USD ($)	Jun. 30, 2017 USD ($)	Dec. 31, 2016 USD ($)
Other Assets and Other Liabilities [Line Items]
Mortgage servicing rights		$ 63,785	$ 63,785		$ 63,598
Security purchase deposit		57,982	57,982		0
Investment receivable		36,463	36,463		1,147
Real estate owned (REO)		$ 2,915	2,915		$ 3,354
Amount related to transfers into REO			2,000
REO liquidations			3,000
Unrealized gain resulting from market valuation adjustments on REO			$ 300
Legacy Sequoia
Other Assets and Other Liabilities [Line Items]
Number of REO properties recorded on balance sheet (in properties) | property		10	10		14
5 Arches, LLC
Other Assets and Other Liabilities [Line Items]
Minority interest acquired	20.00%
Payments to acquire equity method investments	$ 10,000
Option to purchase additional equity, option term	1 year
Option to purchase additional remaining equity	$ 40,000
Loan flow purchase agreement, agreement term	1 year
Equity Method Investments		$ 7,000	$ 7,000
Cost method investment, purchase option		4,000	4,000
Equity method investment earnings (losses), excluding intangible asset amortization		200
Equity method investment earnings (losses)		100
Mortgage servicing rights
Other Assets and Other Liabilities [Line Items]
MSRs		63,785	63,785	$ 64,674	$ 63,598	$ 62,928	$ 63,770	$ 118,526
Mortgage servicing rights		5,020,000	5,020,000		$ 5,560,000
Real estate securities
Other Assets and Other Liabilities [Line Items]
Investment receivable		$ 35,000	$ 35,000