Fair Value of Financial Instruments - Portion of Net Gains (Losses) Attributable to Level 3 Assets and Liabilities Still Held Included in Net Income (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2017	Sep. 30, 2016	Sep. 30, 2017	Sep. 30, 2016
Asset-backed securities issued
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gains (losses) attributable to level 3 liabilities still held included in net income	$ (7,771)	$ 10,522	$ (30,286)	$ (14,419)
Residential Loans Held For Investment at Fair Value | Residential loans at Redwood
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gains (losses) attributable to level 3 assets still held included in net income	14,359	3,818	24,227	32,202
Residential Loans Held For Investment at Fair Value | Residential loans at consolidated Sequoia entities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gains (losses) attributable to level 3 assets still held included in net income	3,497	9,200	22,949	(18,864)
Trading securities retained, at fair value
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gains (losses) attributable to level 3 assets still held included in net income	(36)	8,646	24,452	978
AFS securities retained, at fair value
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gains (losses) attributable to level 3 assets still held included in net income	(3)	0	(248)	(305)
MSRs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gains (losses) attributable to level 3 assets still held included in net income	317	6,549	(1,005)	(36,738)
Loan purchase commitments
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gains (losses) attributable to level 3 assets still held included in net income	2,117	5,381	2,121	5,896
Guarantee Asset
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gains (losses) attributable to level 3 assets still held included in net income	$ (239)	$ 307	$ (1,043)	$ (2,070)