Residential Loans - Geographic Concentration of Residential Loans Recorded on Consolidated Balance Sheet (Details) - Residential loans, held-for-investment - Geographic Concentration Risk - Loan Benchmark	12 Months Ended
	Dec. 31, 2021	Dec. 31, 2020
Concentration Risk [Line Items]
Concentration risk percentage	100.00%	100.00%
California
Concentration Risk [Line Items]
Concentration risk percentage	29.00%	43.00%
Texas
Concentration Risk [Line Items]
Concentration risk percentage	11.00%	10.00%
Washington
Concentration Risk [Line Items]
Concentration risk percentage	8.00%	7.00%
Colorado
Concentration Risk [Line Items]
Concentration risk percentage	7.00%	5.00%
Florida
Concentration Risk [Line Items]
Concentration risk percentage	6.00%	3.00%
Arizona
Concentration Risk [Line Items]
Concentration risk percentage	5.00%
New York
Concentration Risk [Line Items]
Concentration risk percentage	2.00%	1.00%
New Jersey
Concentration Risk [Line Items]
Concentration risk percentage	1.00%	1.00%
Illinois
Concentration Risk [Line Items]
Concentration risk percentage	2.00%	3.00%
MARYLAND
Concentration Risk [Line Items]
Concentration risk percentage	1.00%	2.00%
Ohio
Concentration Risk [Line Items]
Concentration risk percentage	0.00%	0.00%
Other states (none greater than 5%)
Concentration Risk [Line Items]
Concentration risk percentage	28.00%	25.00%
Legacy Sequoia
Concentration Risk [Line Items]
Concentration risk percentage	100.00%	100.00%
Legacy Sequoia | California
Concentration Risk [Line Items]
Concentration risk percentage	18.00%	17.00%
Legacy Sequoia | Texas
Concentration Risk [Line Items]
Concentration risk percentage	5.00%	5.00%
Legacy Sequoia | Washington
Concentration Risk [Line Items]
Concentration risk percentage	1.00%	1.00%
Legacy Sequoia | Colorado
Concentration Risk [Line Items]
Concentration risk percentage	2.00%	3.00%
Legacy Sequoia | Florida
Concentration Risk [Line Items]
Concentration risk percentage	14.00%	14.00%
Legacy Sequoia | Arizona
Concentration Risk [Line Items]
Concentration risk percentage	1.00%
Legacy Sequoia | New York
Concentration Risk [Line Items]
Concentration risk percentage	11.00%	10.00%
Legacy Sequoia | New Jersey
Concentration Risk [Line Items]
Concentration risk percentage	5.00%	5.00%
Legacy Sequoia | Illinois
Concentration Risk [Line Items]
Concentration risk percentage	3.00%	3.00%
Legacy Sequoia | MARYLAND
Concentration Risk [Line Items]
Concentration risk percentage	2.00%	2.00%
Legacy Sequoia | Ohio
Concentration Risk [Line Items]
Concentration risk percentage	5.00%	5.00%
Legacy Sequoia | Other states (none greater than 5%)
Concentration Risk [Line Items]
Concentration risk percentage	33.00%	35.00%
Sequoia
Concentration Risk [Line Items]
Concentration risk percentage	100.00%	100.00%
Sequoia | California
Concentration Risk [Line Items]
Concentration risk percentage	35.00%	34.00%
Sequoia | Texas
Concentration Risk [Line Items]
Concentration risk percentage	12.00%	10.00%
Sequoia | Washington
Concentration Risk [Line Items]
Concentration risk percentage	5.00%	5.00%
Sequoia | Colorado
Concentration Risk [Line Items]
Concentration risk percentage	6.00%	3.00%
Sequoia | Florida
Concentration Risk [Line Items]
Concentration risk percentage	4.00%	6.00%
Sequoia | Arizona
Concentration Risk [Line Items]
Concentration risk percentage	3.00%
Sequoia | New York
Concentration Risk [Line Items]
Concentration risk percentage	2.00%	5.00%
Sequoia | New Jersey
Concentration Risk [Line Items]
Concentration risk percentage	1.00%	2.00%
Sequoia | Illinois
Concentration Risk [Line Items]
Concentration risk percentage	4.00%	2.00%
Sequoia | MARYLAND
Concentration Risk [Line Items]
Concentration risk percentage	2.00%	1.00%
Sequoia | Ohio
Concentration Risk [Line Items]
Concentration risk percentage	0.00%	0.00%
Sequoia | Other states (none greater than 5%)
Concentration Risk [Line Items]
Concentration risk percentage	26.00%	32.00%
Freddie Mac SLST(1)
Concentration Risk [Line Items]
Concentration risk percentage	100.00%	100.00%
Freddie Mac SLST(1) | California
Concentration Risk [Line Items]
Concentration risk percentage	14.00%	14.00%
Freddie Mac SLST(1) | Texas
Concentration Risk [Line Items]
Concentration risk percentage	3.00%	3.00%
Freddie Mac SLST(1) | Washington
Concentration Risk [Line Items]
Concentration risk percentage	2.00%	2.00%
Freddie Mac SLST(1) | Colorado
Concentration Risk [Line Items]
Concentration risk percentage	1.00%	1.00%
Freddie Mac SLST(1) | Florida
Concentration Risk [Line Items]
Concentration risk percentage	10.00%	10.00%
Freddie Mac SLST(1) | Arizona
Concentration Risk [Line Items]
Concentration risk percentage	2.00%
Freddie Mac SLST(1) | New York
Concentration Risk [Line Items]
Concentration risk percentage	10.00%	10.00%
Freddie Mac SLST(1) | New Jersey
Concentration Risk [Line Items]
Concentration risk percentage	7.00%	7.00%
Freddie Mac SLST(1) | Illinois
Concentration Risk [Line Items]
Concentration risk percentage	5.00%	5.00%
Freddie Mac SLST(1) | MARYLAND
Concentration Risk [Line Items]
Concentration risk percentage	5.00%	5.00%
Freddie Mac SLST(1) | Ohio
Concentration Risk [Line Items]
Concentration risk percentage	2.00%	2.00%
Freddie Mac SLST(1) | Other states (none greater than 5%)
Concentration Risk [Line Items]
Concentration risk percentage	39.00%	41.00%