Securitizations and Variable Interest Entities - Summary of Select Information Related to VIEs (Details) - USD ($) $ in Thousands		Dec. 31, 2017	Dec. 31, 2016
Variable Interest Entity [Line Items]
Restricted cash (1)	[1]	$ 242,570	$ 177,810
Securities available for sale		117,573	287,137
Loans held for investment	[1],[2]	2,932,325	4,295,121
Loans held for sale by Company at fair value	[1]	235,825	9,048
Accrued interest receivable	[1]	33,822	40,299
Other assets	[1]	156,278	69,644
Total assets		4,640,831	5,562,631
Accrued interest payable	[1]	32,992	43,574
Accrued expenses and other liabilities (1)	[1]	228,380	85,619
Notes, certificates and secured borrowings	[1]	2,954,768	4,320,895
Payable to securitization note and residual certificate holders (includes $1,479 and $0 at fair value, respectively) (1)	[1]	312,123	0
Warehouse notes payable	[1]	32,100	0
Total liabilities		3,713,074	4,586,861
Consolidated VIEs
Variable Interest Entity [Line Items]
Restricted cash (1)		34,370	0
Securities available for sale		0
Loans held for investment		1,202,260	2,600,422
Loans held for sale by Company at fair value		60,812	0
Accrued interest receivable		15,602	24,037
Other assets		6,324	0
Total assets		1,670,067	2,624,459
Accrued interest payable		14,789	26,839
Accrued expenses and other liabilities (1)		52	0
Notes, certificates and secured borrowings		1,210,349	2,616,023
Payable to securitization note and residual certificate holders (includes $1,479 and $0 at fair value, respectively) (1)		312,123	0
Warehouse notes payable		32,100	0
Total liabilities		1,569,413	2,642,862
Net Assets		100,654	(18,403)
Unconsolidated VIEs
Variable Interest Entity [Line Items]
Restricted cash (1)		0
Securities available for sale		47,049
Loans held for investment		0	0
Loans held for sale by Company at fair value		0
Accrued interest receivable		407	0
Other assets		15,779	10,122
Total assets		63,235	10,122
Accrued interest payable		0	0
Accrued expenses and other liabilities (1)		300
Notes, certificates and secured borrowings		0	0
Payable to securitization note and residual certificate holders (includes $1,479 and $0 at fair value, respectively) (1)		0
Warehouse notes payable		0
Total liabilities		300	0
Net Assets		62,935	10,122
Total
Variable Interest Entity [Line Items]
Restricted cash (1)		34,370
Securities available for sale		47,049
Loans held for investment		1,202,260	2,600,422
Loans held for sale by Company at fair value		60,812
Accrued interest receivable		16,009	24,037
Other assets		22,103	10,122
Total assets		1,733,302	2,634,581
Accrued interest payable		14,789	26,839
Accrued expenses and other liabilities (1)		352
Notes, certificates and secured borrowings		1,210,349	2,616,023
Payable to securitization note and residual certificate holders (includes $1,479 and $0 at fair value, respectively) (1)		312,123
Warehouse notes payable		32,100
Total liabilities		1,569,713	2,642,862
Net Assets		163,589	$ (8,281)
Loans Invested in by Company | Consolidated VIEs
Variable Interest Entity [Line Items]
Loans held for investment		350,699
Loans Invested in by Company | Unconsolidated VIEs
Variable Interest Entity [Line Items]
Loans held for investment		0
Loans Invested in by Company | Total
Variable Interest Entity [Line Items]
Loans held for investment		$ 350,699

[1] Includes amounts in consolidated variable interest entities (VIEs) presented separately in the table below.

[2] Prior period amounts have been reclassified to conform to the current period presentation. See “Notes to Consolidated Financial Statements – Note 1. Basis of Presentation” for additional information.