Fair Value of Assets and Liabilities - Not Measured at Fair Value (Details) - USD ($) $ in Thousands	Mar. 31, 2018	Dec. 31, 2017
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total assets	$ 3,370,523	$ 3,680,629
Payable to securitization residual certificate holders	1,354	1,479
Total liabilities	2,666,070	2,965,512
Level 1 Inputs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total assets	0	0
Payable to securitization residual certificate holders	0	0
Total liabilities	0	0
Level 2 Inputs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total assets	108,295	107,544
Payable to securitization residual certificate holders	0	0
Total liabilities	0	0
Level 3 Inputs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Total assets	3,262,228	3,573,085
Payable to securitization residual certificate holders	1,354	1,479
Total liabilities	2,666,070	2,965,512
Carrying Amount
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Cash and cash equivalents	405,078	401,719
Restricted cash	226,818	242,570
Servicer reserve receivable	5,895	13,685
Deposits	855	855
Total assets	638,646	658,829
Accrued expenses and other liabilities	14,014	13,856
Accounts payable	3,389	11,151
Payables to investors	108,247	143,310
Payable to securitization residual certificate holders	279,561	310,644
Warehouse notes payable	74,000	32,100
Total liabilities	479,211	511,061
Portion Not Recorded at Fair Value | Level 1 Inputs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Cash and cash equivalents	0	0
Restricted cash	0	0
Servicer reserve receivable	0	0
Deposits	0	0
Total assets	0	0
Accrued expenses and other liabilities	0	0
Accounts payable	0	0
Payables to investors	0	0
Payable to securitization residual certificate holders	0	0
Warehouse notes payable	0	0
Total liabilities	0	0
Portion Not Recorded at Fair Value | Level 2 Inputs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Cash and cash equivalents	405,078	401,719
Restricted cash	226,818	242,570
Servicer reserve receivable	5,895	13,685
Deposits	855	855
Total assets	638,646	658,829
Accrued expenses and other liabilities	0	0
Accounts payable	3,389	11,151
Payables to investors	108,247	143,310
Payable to securitization residual certificate holders	279,561	310,644
Warehouse notes payable	0	0
Total liabilities	391,197	465,105
Portion Not Recorded at Fair Value | Level 3 Inputs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Cash and cash equivalents	0	0
Restricted cash	0	0
Servicer reserve receivable	0	0
Deposits	0	0
Total assets	0	0
Accrued expenses and other liabilities	14,014	13,856
Accounts payable	0	0
Payables to investors	0	0
Payable to securitization residual certificate holders	0	0
Warehouse notes payable	74,000	32,100
Total liabilities	88,014	45,956
Balance at Fair Value
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Cash and cash equivalents	405,078	401,719
Restricted cash	226,818	242,570
Servicer reserve receivable	5,895	13,685
Deposits	855	855
Total assets	638,646	658,829
Accrued expenses and other liabilities	14,014	13,856
Accounts payable	3,389	11,151
Payables to investors	108,247	143,310
Payable to securitization residual certificate holders	279,561	310,644
Warehouse notes payable	74,000	32,100
Total liabilities	$ 479,211	$ 511,061