Loans Held For Investment, Loans Held For Sale, Notes, Certificates and Secured Borrowings and Loan Servicing Rights - Past Due Table (Details) $ in Thousands	Mar. 31, 2018 USD ($) Loan	Dec. 31, 2017 USD ($) Loan
Loans and Loans Held for Sale
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, 90 days or more past due	$ 27,477	$ 36,588
Outstanding principal balance, non-accrual loans		3,289
Net fair value adjustments, 90 days or more past due	(22,558)	(30,071)
Net fair value adjustments, non-accrual loans		(2,675)
Fair value, past 90 days due	$ 4,919	6,517
Fair value, non-accrual loans		$ 614
Number of loans, 90 days past due | Loan	2,852	3,779
Number of loans, non-accrual loans | Loan		591
Loans Invested in by Company
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, 90 days or more past due	$ 2,124	$ 1,015
Outstanding principal balance, non-accrual loans		122
Net fair value adjustments, 90 days or more past due	(1,749)	(861)
Net fair value adjustments, non-accrual loans		(107)
Fair value, past 90 days due	$ 375	154
Fair value, non-accrual loans		$ 15
Number of loans, 90 days past due | Loan	317	257
Number of loans, non-accrual loans | Loan		34