Loans Held for Investment, Loans Held for Sale, Notes, Certificates and Secured Borrowings - Outstanding loans (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2019 USD ($) Loan	Dec. 31, 2018 USD ($) Loan
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Non-accrual status threshold	90 days
Loans and Loans Held for Sale
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Non-accrual status threshold	90 days
Loans Held for Investment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, past 90 days and non-accrual	$ 10,755	$ 19,707
Net fair vale adjustments, past 90 days and non-accrual	(9,663)	(16,166)
Fair value, past 90 days and non-accrual	$ 1,092	$ 3,541
Number of loans 90 days or more past due and nonaccrual | Loan	1,428	2,309
Loans Invested in by Company
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, past 90 days and non-accrual	$ 2,315	$ 2,060
Net fair vale adjustments, past 90 days and non-accrual	(2,016)	(1,710)
Fair value, past 90 days and non-accrual	$ 299	$ 350
Number of loans 90 days or more past due and nonaccrual | Loan	338	356