Loans, Loans Held For Sale, Notes, Certificates and Secured Borrowings and Loan Servicing Rights - Outstanding loans (Details) $ in Thousands	Dec. 31, 2017 USD ($) Loan	Dec. 31, 2016 USD ($) Loan
Loans and Loans Held for Sale
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, past 90 days	$ 36,588	$ 45,207
Outstanding principal balance, non-accrual	3,289	4,965
Net fair value adjustments, past 90 days	(30,071)	(39,734)
Net fair value adjustments, non-accural	(2,675)	(4,312)
Fair value, past 90 days	6,517	5,473
Fair value, non-accrual	$ 614	$ 653
Number of loans (not in thousands), past 90 days | Loan	3,779	3,887
Number of loans (not in thousands), non-accrual | Loan	591	465
Loans Invested in by Company
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Outstanding principal balance, past 90 days	$ 1,015	$ 511
Outstanding principal balance, non-accrual	122	90
Net fair value adjustments, past 90 days	(861)	(449)
Net fair value adjustments, non-accural	(107)	(80)
Fair value, past 90 days	154	62
Fair value, non-accrual	$ 15	$ 10
Number of loans (not in thousands), past 90 days | Loan	257	154
Number of loans (not in thousands), non-accrual | Loan	34	18