Debt, net Table of Debt Obligations, Net of Discounts and Deferred Financing Costs (Details) - USD ($) $ in Thousands			6 Months Ended
		May 04, 2018	Jun. 30, 2018		Jun. 21, 2018	Apr. 16, 2018	Dec. 31, 2017
Debt Instrument [Line Items]
Long-term Debt, Gross			$ 376,485				$ 356,104
Unamortized discount, net			(659)				(191)
Unamortized deferred financing costs			(9,611)				(9,832)
Debt, net			366,215				346,081
Secured corporate credit agreements
Debt Instrument [Line Items]
Long-term Debt, Gross			74,030				28,500
Long-term Debt, Other Disclosures [Abstract]
Debt instrument, maximum borrowing capacity		$ 75,000	$ 155,000
Secured corporate credit agreements | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Description of Variable Rate Basis			LIBOR
Junior subordinated notes
Debt Instrument [Line Items]
Long-term Debt, Gross			$ 125,000				125,000
Long-term Debt, Other Disclosures [Abstract]
Debt instrument, maximum borrowing capacity			125,000
Preferred trust securities
Debt Instrument [Line Items]
Long-term Debt, Gross			35,000				35,000
Long-term Debt, Other Disclosures [Abstract]
Debt instrument, maximum borrowing capacity			$ 35,000
Preferred trust securities | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Description of Variable Rate Basis			LIBOR
Corporate Debt Payable
Debt Instrument [Line Items]
Long-term Debt, Gross			$ 234,030				188,500
Asset based revolving financing (1) (2)
Debt Instrument [Line Items]
Long-term Debt, Gross	[1],[2]		100,557				$ 118,794
Long-term Debt, Other Disclosures [Abstract]
Debt instrument, maximum borrowing capacity			$ 175,000
Long-term Debt, Weighted Average Interest Rate, at Point in Time			4.35%				3.73%
Asset based revolving financing (1) (2) | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Description of Variable Rate Basis			LIBOR
Residential mortgage warehouse borrowings (3)
Debt Instrument [Line Items]
Long-term Debt, Gross	[3]		$ 41,898	[1]	$ 25,000	$ 50,000	$ 48,810	[1]
Long-term Debt, Other Disclosures [Abstract]
Debt instrument, maximum borrowing capacity			$ 76,000
Long-term Debt, Weighted Average Interest Rate, at Point in Time			4.55%				3.70%
Residential mortgage warehouse borrowings (3) | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Description of Variable Rate Basis			LIBOR
Asset Backed Debt Payable
Debt Instrument [Line Items]
Long-term Debt, Gross			$ 142,455				$ 167,604
Minimum | Secured corporate credit agreements | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Basis Spread on Variable Rate		1.25%	1.00%
Minimum | Asset based revolving financing (1) (2) | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Basis Spread on Variable Rate	[1],[2]		2.25%
Minimum | Residential mortgage warehouse borrowings (3) | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Basis Spread on Variable Rate	[3]		2.50%
Maximum | Secured corporate credit agreements | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Basis Spread on Variable Rate		5.50%	5.50%
Maximum | Junior subordinated notes
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Interest Rate, Stated Percentage			8.50%
Maximum | Preferred trust securities
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Interest Rate, Stated Percentage			4.10%
Maximum | Asset based revolving financing (1) (2) | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Basis Spread on Variable Rate	[1],[2]		2.60%
Maximum | Residential mortgage warehouse borrowings (3) | London Interbank Offered Rate (LIBOR)
Long-term Debt, Other Disclosures [Abstract]
Debt Instrument, Basis Spread on Variable Rate	[3]		3.00%

[1] Asset based debt is generally recourse only to specific assets and related cash flows.

[2] The weighted average coupon rate for asset based revolving financing was 4.35% and 3.73% at June 30, 2018 and December 31, 2017, respectively.

[3] The weighted average coupon rate for residential mortgage warehouse borrowings was 4.55% and 3.70% at June 30, 2018 and December 31, 2017, respectively.