Operating Group Information - Sales by operating group (Details) - USD ($) $ in Thousands	3 Months Ended
	May 02, 2020	May 04, 2019
Operating groups
Net sales	$ 160,343	$ 281,973
Operating Segments | Tommy Bahama
Operating groups
Net sales	86,984	164,730
Operating Segments | Lilly Pulitzer
Operating groups
Net sales	49,149	72,595
Operating Segments | Lanier Apparel
Operating groups
Net sales	10,725	26,154
Operating Segments | Southern Tide
Operating groups
Net sales	8,301	14,134
Corporate and Other
Operating groups
Net sales	5,184	4,360
Retail
Operating groups
Net sales	$ 45,503	$ 108,006
Net sales (as a percent)	28.00%	38.00%
Retail | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	39.00%	47.00%
Retail | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	24.00%	42.00%
E-commerce
Operating groups
Net sales	$ 53,694	$ 47,989
Net sales (as a percent)	34.00%	17.00%
E-commerce | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	28.00%	15.00%
E-commerce | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	50.00%	25.00%
E-commerce | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	21.00%	13.00%
E-commerce | Corporate and Other
Operating groups
Net sales (as a percent)	63.00%	60.00%
Restaurant
Operating groups
Net sales	$ 11,763	$ 23,601
Net sales (as a percent)	7.00%	9.00%
Restaurant | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	13.00%	14.00%
Wholesale
Operating groups
Net sales	$ 49,058	$ 101,926
Net sales (as a percent)	31.00%	36.00%
Wholesale | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	20.00%	24.00%
Wholesale | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	26.00%	33.00%
Wholesale | Operating Segments | Lanier Apparel
Operating groups
Net sales (as a percent)	100.00%	100.00%
Wholesale | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	79.00%	87.00%
Wholesale | Corporate and Other
Operating groups
Net sales (as a percent)	32.00%	32.00%
Other
Operating groups
Net sales	$ 325	$ 451
Other | Corporate and Other
Operating groups
Net sales (as a percent)	5.00%	8.00%