Operating Group Information - Sales by operating group (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Oct. 30, 2021	Oct. 31, 2020	Oct. 30, 2021	Oct. 31, 2020
Operating groups
Net sales	$ 247,729	$ 175,135	$ 842,163	$ 527,466
Operating Segments | Tommy Bahama
Operating groups
Net sales	148,454	94,905	513,985	277,143
Operating Segments | Lilly Pulitzer
Operating groups
Net sales	72,157	53,714	233,066	176,723
Operating Segments | Southern Tide
Operating groups
Net sales	13,151	10,023	43,204	27,136
Operating Segments | Lanier Apparel
Operating groups
Net sales	4,232	10,810	24,743	29,985
Corporate and Other
Operating groups
Net sales	9,735	5,683	27,165	16,479
Retail
Operating groups
Net sales	$ 92,579	$ 47,415	$ 319,493	$ 134,701
Net sales (as a percent)	37.00%	27.00%	38.00%	26.00%
Retail | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	47.00%	37.00%	46.00%	36.00%
Retail | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	31.00%	22.00%	35.00%	19.00%
Retail | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	7.00%	4.00%	7.00%	3.00%
E-commerce
Operating groups
Net sales	$ 82,402	$ 66,135	$ 261,393	$ 226,618
Net sales (as a percent)	33.00%	38.00%	31.00%	43.00%
E-commerce | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	22.00%	27.00%	24.00%	34.00%
E-commerce | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	58.00%	66.00%	48.00%	64.00%
E-commerce | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	21.00%	24.00%	23.00%	29.00%
E-commerce | Corporate and Other
Operating groups
Net sales (as a percent)	49.00%	54.00%	53.00%	65.00%
Restaurant
Operating groups
Net sales	$ 19,748	$ 12,214	$ 70,784	$ 32,505
Net sales (as a percent)	8.00%	7.00%	8.00%	6.00%
Restaurant | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	13.00%	13.00%	14.00%	12.00%
Wholesale
Operating groups
Net sales	$ 52,658	$ 49,221	$ 189,133	$ 132,768
Net sales (as a percent)	21.00%	28.00%	22.00%	25.00%
Wholesale | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	18.00%	23.00%	16.00%	18.00%
Wholesale | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	11.00%	12.00%	17.00%	17.00%
Wholesale | Operating Segments | Southern Tide
Operating groups
Net sales (as a percent)	72.00%	72.00%	70.00%	68.00%
Wholesale | Operating Segments | Lanier Apparel
Operating groups
Net sales (as a percent)	100.00%	100.00%	100.00%	100.00%
Wholesale | Corporate and Other
Operating groups
Net sales (as a percent)	48.00%	44.00%	43.00%	31.00%
Other
Operating groups
Net sales	$ 342	$ 150	$ 1,360	$ 874
Other | Corporate and Other
Operating groups
Net sales (as a percent)	3.00%	2.00%	4.00%	4.00%