Operating Group Information - Sales by operating group (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Oct. 29, 2022	Oct. 30, 2021	Oct. 29, 2022	Oct. 30, 2021
Operating groups
Net sales	$ 313,033	$ 247,729	$ 1,029,044	$ 842,163
Operating Segments | Tommy Bahama
Operating groups
Net sales	178,645	148,454	650,677	513,985
Operating Segments | Lilly Pulitzer
Operating groups
Net sales	84,053	72,157	264,763	233,066
Operating Segments | Johnny Was
Operating groups
Net sales	22,661		22,661
Operating Segments | Emerging Brands
Operating groups
Net sales	26,912	22,082	88,588	67,336
Operating Segments | Lanier Apparel
Operating groups
Net sales		4,232		24,743
Corporate and Other
Operating groups
Net sales	762	804	2,355	3,033
Retail
Operating groups
Net sales	$ 112,344	$ 92,579	$ 402,400	$ 319,493
Net sales (as a percent)	36.00%	37.00%	39.00%	38.00%
Retail | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	44.00%	47.00%	46.00%	46.00%
Retail | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	27.00%	31.00%	34.00%	35.00%
Retail | Operating Segments | Johnny Was
Operating groups
Net sales (as a percent)	38.00%		38.00%
Retail | Operating Segments | Emerging Brands
Operating groups
Net sales (as a percent)	5.00%	4.00%	5.00%	4.00%
E-commerce
Operating groups
Net sales	$ 107,756	$ 82,402	$ 323,045	$ 261,393
Net sales (as a percent)	34.00%	33.00%	31.00%	31.00%
E-commerce | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	20.00%	22.00%	23.00%	24.00%
E-commerce | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	62.00%	58.00%	49.00%	48.00%
E-commerce | Operating Segments | Johnny Was
Operating groups
Net sales (as a percent)	41.00%		41.00%
E-commerce | Operating Segments | Emerging Brands
Operating groups
Net sales (as a percent)	40.00%	34.00%	39.00%	37.00%
Restaurant
Operating groups
Net sales	$ 23,157	$ 19,748	$ 81,333	$ 70,784
Net sales (as a percent)	7.00%	8.00%	8.00%	8.00%
Restaurant | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	13.00%	13.00%	12.00%	14.00%
Wholesale
Operating groups
Net sales	$ 69,292	$ 52,658	$ 220,707	$ 189,133
Net sales (as a percent)	22.00%	21.00%	21.00%	22.00%
Wholesale | Operating Segments | Tommy Bahama
Operating groups
Net sales (as a percent)	23.00%	18.00%	19.00%	16.00%
Wholesale | Operating Segments | Lilly Pulitzer
Operating groups
Net sales (as a percent)	11.00%	11.00%	17.00%	17.00%
Wholesale | Operating Segments | Johnny Was
Operating groups
Net sales (as a percent)	21.00%		21.00%
Wholesale | Operating Segments | Emerging Brands
Operating groups
Net sales (as a percent)	55.00%	62.00%	56.00%	59.00%
Wholesale | Operating Segments | Lanier Apparel
Operating groups
Net sales (as a percent)		100.00%		100.00%
Wholesale | Corporate and Other
Operating groups
Net sales (as a percent)	47.00%	67.00%	50.00%	65.00%
Other
Operating groups
Net sales	$ 484	$ 342	$ 1,559	$ 1,360
Other | Corporate and Other
Operating groups
Net sales (as a percent)	53.00%	33.00%	50.00%	35.00%