Intangible Assets and Goodwill - Goodwill (Details) - USD ($) $ in Thousands	3 Months Ended	9 Months Ended	12 Months Ended
	May 02, 2020	Jan. 30, 2021	Jan. 30, 2021	Feb. 01, 2020	Feb. 02, 2019
Changes in the carrying amount of goodwill
Balance at the beginning of the period	$ 66,578		$ 66,578	$ 66,621	$ 66,703
Impairment	(43,000)	$ 0	(42,745)	0	0
Other, including foreign currency			77	(43)	(82)
Balance at the end of the period		23,910	23,910	66,578	66,621
Operating Segments | Tommy Bahama
Changes in the carrying amount of goodwill
Balance at the beginning of the period	711		711	754	821
Other, including foreign currency			77	(43)	(67)
Balance at the end of the period		788	788	711	754
Operating Segments | Lilly Pulitzer
Changes in the carrying amount of goodwill
Balance at the beginning of the period	19,522		19,522	19,522	19,522
Balance at the end of the period		19,522	19,522	19,522	19,522
Operating Segments | Southern Tide
Changes in the carrying amount of goodwill
Balance at the beginning of the period	42,745		42,745	42,745	42,745
Impairment			(42,745)
Balance at the end of the period				42,745	42,745
Corporate and Other
Changes in the carrying amount of goodwill
Balance at the beginning of the period	$ 3,600		3,600	3,600	3,615
Other, including foreign currency					(15)
Balance at the end of the period		$ 3,600	$ 3,600	$ 3,600	$ 3,600