Summary of Significant Accounting Policies - Additional Information (Details)	12 Months Ended
	Feb. 03, 2018 USD ($) Store Segment shares	Jan. 28, 2017 USD ($)	Jan. 30, 2016 USD ($)
Accounting Policies [Line Items]
Impairment charge	$ 4,100,000	$ 400,000	$ 0
Number of reportable segments | Segment	1
Number of operating segments | Segment	2
Amortization expense of intangibles	$ 407,000	400,000	600,000
Direct response advertising costs, amortization period	1 year
Direct response costs deferred	$ 0	0
Stock compensation expense	1,636,000	1,256,000	2,195,000
Total grant-date fair value of options vested	1,800,000	2,900,000	1,000,000
Gift card liability, net of breakage	$ 2,200,000	2,400,000
Unvested time-based restricted stock
Accounting Policies [Line Items]
Dilutive shares | shares	36,666
Deferred Stock
Accounting Policies [Line Items]
Shares excluded from computation of basic earnings per share | shares	115,457
Time-Vested Awards
Accounting Policies [Line Items]
Unrecognized stock compensation cost	$ 1,800,000
Unrecognized stock compensation cost weighted average recognition period	20 months
Selling, General and Administrative Expenses
Accounting Policies [Line Items]
Advertising expenses	$ 29,500,000	18,200,000	$ 23,600,000
Casual Male Trademark
Accounting Policies [Line Items]
Intangible assets, amortization period	2018
Accumulated amortization of intangibles	$ 7,400,000	7,000,000
Amortization expense of intangibles	307,000
Customer Lists
Accounting Policies [Line Items]
Gross carrying amount of intangibles	7,700,000	$ 7,700,000
Technology Projects
Accounting Policies [Line Items]
Impairment charge	1,900,000
Rochester Trademark
Accounting Policies [Line Items]
Indefinite-lived intangible asset, carrying value	$ 1,500,000
Minimum
Accounting Policies [Line Items]
Credit card and debit card receivables from banks settlement period	2 days
Maximum
Accounting Policies [Line Items]
Credit card and debit card receivables from banks settlement period	4 days
DXL Stores | UNITED STATES
Accounting Policies [Line Items]
Number of stores | Store	212
Casual Male XL Retail and Outlet Stores | UNITED STATES
Accounting Policies [Line Items]
Number of stores | Store	78
Casual Male XL outlets
Accounting Policies [Line Items]
Number of stores | Store	33
DXL Outlets
Accounting Policies [Line Items]
Number of stores | Store	14
Rochester Clothing Stores | UNITED STATES
Accounting Policies [Line Items]
Number of stores | Store	5
Rochester Clothing Stores | UNITED KINGDOM
Accounting Policies [Line Items]
Number of stores | Store	1
Rochester Clothing Stores | Canada
Accounting Policies [Line Items]
Number of stores | Store	2