Summary of Significant Accounting Policies - Additional Information (Details)	12 Months Ended
	Jan. 28, 2017 USD ($) Store Segment shares	Jan. 30, 2016 USD ($)	Jan. 31, 2015 USD ($)	Feb. 01, 2014 shares
Accounting Policies [Line Items]
Number of reportable segments | Segment	1
Number of operating segments | Segment	2
Amortization expense of intangibles	$ 441,000	$ 600,000	$ 1,100,000
Direct response costs deferred	$ 0	0
Shares of restricted stock outstanding | shares	856,332
Stock compensation expense	$ 1,256,000	2,195,000	2,996,000
Total grant-date fair value of options vested	2,900,000	1,000,000	1,200,000
Impairment charges	400,000		300,000
Gift card liability, net of breakage	$ 2,400,000	2,400,000
Unvested time-based restricted stock
Accounting Policies [Line Items]
Dilutive shares | shares	8,334
Unvested performance-based restricted stock
Accounting Policies [Line Items]
Securities excluded from computation of earnings per share, respective performance targets not yet achieved, amount | shares	847,998
Performance-Based Stock Options and Restricted Stock Awards
Accounting Policies [Line Items]
Securities excluded from computation of earnings per share, respective performance targets not yet achieved, amount | shares	1,059,941
Deferred Stock
Accounting Policies [Line Items]
Shares excluded from computation of basic earnings per share | shares	64,876
Outstanding shares | shares				64,876
Time-Vested Awards
Accounting Policies [Line Items]
Unrecognized stock compensation cost	$ 1,200,000
Unrecognized stock compensation cost weighted average recognition period	20 months
Selling, General and Administrative Expenses
Accounting Policies [Line Items]
Advertising expenses	$ 18,200,000	23,600,000	$ 26,000,000
Customer Lists
Accounting Policies [Line Items]
Gross carrying amount of intangibles	7,700,000	7,700,000
Casual Male Trademark
Accounting Policies [Line Items]
Accumulated amortization of intangibles	7,000,000	$ 6,500,000
Amortization expense of intangibles	341,000
Rochester Trademark
Accounting Policies [Line Items]
Indefinite-lived intangible asset, carrying value	$ 1,500,000
Minimum
Accounting Policies [Line Items]
Credit card and debit card receivables from banks settlement period	2 days
Maximum
Accounting Policies [Line Items]
Credit card and debit card receivables from banks settlement period	4 days
Direct response advertising costs, amortization period	1 year
DXL Stores | UNITED STATES
Accounting Policies [Line Items]
Number of stores | Store	192
Casual Male XL Retail and Outlet Stores
Accounting Policies [Line Items]
Intangible assets, amortization period	2018
Casual Male XL Retail and Outlet Stores | UNITED STATES
Accounting Policies [Line Items]
Number of stores | Store	97
Casual Male XL outlets
Accounting Policies [Line Items]
Number of stores | Store	36
DXL Outlets
Accounting Policies [Line Items]
Number of stores | Store	13
Rochester Clothing Stores | UNITED STATES
Accounting Policies [Line Items]
Number of stores | Store	5
Rochester Clothing Stores | UNITED KINGDOM
Accounting Policies [Line Items]
Number of stores | Store	1