Exhibit 99.1

CASUAL MALE RETAIL GROUP, INC.

Revises Fiscal 2011 Earnings Guidance

CANTON, MA (February 2, 2012) — Casual Male Retail Group, Inc. (NASDAQ: “CMRG”), the largest retailer of big & tall men’s apparel and accessories, today announced revised earnings guidance for the fiscal year ending January 28, 2012 (“fiscal 2011”). The Company expects earnings per share for fiscal 2011 to be $0.28 to $0.29 per diluted share compared to previous guidance of $0.35 to $0.38 per diluted share.

“We experienced a negative traffic trend during the fourth quarter as a result of the sluggish economy and the unseasonable winter weather,” said David A. Levin, President and Chief Executive Officer. “In an effort to maintain appropriate inventory levels and improve customer traffic, we increased our promotional activity and reduced pricing on our seasonal inventory during the fourth quarter, which had a negative impact on our expected gross margin rate for fiscal 2011.”

For the fourth quarter and fiscal year 2011, the Company expects that comparable sales for the Company increased 0.8% and 2.1%, respectively, as compared to the comparable prior year period. As a result, sales for fiscal 2011 will approximate $397.7 million, as compared to the Company’s previously reported sales range of $397.5 to $402.5 million. The Company expects the gross margin rate for fiscal 2011 to range from 46.1% to 46.2%, a decrease from its previous guidance of 46.9% to 47.3%. Operating expenses (SG&A) for fiscal 2011, which were previously expected to increase by approximately 3%, are expected to be slightly more favorable partly due to there being no management incentive bonuses since performance is below profitability targets.

For the fourth quarter of fiscal 2011, comparable sales from the Company’s 16 DestinationXL® (“DXL®”) stores increased 9.0% as compared to the prior year’s fourth quarter. For fiscal 2012, the Company plans to open 35-40 additional DXL stores, resulting in approximately 51-56 DXL stores operating at the end of fiscal 2012, with at least one store located in most of the major metropolitan cities across the United States. At the same time, the Company expects to close approximately 60 existing stores, most of which are in connection with opening these DXL stores, resulting in an estimated store count of approximately 425 stores at the end of fiscal 2012.

The Company plans to report its fourth quarter and fiscal 2011 financial results on March 15, 2012 when management will also conduct its quarterly conference call to discuss its results for fiscal 2011 as well as its fiscal 2012 outlook. During the conference call, the Company may discuss and answer questions concerning business and financial developments and trends. The Company’s responses to questions, as well as other matters discussed during the conference call, may contain or constitute information that has not been disclosed previously.

Casual Male Retail Group, Inc., the largest retailer of big and tall men’s apparel with operations throughout the United States, Canada and Europe, operates 420 CasualMaleXL retail and outlet

stores, 16 DestinationXL stores, 14 Rochester Clothing stores, and direct-to-consumer businesses which include several catalogs and e-commerce sites, including www.destinationxl.com. The Company is headquartered in Canton, Massachusetts, and its common stock is listed on the NASDAQ Global Market under the symbol “CMRG.”

Certain information contained in this press release, including the Company’s expectations regarding the fourth quarter and fiscal 2011, as well as the Company’s expectations regarding store openings and closing during fiscal 2012, constitutes forward-looking statements under the federal securities laws. The discussion of forward-looking information requires management of the Company to make certain estimates and assumptions regarding the Company’s strategic direction and the effect of such plans on the Company’s financial results. The Company’s actual results and the implementation of its plans and operations may differ materially from forward-looking statements made by the Company. These factors include, without limitation, the following: the risk that additional information may arise during the Company’s close process or as a result of subsequent events that would require the Company to make adjustments to the financial information; the risk that adjustments to the Company’s financial statements may be identified through the course of the Company’s independent registered public accounting firm completing its integrated audit of the Company’s financial statements and financial controls; and the risk that the Company will be unsuccessful in implementing its strategic plans with respect to store openings and closings in fiscal 2012. The Company also encourages readers of forward-looking information concerning the Company to refer to its prior filings with the Securities and Exchange Commission, including without limitation, its Annual Report on Form 10-K filed on March 18, 2011, that set forth certain risks and uncertainties that may have an impact on future results and direction of the Company.

Forward-looking statements contained in this press release speak only as of the date of this release. Subsequent events or circumstances occurring after such date may render these statements incomplete or out of date. The Company undertakes no obligation and expressly disclaims any duty to update such statements.

For additional information, contact:

Jeff Unger

Vice President of Investor Relations

Casual Male Retail Group, Inc.

561-482-9715

jeffunger@usa.net