June 3, 2010

BY EDGAR

Securities and Exchange Commission

100 F Street, N.E., Mail Stop 3561

Washington, D.C. 20549

Attn: John Reynolds

Assistant Director

Re: Casual Male Retail Group, Inc.

Form 10-K for the Fiscal Year Ended January 30, 2010

Filed March 19, 2010

File No. 01-34219

Dear Mr. Reynolds

This letter is submitted on behalf of Casual Male Retail Group, Inc. (the “Company”) in response to the comments of the staff of the Division of Corporation Finance (the “Staff”) of the Securities and Exchange Commission (the “Commission”) with respect to the Company’s Form 10-K for the fiscal year ended January 30, 2010, filed on March 19, 2010 (the “Form 10-K”), as set forth in a letter dated May 24, 2010 to David A. Levin (the “Comment Letter”).

For reference purposes, the text of the Staff comments contained in the Comment Letter has been reproduced herein with responses below each numbered comment. For your convenience, we have italicized the reproduced Staff comments from the Comment Letter and we have bolded the headings of our responses thereto.

Exhibits to Form 10-K

Comment No. 1

We note that exhibits 10.9, 10.14 and 10.15 to the Form 10-K do not have some or all of the schedules, attachments, or exhibits as listed in the exhibit itself. Please file these exhibits in their entirety with your next periodic report, as required by Item 601(b)(10) of Regulation S-K or advise.

Response to Comment No. 1

In response to the Staff’s comment, with its next Quarterly Report on Form 10-Q (for the quarter ending July 31, 2010), the Company will re-file Exhibits 10.9, 10.14 and 10.15 to its Form 10-K for the year ended January 30, 2010, each with all schedules, attachments and/or exhibits relating to such agreements, subject to any request for confidentiality where appropriate.

*         *         *         *         *

Securities and Exchange Commission

June 3, 2010

Page 2

In connection with the Company’s responses contained in this letter, the Company acknowledges that:

• The Company is responsible for the adequacy and accuracy of the disclosure in the filing;

• Staff comments or changes to disclosure in response to Staff comments do not foreclose the Commission from taking any action with respect to the filing; and

• The Company may not assert Staff comments as a defense in any proceeding initiated by the Commission or any person under the federal securities laws of the United States.

If you should have any questions about this letter, please do not hesitate to call the undersigned at (781) 828-9300.

Sincerely,

/s/ Dennis R. Hernreich

Dennis R. Hernreich

Chief Financial Officer