EXHIBIT 23.1


                     Consent of Independent Auditors


We consent to the incorporation by reference in the Registration
Statement(Form S-8) pertaining to the Casual Male Retail Group, Inc.
401(K) Salaried Savings Plan and the Casual Male Retail Group, Inc.
401(K) Hourly Savings Plan, of our report dated March 26, 2003, with
respect to the consolidated financial statements and schedule of Casual
Male Retail Group,Inc. included in its Annual Report (Form 10-K) for
the year ended February 1, 2003, filed with the Securities and Exchange
Commission.

								/s/ ERNST & YOUNG LLP


Boston, Massachusetts
January 26, 2004