Filed Pursuant to Rule 424(b)(3)
                                                 Registration No. 333-118966

            Prospectus Supplement to Prospectus dated November 24, 2004

                                1,818,504 Shares

                       Casual Male Retail Group, Inc.

                                 Common Stock

This prospectus supplement updates the prospectus dated November 24, 2004
relating to the offer for sale of up to an aggregate of 1,818,504 shares of
common stock of Casual Male Retail Group, Inc. by the selling stockholders
identified in the prospectus and this prospectus supplement, and any of their
pledgees, donees, transferees or other successors in interest.

We are providing this prospectus supplement to update the table in the
prospectus under the caption "Selling Securityholders" to reflect (i) a transfer
of a warrant to purchase 164,000 shares of common stock from Harbour Holdings,
Ltd. to Millennium Partners, L.P. and (ii) a transfer of a warrant to purchase
116,000 shares of common stock from Skylands Special Investment LLC (f/k/a
Strong Special Investment, L.P.) to Millennium Partners, L.P..  The amounts set
forth below are based upon information provided to us by the selling stockholder
(or their representatives), or on our records, and are accurate to the best of
our knowledge.  Unless we indicate otherwise, the information in this prospectus
supplement is as of September 18, 2006.


                    Number of Shares                  Common Stock Beneficially
                     of Common Stock  Number of Shares    Owned After the Sales
        Selling   Beneficially Owned   of Common Stock  -----------------------
Securityholders      Before Any Sale   Subject to Sale   Number **   Percent***
- --------------    ------------------  ----------------  -----------------------

Harbour Holdings,
Ltd.                          N/A                  0      N/A           N/A

Skylands Special
Investment LLC
(f/k/a Strong
Special Investment,
L.P.)                         N/A                  0      N/A           N/A

Millennium Partners,
L.P. (1)                  296,237 (2)        280,000    16,237 (2)       *
_____________________
*	Less than 1%.

**	Assumes that the selling securityholders will sell all of their shares of
common stock subject to sale pursuant to this prospectus and prospectus
supplement. We cannot assure you that the selling securityholders will sell
all or any of their shares of common stock.

***	Percentage ownership is based on 34,417,684] shares of common stock
outstanding as of September 18, 2006, plus securities deemed to be outstanding
with respect to individual stockholders pursuant to Rule 13d-3(d)(1) under the
Securities Exchange Act of 1934.

(1)	Millennium Management, L.L.C., a Delaware limited liability company, is
the managing partner of Millennium Partners, L.P., a Cayman Islands exempted
limited partnership, and consequently may be deemed to have voting control and
investment discretion over securities owned by Millennium Partners, L.P.
Israel A. Englander is the managing member of Millennium Management, L.L.C.
As a result, Mr. Englander may be deemed to be the beneficial owner of any
shares deemed to be beneficially owned by Millennium Management, L.L.C.  The
foregoing should not be construed in and of itself as an admission by either
Millennium Management, L.L.C. or Mr. Englander as to beneficial ownership of
the shares of our common stock owned by Millennium Partners, L.P.

(2)	Includes 16,237 shares of our common stock held by Millenco, L.P., an
affiliate of Millenium Partners, L.P. as of the close of business on September
18, 2006.

This prospectus supplement is not complete without the prospectus dated November
24, 2004, and we have not authorized anyone to deliver or use this prospectus
supplement without the prospectus.

          The date of this prospectus supplement is September 29, 2006.