Filed Pursuant to Rule 424(b)(3)
                                                 Registration No. 333-118966

          Prospectus Supplement to Prospectus dated November 24, 2004

                             1,818,504 Shares

                    Casual Male Retail Group, Inc.

                              Common Stock

This prospectus supplement updates the prospectus dated November 24, 2004
relating to the offer for sale of up to an aggregate of 1,818,504 shares of
common stock of Casual Male Retail Group, Inc. by the selling stockholders
identified in the prospectus and this prospectus supplement, and any of their
pledgees, donees,transferees or other successors in interest.

We are providing this prospectus supplement to update the table in the
prospectus under the caption "Selling Securityholders" to reflect (i) a
transfer of warrants to purchase 20,000 shares of common stock from Prism
Partners I, L.P. to Capital Ventures International and (ii) a transfer of
warrants to purchase 20,000 shares of common stock from Prism Partners II
Offshore Fund to Capital Ventures International.  The amounts set forth below
are based upon information provided to us by the selling stockholder (or their
representatives), or on our records, and are accurate to the best of our
knowledge.  Unless we indicate otherwise, the information in this prospectus
supplement is as of September 28, 2006.


                    Number of Shares                  Common Stock Beneficially
                     of Common Stock  Number of Shares    Owned After the Sales
        Selling   Beneficially Owned   of Common Stock  -----------------------
Securityholders      Before Any Sale   Subject to Sale   Number **   Percent***
- --------------    ------------------  ----------------  -----------------------

Prism Partners I,
L.P.                            N/A                 0         N/A           N/A

Prism Partners II
Offshore Fund                   N/A                 0        N/A            N/A

Capital Ventures
International (1)           130,000           130,000          0             *
_____________________
*	Less than 1%.

**	Assumes that the selling securityholders will sell all of their shares of
common stock subject to sale pursuant to this prospectus and prospectus
supplement. We cannot assure you that the selling securityholders will sell
all or any of their shares of common stock.

***	Percentage ownership is based on 34,443,160 shares of common stock
outstanding as of September 28, 2006, plus securities deemed to be outstanding
with respect to individual stockholders pursuant to Rule 13d-3(d)(1) under the
Securities Exchange Act of 1934.

(1)	Heights Capital Management, Inc., the authorized agent of Capital Ventures
International ("CVI"), has discretionary authority to vote and dispose of the
shares held by CVI and may be deemed to be the beneficial owner of these
shares.

This prospectus supplement is not complete without the prospectus dated November
24, 2004, and we have not authorized anyone to deliver or use this prospectus
supplement without the prospectus.

             The date of this prospectus supplement is October 11, 2006.