UNITED STATES

SECURITIES AND EXCHANGE COMMISSION

Washington, D.C. 20549

SCHEDULE 14A

Proxy Statement Pursuant to Section 14(a) of the Securities

Exchange Act of 1934

FILED BY THE REGISTRANT  x                FILED BY A PARTY OTHER THAN THE REGISTRANT  ¨

Check the appropriate box:

¨ Preliminary Proxy Statement

¨ Confidential, for Use of the Commission Only (as permitted by Rule 14a-6(e)(2))

x Definitive Proxy Statement

¨ Definitive Additional Materials

¨ Soliciting Material under § 240.14a-12

CASUAL MALE RETAIL GROUP, INC.

(Name of Registrant as Specified In Its Charter)

(Name of Person(s) Filing Proxy Statement, if other than the Registrant)

PAYMENT OF FILING FEE (CHECK THE APPROPRIATE BOX):

x No fee required.

¨ Fee computed on table below per Exchange Act Rules 14a-6(i)(1) and 0-11.

(1) Title of each class of securities to which transaction applies:

(2) Aggregate number of securities to which transaction applies:

(3) Per unit price or other underlying value of transaction computed pursuant to Exchange Act Rule 0-11 (set forth the amount on which the filing fee is calculated and state how it was determined):

(4) Proposed maximum aggregate value of transaction:

$

(5) Total fee paid:

$

¨ Fee paid previously with preliminary materials.

¨ Check box if any part of the fee is offset as provided by Exchange Act Rule 0-11(a)(2) and identify the filing for which the offsetting fee was paid previously. Identify the previous filing by registration statement number, or the Form or Schedule and the date of its filing.

(1) Amount Previously Paid:

(2) Form, Schedule or Registration Statement No.:

(3) Filing Party:

(4) Date Filed:

CASUAL MALE RETAIL GROUP, INC.

Notice of Annual Meeting of Stockholders

to be held on August 4, 2011

Notice is hereby given that the 2011 Annual Meeting of Stockholders of Casual Male Retail Group, Inc. (the “Company”) will be held at the corporate offices of the Company, 555 Turnpike Street, Canton, Massachusetts 02021 at 9:00 A.M., local time, on Thursday, August 4, 2011 for the following purposes:

1. To elect eight directors to serve until the next Annual Meeting of Stockholders and until their respective successors have been duly elected and qualified.

2. To hold an advisory vote on named executive officer compensation.

3. To hold an advisory vote on the frequency of holding advisory votes on named executive officer compensation.

4. To ratify the appointment of Ernst & Young LLP as our independent registered public accounting firm for fiscal year ending January 28, 2012.

5. To transact such other business as may properly come before the meeting or any adjournment thereof.

These proposals are more fully described in the Proxy Statement following this Notice.

The Board of Directors recommends that you vote (i) FOR the election of all eight nominees to serve as directors of the Company, (ii) FOR the advisory vote on named executive compensation, (iii) FOR the “one year” option with respect to the advisory vote on the preferred frequency of holding advisory votes on named executive officer compensation, and (iv) FOR the ratification of the appointment of Ernst & Young LLP as our independent registered public accounting firm.

Along with the attached Proxy Statement, we are sending you copies of our Annual Report on Form 10-K for the fiscal year ended January 29, 2011.

The Board of Directors has fixed the close of business on June 13, 2011 as the record date for the determination of the stockholders entitled to notice of, and to vote at, the Annual Meeting. Accordingly, only stockholders of record at the close of business on that date will be entitled to vote at the Annual Meeting. A list of the stockholders of record as of the close of business on June 13, 2011, will be available for inspection by any of our stockholders for any purpose germane to the Annual Meeting during normal business hours at our principal executive offices, 555 Turnpike Street, Canton, Massachusetts 02021, beginning on July 25, 2011 and at the Annual Meeting.

Stockholders are cordially invited to attend the Annual Meeting in person. Regardless of whether you plan to attend the Annual Meeting, please mark, date, sign and return the enclosed proxy to ensure that your shares are represented at the Annual Meeting. Stockholders of record at the close of business on the record date whose shares are registered directly in their name, and not in the name of a broker or other nominee, may vote their shares in person at the Annual Meeting, even though they have sent in proxies.

By order of the Board of Directors,

/s/    DENNIS R. HERNREICH

DENNIS R. HERNREICH

Secretary

Canton, Massachusetts

June 21, 2011

Important Notice Regarding the Availability of Proxy Materials for the Annual Meeting of Stockholders to be Held on August 4, 2011:

The Proxy Statement and 2011 Annual Report to Stockholders are available at www.casualmale.com/proxymaterials

IMPORTANT: Please mark, date, sign and return the enclosed proxy as soon as possible. The proxy is revocable and it will not be used if you give written notice of revocation to the Secretary of the Company at 555 Turnpike Street, Canton, Massachusetts 02021, prior to the vote to be taken at the Annual Meeting, if you lodge a later-dated proxy or if you attend and vote at the Annual Meeting.

CASUAL MALE RETAIL GROUP, INC.

555 Turnpike Street

Canton, Massachusetts 02021

(781) 828-9300

Proxy Statement

Annual Meeting of Stockholders

August 4, 2011

USE OF PROXIES

This Proxy Statement and the enclosed form of proxy are being mailed to stockholders on or about June 21, 2011, in connection with the solicitation by the Board of Directors of Casual Male Retail Group, Inc. (the “Company”) of proxies to be used at the Annual Meeting of Stockholders, to be held at the Company’s corporate headquarters located at 555 Turnpike Street, Canton, Massachusetts 02021 at 9:00 A.M, local time, on Thursday, August 4, 2011 and at any and all adjournments thereof (the “Annual Meeting”). When proxies are returned properly executed, the shares represented will be voted in accordance with the stockholders’ instructions.

Stockholders are encouraged to vote on the matters to be considered. However, if no instructions have been specified by a stockholder, the shares covered by an executed proxy will be voted (i) FOR the election of all eight nominees to serve as directors of the Company, (ii) FOR the advisory vote on named executive officer compensation, (iii) FOR the “one year” option with respect to the advisory vote on the preferred frequency of holding advisory votes on named executive officer compensation, (iv) FOR the ratification of the appointment of Ernst & Young LLP as our independent registered public accounting firm, and (v) in the discretion of the proxies named in the proxy card with respect to any other matters properly brought before the Annual Meeting. Any stockholder may revoke such stockholder’s proxy at any time before it has been exercised by attending the Annual Meeting and voting in person or by filing with the Secretary of the Company either an instrument in writing revoking the proxy or another duly executed proxy bearing a later date.

Only holders of our common stock, par value $0.01 per share, of record at the close of business on June 13, 2011, the record date for the Annual Meeting, will be entitled to notice of and to vote at the Annual Meeting. On that date, there were 48,460,485 shares of common stock issued and outstanding, excluding shares held by the Company in treasury. Each share is entitled to one vote at the Annual Meeting.

A plurality of the votes properly cast for the election of directors is required to elect directors. A majority of the votes properly cast is required for all other matters. With respect to Proposal 3, if none of the three frequency options receives the vote of the holders of a majority of the votes cast, the Company will consider the frequency option (one year, two years or three years) receiving the highest number of votes cast by stockholders to be the frequency that has been recommended by stockholders. However, as described in more detail in Proposal 3, because this proposal is advisory and non-binding, the Board of Directors may decide that it is in the best interest of the stockholders and the Company to hold future advisory votes on named executive officer compensation more or less frequently. Proposal 2 is also a non-binding proposal.

No votes may be taken at the Annual Meeting, other than a vote to adjourn, unless a quorum, consisting of a majority of the shares of our common stock outstanding as of the record date, is present in person or represented by proxy at the Annual Meeting. Any stockholder who attends the Annual Meeting may not withhold such stockholder’s shares from the quorum count by declaring such shares absent from the Annual Meeting. Shares voted to abstain or to withhold as to a particular matter, or as to which a nominee (such as a broker holding shares in street name for a beneficial owner) has no voting authority in respect of a particular matter, shall be deemed present for quorum purposes. With respect to all proposals presented at the Annual Meeting, such shares will not be deemed to be voting with respect to such matter and will not count as votes for or against such matter. Votes will be tabulated by our transfer agent subject to the supervision of persons designated by the Board of Directors as inspectors.

1

Security Ownership of Certain Beneficial Owners

The following table sets forth certain information with respect to persons known to us to be the beneficial owners of more than five percent of the issued and outstanding shares of our common stock as of May 15, 2011. We were informed that, except as indicated, each person has sole voting and investment power with respect to all shares of our common stock shown as beneficially owned by such person, subject to community property laws where applicable.

Name and Address of Beneficial Owner	Number of Shares Beneficially Owned			Percent of Class(1)
Chilton Investment Company, LLC		5,577,290	(2)		11.52	%
1266 East Main Street, 7th Floor
Stamford, Connecticut 06902
Seymour Holtzman		5,653,769	(3)		11.30	%
100 N. Wilkes Barre Blvd.
Wilkes Barre, Pennsylvania 18702
FMR LLC		4,116,928	(4)		8.50	%
82 Devonshire Street
Boston, Massachusetts 02109
Royce & Associates, LLC		3,858,734	(5)		7.97	%
745 Fifth Avenue
New York, New York 10151
Glenhill Advisors, LLC		3,783,690	(6)		7.81	%
Glenn J. Krevlin
Glenhill Capital Management, LLC
156 West 56th Street, 17th Floor
New York, New York 10019
RBC Global Asset Management (U.S.) Inc.		2,998,203	(7)		6.19	%
100 South Fifth Street
Suite 2300
Minneapolis, Minnesota 55402

(1) Beneficial ownership is determined in accordance with the rules of the Securities and Exchange Commission (“SEC”) and generally includes voting or investment power with respect to securities. In computing the number of shares beneficially owned by a person and the percentage ownership of that person, shares of our common stock subject to options and warrants held by that person that are currently exercisable, or that become exercisable within 60 days, are deemed outstanding. Such shares, however, are not deemed outstanding for the purpose of computing the percentage ownership of any other person. Percentage ownership is based on 48,431,074 shares of our common stock outstanding as of May 15, 2011, plus securities deemed to be outstanding with respect to individual stockholders pursuant to Rule 13d-3(d)(1) under the Securities Exchange Act of 1934, as amended (the “Exchange Act”).

(2) Based on Amendment No. 7 to Schedule 13G, dated as of December 31, 2010, stating that Chilton Investment Company, LLC was the beneficial owner of the number of shares of common stock set forth opposite its name in the table.

(3) Represents 4,043,769 outstanding shares of common stock and 1,610,000 shares subject to options exercisable within 60 days.

(4) Based on Amendment No. 3 to Schedule 13G, dated as of December 31, 2010, stating that FMR LLC was the beneficial owner of the number of shares of common stock set forth opposite its name in the table.

(5) Based on Amendment No. 4 to Schedule 13G, dated December 31, 2010, stating that Royce & Associates, LLC was the beneficial owner of the number of shares of common stock set forth opposite its name in the table.

2

(6) Based on Amendment No. 2 to Schedule 13G, dated as of December 31, 2010, stating that Glenhill Advisors, LLC, Glenn J. Krevlin and Glenhill Capital Management, LLC were the beneficial owners of the number of shares of common stock set forth opposite their names in the table. Glenn J. Krevlin is the managing member and control person of Glenhill Advisors, LLC. Glenhill Advisors, LLC is the managing member of Glenhill Capital Management, LLC. Glenhill Capital Management, LLC is the general partner and investment advisor of Glenhill Capital LP, a security holder of the Company, managing member of Glenhill Concentrated Long Master Fund, LLC, a security holder of the Company, and sole shareholder of Glenhill Capital Overseas GP, Ltd. Glenhill Capital Overseas GP, Ltd. is general partner of Glenhill Capital Overseas Master Fund, LP, a security holder of the Company.

(7) Based on Amendment No. 1 to Schedule 13G, dated December 31, 2010, stating that RBC Global Asset Management (U.S.) Inc. was the beneficial owner of the number of shares of common stock set forth opposite its name in the table.

Security Ownership of Management

The following table sets forth certain information as of May 15, 2011, with respect to our directors, our Named Executive Officers (as defined below under “Executive Compensation”) and our directors and executive officers as a group. Except as indicated, each person has sole voting and investment power with respect to all shares of our common stock shown as beneficially owned by such person, subject to community property laws where applicable.

Name and Title	Number of Shares Beneficially Owned			Percent of Class(1)
Seymour Holtzman		5,653,769	(2)		11.30	%
Chairman of the Board and Director
David A. Levin		1,660,555	(3)		3.4	%
Chief Executive Officer, President and Director
Dennis R. Hernreich		704,520	(4)		1.5	%
Chief Financial Officer, Executive Vice President,
Chief Operating Officer, Treasurer and Secretary
Robert S. Molloy		95,836	(5)		*
Senior Vice President and General Counsel
Doug Hearn		13,332	(6)		*
Former Senior Vice President, General Merchandise Manager,
Casual Male XL and Global Sourcing
Henry J. Metscher		38,833	(7)		*
Former Senior Vice President, General Merchandise Manager,
Direct Businesses and President of Footwear
Alan S. Bernikow, Director		92,629	(8)		*
Jesse H. Choper, Director		166,258	(9)		*
John E. Kyees, Director		11,677	(10)		*
Ward K. Mooney, Director		40,273	(11)		*
George T. Porter, Jr., Director		137,224	(12)		*
Mitchell S. Presser, Director		74,924	(11)		*
Directors and Executive Officers as a group (22 persons)		9,229,976	(13)		18.0	%

* Less than 1%

(1) Beneficial ownership is determined in accordance with the rules of the SEC and generally includes voting or investment power with respect to securities. In computing the number of shares beneficially owned by a person and the percentage ownership of that person, shares of our common stock subject to options and warrants held by that person that are currently exercisable, or that become exercisable within 60 days, are

3

deemed outstanding. Such shares, however, are not deemed outstanding for the purpose of computing the percentage ownership of any other person. Percentage ownership is based on 48,431,074 shares of our common stock outstanding as of May 15, 2011, plus securities deemed to be outstanding with respect to individual stockholders pursuant to Rule 13d-3(d)(1) under the Exchange Act.

(2) Mr. Holtzman may be deemed to have shared voting and investment power over 5,653,769 shares of common stock, which includes 1,610,000 shares subject to stock options exercisable within 60 days.

(3) Includes 636,324 shares subject to stock options exercisable within 60 days, 204,232 shares of unvested restricted stock and 3,539 shares held pursuant to his 401(k) Plan account.

(4) Includes 250,763 shares subject to stock options exercisable within 60 days, 234,867 shares of unvested restricted stock and 10,614 shares held pursuant to his 401(k) Plan account.

(5) Includes 30,606 shares subject to stock options exercisable within 60 days and 44,897 shares of unvested restricted stock.

(6) Includes 6,667 shares subject to stock options exercisable within 60 days.

(7) Includes 10,000 shares subject to stock options exercisable within 60 days.

(8) Includes 70,000 shares subject to stock options exercisable within 60 days.

(9) Includes 85,000 shares subject to stock options exercisable within 60 days.

(10) Includes 11,667 shares subject to stock options exercisable within 60 days.

(11) Includes 25,000 shares subject to stock options exercisable within 60 days.

(12) Includes 55,000 shares subject to stock options exercisable within 60 days.

(13) Includes 2,921,037 shares subject to stock options exercisable within 60 days, 768,437 of unvested shares of restricted stock and 19,525 shares held pursuant to respective 401(k) Plan accounts.

4

PROPOSAL 1

ELECTION OF DIRECTORS

The Board of Directors, in accordance with our By-Laws, as amended (the “By-Laws”), has set the number of members of our Board of Directors at nine. At the Annual Meeting, eight nominees are to be elected to serve on the Board until the 2012 Annual Meeting of Stockholders and until their respective successors have been duly elected and qualified (leaving one vacancy). All of the nominees currently serve as members of our Board of Directors. Unless a proxy shall specify that it is not to be voted for a nominee, it is intended that the shares represented by each duly executed and returned proxy will be voted in favor of the election as directors of Seymour Holtzman, David A. Levin, Alan S. Bernikow, Jesse Choper, John E. Kyees, Ward K. Mooney, George T. Porter Jr. and Mitchell S. Presser. Although the Board of Directors does not intend to nominate an individual to fill the vacancy at this time, in accordance with our By-Laws, a new director could be named at any time. Although management expects all nominees to accept nomination and to serve if elected, proxies will be voted for a substitute if a nominee is unable or unwilling to accept nomination or election. Cumulative voting is not permitted.

The Board of Directors recommends that you vote “FOR”

the election of the eight individuals named below as directors of our Company.

Set forth below is certain information regarding our directors, including information furnished by them as to their principal occupations and business experience for the past five years, certain directorships held by each director within the past five years, their respective ages as of May 15, 2011 and the year in which each became a director of our Company:

NAME	AGE		POSITION	DIRECTOR SINCE
Seymour Holtzman		75	Chairman of the Board and Director		2000
David A. Levin		60	President, Chief Executive Officer and Director		2000
Alan S. Bernikow		70	Director(1),(2)		2003
Jesse Choper		75	Director(1),(2),(3)		1999
John Edward Kyees		64	Director(1),(4)		2010
Ward K. Mooney		62	Director(3)		2006
George T. Porter, Jr.		64	Director(3),(5)		1999
Mitchell S. Presser		46	Director(2)		2007

(1) Current member of the Audit Committee.

(2) Current member of the Nominating and Corporate Governance Committee.

(3) Current member of the Compensation Committee.

(4) Mr. Kyees was appointed a director of the Company on May 3, 2010. Mr. Kyees was also appointed a member of the Audit Committee effective May 3, 2010.

(5) Mr. Porter was a member of the Audit Committee until his seat was filled by Mr. Kyees effective May 3, 2010.

Seymour Holtzman, has been a director since April 7, 2000 and Chairman of the Board since April 11, 2000. On May 25, 2001, the Board of Directors hired Mr. Holtzman as an employee. Mr. Holtzman has been involved in the retail business for over 35 years. For many years, he has been the President and Chief Executive Officer of Jewelcor, Incorporated, a former New York Stock Exchange listed company that operated a chain of retail stores. From 1986 to 1988, Mr. Holtzman was Chairman of the Board and Chief Executive Officer of Gruen Marketing Corporation, an American Stock Exchange listed company involved in the nationwide distribution of watches. For at least the last five years, Mr. Holtzman has served as Chairman and Chief Executive Officer of Jewelcor Management, Inc., a company primarily involved in investment and management services. Mr. Holtzman is owner and Chief Executive Officer of each of C.D. Peacock, Inc., a Chicago, Illinois retail jewelry establishment,

5

and Homeclick, LLC, a privately held internet retailer specializing in luxury brands for the home. Mr. Holtzman was the Chief Executive Officer and Co-Chairman of the Board of George Foreman Enterprises, Inc. (OTCBB: “GFME.PK”), formerly MM Companies, Inc., until his resignation in November 2010. Mr. Holtzman is a successful entrepreneur with extensive experience working with public companies and provides valuable insight to the Board with respect to strategic planning.

David A. Levin has been our President and Chief Executive Officer since April 10, 2000 and a director since April 11, 2000. From 1999 to 2000, he served as the Executive Vice President of eOutlet.com. Mr. Levin was President of Camp Coleman, a division of The Coleman Company, from 1998 to 1999. Prior to that, Mr. Levin was President of Parade of Shoes, a division of J. Baker, Inc., from 1995 to 1997. Mr. Levin was also President of Prestige Fragrance & Cosmetics, a division of Revlon, Inc., from 1991 to 1995. Mr. Levin has worked in the retail industry for over 30 years. Since joining the Company, Mr. Levin has been instrumental in transforming the Company from a company which exclusively operated Levi Strauss & Co. branded apparel to the largest specialty retailer of big & tall men’s apparel. In addition, Mr. Levin brings to the Board valuable experience in merchandising and marketing initiatives.

Alan S. Bernikow has been a director since June 29, 2003. From 1998 until his retirement in May 2003, Mr. Bernikow was the Deputy Chief Executive Officer at Deloitte & Touche LLP where he was responsible for assisting the firm on special projects such as firm mergers and acquisitions, litigation matters and partner affairs. He was a member of Deloitte & Touche’s Executive & Management Committees; Chairman for the Professional Asset Indemnity Limited (“PAIL” Bermuda) Big 4 Insurance Representatives; and President for the PAIL Vermont Insurance Company Big 4 Insurance Representatives. Mr. Bernikow joined Touche Ross, the predecessor firm of Deloitte & Touche LLP, in 1977, prior to which Mr. Bernikow was the National Administrative Partner in Charge for the accounting firm of J.K. Lasser & Company. Mr. Bernikow is a member of the Board of Directors of Revlon, Inc. and Revlon Consumer Products Corporation and serves as Chairman of the Audit Committee of Revlon, Inc.; as well as Chairman of the Revlon, Inc. Compensation and Stock Plan Committee; a member of the Board of Directors of Mack-Cali, as well as the Chairman of the Audit Committee of Mack-Cali; a member of the Board of Premier American Bank, as well a member of the Compensation Committee and Chairman of the Audit Committee of Premier American Bank; and serves as a Director of the Board of the UBS Global Asset Management (US) Inc.—a wholly owned subsidiary of UBS AG, including serving as Chairman of its Audit Committee. Mr. Bernikow has had extensive international experience in his role in Deloitte & Touche’s management/risk management group, as well as worldwide insurance responsibilities. Mr. Bernikow provides the Board with substantial financial expertise and strategic planning as a result of his years of experience at Deloitte & Touche LLP. His strong financial background qualifies him as an “audit committee financial expert”. Mr. Bernikow provides the Board with valuable insight with respect to financial reporting based on his experiences serving on the audit committees of several boards.

Jesse Choper has been a director since October 8, 1999. Mr. Choper is the Earl Warren Professor of Public Law at the University of California at Berkeley School of Law, where he has taught since 1965. From 1960 to 1961, Professor Choper was a law clerk for Supreme Court Chief Justice Earl Warren. Mr. Choper is a member of the California Horseracing Board. Mr. Choper was a member of the Board of Directors of George Foreman Enterprises, Inc. (OTCBB: “GFME.PK”) until his resignation in November 2010. Mr. Choper provides valuable legal expertise to the Board. His specific legal background makes him an authority on ethical behavior and he provides valuable insight with respect to corporate governance. Mr. Choper’s tenure and service as a director for over ten years is also considered a valuable asset to the Board.

John E. Kyees has been a director since May 3, 2010. From 2003 until his retirement in 2010, Mr. Kyees was the Chief Financial Officer of Urban Outfitters, Inc. Mr. Kyees continues to serve as the Chief of Investor Relations for Urban Outfitters. Prior to that, from 2002 to 2003, Mr. Kyees was the Chief Financial Officer and Chief Administrative Officer of bebe Stores, Inc. Mr. Kyees is a member of the Board of Directors of Vera Bradley and serves as Chairman of the Audit Committee and Compensation Committee and is a member of the Nominating and Corporate Governance Committee. Mr. Kyees brings to the Board extensive executive level retail experience

6

having served as Chief Financial Officer for several prominent retailers. His insight with respect to merchandising, operational activities and finance is an asset to our Board. Institutional Investor magazine selected Mr. Kyees as a top specialty retail chief financial officer on five separate occasions, evidencing his strong skills in corporate finance, strategic and accounting matters.

Ward K. Mooney has been a director since July 31, 2006. In 2005, Mr. Mooney was a founding partner of Crystal Financial LLC and since March 2010 has been the Chief Executive Officer. Prior to that, Mr. Mooney was the President of Bank of America Retail Finance Group and Chief Operating Officer of Back Bay Capital, both of which were formerly Bank of Boston businesses which Mr. Mooney founded. Mr. Mooney provides the Board with valuable insight with respect to his extensive experience as a lender in the apparel industry.

George T. Porter, Jr. has been a director since October 28, 1999. Mr. Porter was President of Levi’s USA for Levi Strauss & Co. from 1994 to 1997. Beginning in 1974, Mr. Porter held various positions at Levi Strauss & Co., including President of Levi’s Men’s Jeans Division. Mr. Porter was also Corporate Vice President and General Manager of Nike USA from 1997 to 1998. Mr. Porter provides the Board with extensive merchandising experience having worked at two highly prominent companies. Mr. Porter’s tenure and service as a director for over ten years is also considered a valuable asset to the Board.

Mitchell S. Presser has been a director since May 1, 2007. Since November 2006, Mr. Presser has been a founding partner of Paine & Partners, LLC, a private equity firm. Prior to that, Mr. Presser was a law partner with Wachtell, Lipton, Rosen & Katz, specializing in mergers & acquisitions. Mr. Presser serves as a director on the boards of three privately-held companies. Mr. Presser’s extensive experience in private equity and strategic planning provides valuable insight to the Board.

All directors hold office until the next Annual Meeting of Stockholders and until their respective successors have been duly elected and qualified or until their earlier death, resignation or removal.

Non-Director Executive Officers

Dennis R. Hernreich, 54, has been our Executive Vice President, Chief Operating Officer, Chief Financial Officer, Treasurer and Secretary since September 2002. Prior to that, Mr. Hernreich served as our Senior Vice President, Chief Financial Officer and Treasurer upon joining us on September 5, 2000. Prior to joining our Company, from 1996 through 1999, Mr. Hernreich held the position of Senior Vice President and Chief Financial Officer of Loehmann’s, a national retailer of women’s apparel. From 1999 to August 2000, Mr. Hernreich was Senior Vice President and Chief Financial Officer of Pennsylvania Fashions, Inc., a 275-store retail outlet chain operating under the name Rue 21.

Francis Chane, 48, has been our Senior Vice President of Distribution & Logistics since June 2011. Prior to that, Mr. Chane was our Vice President of Distribution & Logistics since joining the Company in June 2008. Prior to joining our Company, Mr. Chane was Vice President Operations & Facilities for Redcats USA, a division of the French multi-national company PPR, from 1999 to April 2008.

Angela Chew, 44, has been our Vice President of Global Sourcing since May 2010. From October 2009 through May 2010, Ms. Chew was our Senior Director of Global Sourcing. Prior to that, Ms. Chew was our Director of Global Sourcing from February 2009 to October 2009. Prior to joining our Company, from October 2007 to December 2008 Ms. Chew was the Senior Product Merchant for Redcats USA. From 2007 to 2009, Ms. Chew was an Independent Retail Consultant and Analyst with the Gerson Lehrman Group and, from August 2006 to December 2006, the Executive Vice President of Global Sourcing for Rocawear.

Richard Della Bernarda, 49, has been our Senior Vice President and Chief Marketing Officer since June 2007. Mr. Della Bernarda began his career with Casual Male Corp. in 1992 and joined our Company in May 2002 as part of our acquisition of Casual Male Corp. as our Senior Vice President of Marketing.

7

Kenneth M. Ederle, 46, has been our Senior Vice President—General Merchandise Manager DXL since May 2011. Prior to that, Mr. Ederle was our Vice President, General Merchandise Manager of Rochester Clothing from August 2008 until May 2011. From January 2008 to August 2008, Mr. Ederle was our Merchandise Manager of Sportswear for Rochester Clothing and prior to that was one of our Merchandise Managers for Casual Male from November 2006 to December 2007. Prior to joining the Company in 2006, Mr. Ederle was a Senior Buyer and Senior Planner for Limited Brands.

Jack R. McKinney, 55, has been our Senior Vice President and Chief Information Officer since June 2002. Mr. McKinney began his career with Casual Male Corp. in 1997 and joined our Company in May 2002 as part of our acquisition of Casual Male Corp.

Robert S. Molloy, 51, has been our Senior Vice President and General Counsel since April 2010. Prior to that, Mr. Molloy was the Vice President and General Counsel since joining the Company in February 2008. Prior to joining the Company, Mr. Molloy served as Vice President, Assistant General Counsel at Staples, Inc. from May 1999 to February 2008. Prior to May 1999, Mr. Molloy served as a trial attorney.

Francie Nguyen, 46, joined our Company in May 2011 as Senior Vice President—CMRG Direct Business. Prior to that, Ms. Nguyen was the General Manager, President of Women’s, Men’s and Kids Apparel, Footwear and Home at Hanover Direct from May 2008 until May 2011. From October 2005 to May 2008, Ms. Nguyen was the Vice President of Merchandising, Women’s Fashion and Home at Spiegel.

Brian Reaves, 50, joined our Company in May 2010 as our Senior Vice President of Store Sales and Operations. Prior to joining our Company, Mr. Reaves was the Vice President—Outreach and Group Sales for David’s Bridal from 2007 to 2009. Before that, Mr. Reaves was the Senior Vice President of Sales for The Bridal Group from 2004 to 2007.

Peter E. Schmitz, 52, joined the Company in August 2007 as our Senior Vice President, Real Estate and Store Development. Prior to that, Mr. Schmitz was the Vice President of Real Estate for Brooks/Eckerd Pharmacy Chain since 1995.

Vickie S. Smith, 54, joined the Company in February 2008 as our Senior Vice President, Planning and Allocation. Prior to that, Ms. Smith worked at Urban Brands as Senior Vice President, Planning, Allocation and Marketing from May 2006 to November 2007. From May 2001 to December 2005, Ms. Smith was the Vice President, Corporate Planning and Allocation at JC Penney.

Peter H. Stratton, Jr., 39, has been our Senior Vice President of Finance, Corporate Controller and Chief Accounting Officer since August 2009. Mr. Stratton joined us in June 2009 as Vice President of Finance. From May 2007 to June 2009, he served as Senior Director of Corporate Accounting at BearingPoint, Inc. Prior to May 2007, Mr. Stratton held various finance and accounting leadership positions at Legal Sea Foods, Inc., Shaw’s Supermarkets, Inc. and Cintas Corporation.

Walter E. Sprague, 62, has been our Senior Vice President of Human Resources since May 2006. From August 2003 through April 2006, Mr. Sprague was our Vice President of Human Resources. Prior to joining our Company, Mr. Sprague was the Managing Director Northeast, for Marc-Allen Associates, a nationwide executive recruiting firm. From 1996 to 2002, Mr. Sprague was the Assistant Vice President—Senior Director of Human Resources for Foot Locker Inc. and prior to that, the Assistant Vice President—Senior Director of Human Resources for Woolworth Corporation, the predecessor company to Foot Locker Inc.

John R. Wagner, 56, has been our Vice President, Merchandise Manager for Tailored Clothing and Furnishings since November 2010. In April 2011, Mr. Wagner assumed the oversight responsibilities for Shoes XL and Accessories for Rochester Clothing and Casual Male XL. Prior to joining our company, Mr. Wagner was President of Innovative Sourcing Group, a New York based product development and sourcing firm specializing

8

in men’s apparel. From 2001 to 2007, Mr. Wagner held the positions of Vice President of Manufacturing and Product Development and Vice President of Tailored Clothing for S&K Famous Brands, a men’s specialty retail chain based in Richmond, Virginia.

There are no family relationships between any of our directors and executive officers.

CORPORATE GOVERNANCE

Board of Directors

Our Board of Directors is currently comprised of eight members (there is currently one vacancy which will not be filled at the Annual Meeting). A majority of the members of the Board are “independent” under the rules of the Nasdaq Global Select Market (“Nasdaq”). The Board has determined that the following directors are independent: Messrs. Bernikow, Choper, Kyees, Mooney, Porter and Presser.

Our Board of Directors met six times during our fiscal year ended January 29, 2011 (“fiscal 2010”). Each of the directors attended at least 75 percent of the aggregate of the total meetings of the Board and the total meetings of the committees of the Board on which each director served.

We believe that it is important for and we encourage the members of our Board of Directors to attend Annual Meetings of Stockholders. Last year, all members of the Board of Directors attended the Annual Meeting of Stockholders held on August 5, 2010.

Corporate Governance Highlights

We comply with the corporate governance requirements imposed by the Sarbanes-Oxley Act of 2002, the SEC and Nasdaq. To assist the Board in fulfilling its responsibilities, it has adopted certain Corporate Governance Guidelines (the “Governance Guidelines”). Many features of our corporate governance principles are discussed in other sections of this proxy statement, but some of the highlights are:

• Published Governance Guidelines. A copy of the Governance Guidelines can be found under “Corporate Governance” on the Investor Relations page of our website at www.casualmaleXL.com.

• Annual Election of Directors. Our directors are elected annually for a term of office to expire at the next Annual Meeting (subject to the election and qualification of their successors).

• Independent Board. The vast majority of our Board is comprised of independent directors, with the exception only of our Chairman of the Board and our Chief Executive Officer.

• Independent Board Committees. All members of our Audit, Compensation, and Nominating and Corporate Governance Committees are independent directors, and none of such members receives compensation from us other than for service on our Board of Directors or its committees.

• Independent Executive Sessions. As contemplated by the Governance Guidelines, the Board of Directors is required to have at least semiannual executive sessions where independent directors meet without the Chairman (who does not qualify as “independent” under Nasdaq rules) and management. For fiscal 2010, the Board of Directors held two independent executive sessions. In addition, periodically throughout the year, the full Board of Directors, including the Chairman, may meet without management participation.

• Committee Authority to Retain Independent Advisors. Each of the Audit, Compensation and Nominating and Corporate Governance Committees has the authority to retain independent advisors, with all fees and expenses to be paid by the Company.

9

• Audit Committee Policies and Procedures. Under its charter, the Audit Committee’s prior approval is required for all audit services and permitted non-audit services (other than de minimis permitted non-audit services as defined and permitted by the Sarbanes-Oxley Act of 2002) to be provided by our independent registered public accounting firm.

• Audit Committee Financial Expert. Our Board has determined that Mr. Bernikow is an “audit committee financial expert” under the rules of the SEC and is independent as defined by Nasdaq listing standards.

• Stock Ownership Guidelines. While we do not have any specific guidelines for stock ownership, we do encourage our directors and members of senior management to invest in the Company. Through our Non-Employee Director Compensation Plan, which is described below under “Director Compensation,” each director may elect to receive any or all of his compensation in shares of our common stock or in options to purchase shares of our common stock. With respect to our senior management team, the Compensation Committee requires that each participant in our long-term incentive plan irrevocably elect to receive at least 50% of any award earned in the form of equity. A participant may choose shares of restricted stock or stock options, or any combination thereof.

• Repricings Prohibited. Our 2006 Incentive Compensation Plan, as amended, prohibits the repricing of stock options or stock appreciation rights, or any action that may be treated as a repricing, without shareholder approval.

• No Stockholder Rights Plan. We do not currently have a stockholder rights plan in effect and are not considering adopting one.

• Clawback Policy. Our employment agreements with our members of senior management contain a clawback provision which provides for remedies in the event we learn after the senior executive is terminated by us other than for “justifiable cause” that the senior executive could have been terminated for “justifiable cause.” Although the final rules have not yet been promulgated, the Dodd-Frank Act will also require that we implement a policy providing for the recovery of erroneously paid incentive-based compensation following a required accounting restatement. Once the final rules are issued by the SEC, we will revise, in a timely manner, the clawback provision of our employment agreements.

Committees of the Board

Our Board of Directors has three standing committees: the Nominating and Corporate Governance Committee, the Audit Committee and the Compensation Committee. The Nominating and Corporate Governance Committee, the Audit Committee and the Compensation Committee are each comprised of directors who are “independent” under the rules of Nasdaq.

Nominating and Corporate Governance Committee

The Nominating and Corporate Governance Committee has a written charter, which can be found under “Corporate Governance” on the Investor Relations page of our website at www.casualmaleXL.com. The Corporate Governance Guidelines are also posted on the Investor Relations page of our website. The Committee was established to perform functions related to governance of our Company, including, but not limited to, planning for the succession of our CEO and such other officers as the Committee shall determine from time to time, recommending to the Board of Directors individuals to stand for election as directors, overseeing and recommending the selection and composition of committees of the Board of Directors, and developing and recommending to the Board of Directors a set of corporate governance principles applicable to our Company. The present members of the Nominating and Corporate Governance Committee are Messrs. Choper, Bernikow and Presser, each of whom is “independent” under the rules of Nasdaq. Although Mr. Holtzman is not a member of the Nominating and Corporate Governance Committee and is not entitled to vote or receive compensation for his participation, he routinely attends all meetings of the Nominating and Corporate Governance Committee. The Nominating and Corporate Governance Committee met ten times during fiscal 2010.

The Board’s current policy with regard to the consideration of director candidates recommended by stockholders is that the Nominating and Corporate Governance Committee will review and consider any director candidates

10

who have been recommended by stockholders in compliance with the procedures established from time to time by the Committee (the current procedures are described below), and conduct inquiries it deems appropriate. The Nominating and Corporate Governance Committee will consider for nomination any such proposed director candidate who is deemed qualified by the Nominating and Corporate Governance Committee in light of the minimum qualifications and other criteria for Board membership approved by the Committee from time to time.

While the Nominating and Corporate Governance Committee does not have a formal diversity policy for Board membership and identifies qualified candidates without regard to race, color, disability, gender, national origin, religion or creed, it does seek to ensure the fair representation of all stockholder interests on the Board. In that regard, in considering candidates for the Board, the Nominating and Corporate Governance Committee considers, among other factors, diversity with respect to viewpoint, skills and experience. The Board believes that the use of these general criteria, along with the minimum qualifications listed below, will result in nominees who represent a mix of backgrounds and experiences that will enhance the quality of the Board.

At a minimum, the Nominating and Corporate Governance Committee must be satisfied that each nominee, both those recommended by the Committee and those recommended by stockholders, meets the following minimum qualifications:

• The nominee should have a reputation for integrity, honesty and adherence to high ethical standards.

• The nominee should have demonstrated business acumen, experience and ability to exercise sound judgments in matters that relate to our current and long-term objectives and should be willing and able to contribute positively to our decision-making process.

• The nominee should have a commitment to understand our Company and our industry and to regularly attend and participate in meetings of the Board and its committees.

• The nominee should have the interest and ability to understand the sometimes conflicting interests of the various constituencies of ours, which includes stockholders, employees, customers, governmental units, creditors and the general public, and to act in the interests of all of our stakeholders.

• The nominee should not have, nor appear to have, a conflict of interest that would impair the nominee’s ability to represent the interests of all of our stockholders and to fulfill the responsibilities of a director.

The current procedures to be followed by stockholders in submitting recommendations for director candidates are as follows:

1. All stockholder recommendations for director candidates must be submitted to the Secretary at our corporate offices located at 555 Turnpike Street, Canton, Massachusetts, 02021 who will forward all recommendations to the Nominating and Corporate Governance Committee.

2. All stockholder recommendations for director candidates must be submitted to us not less than 120 calendar days prior to the date on which our proxy statement was released to stockholders in connection with our previous year’s annual meeting.

3. All stockholder recommendations for director candidates must include the following information:

a. The name and address of record of the stockholder.

b. A representation that the stockholder is a record holder of our securities, or if the stockholder is not a record holder, evidence of ownership in accordance with Rule 14a-8(b)(2) of the Exchange Act.

c. The name, age, business and residential address, educational background, current principal occupation or employment, and principal occupation or employment for the preceding five (5) full fiscal years of the proposed director candidate.

d. A description of the qualifications and background of the proposed director candidate which addresses the minimum qualifications and other criteria for Board membership approved by the Board from time to time.

11

e. A description of all arrangements or understandings between the stockholder and the proposed director candidate.

f. The consent of the proposed director candidate (i) to be named in the proxy statement relating to our annual meeting of stockholders and (ii) to serve as a director if elected at such annual meeting.

g. Any other information regarding the proposed director candidate that is required to be included in a proxy statement filed pursuant to the rules of the SEC.

Except where we are legally required by contract or otherwise to provide third parties with the ability to nominate directors, the Nominating and Corporate Governance Committee is responsible for identifying and evaluating individuals, including nominees recommended by stockholders, believed to be qualified to become Board members and recommending to the Board the persons to be nominated by the Board for election as directors at any annual or special meeting of stockholders and the persons to be elected by the Board to fill any vacancies on the Board. The Committee may solicit recommendations from any or all of the following sources: non-management directors, the CEO, other executive officers, third-party search firms or any other source it deems appropriate. The Committee will review and evaluate the qualifications of any such proposed director candidate, and conduct inquiries it deems appropriate. The Committee will evaluate all such proposed director candidates in the same manner, with no regard to the source of the initial recommendation of such proposed director candidate. Accordingly, there are no differences in the manner in which the Nominating and Corporate Governance evaluates director nominees recommended by stockholders. In identifying and evaluating candidates for membership on the Board of Directors, the Committee will take into account all factors it considers appropriate, which may include strength of character, mature judgment, career specialization, relevant technical skills, diversity, and the extent to which the candidate would fill a present need on the Board of Directors.

Audit Committee

We have a separately-designated standing Audit Committee established in accordance with section 3(a)(58)(A) of the Exchange Act. The Audit Committee is currently comprised of Messrs. Choper, Bernikow and Kyees. Mr. Kyees replaced Mr. Porter as a member of the Audit Committee on May 3, 2010. Each of the members of the Audit Committee is independent, as independence for Audit Committee members is defined under the rules of Nasdaq. In addition, the Board of Directors has determined that Mr. Bernikow is an “audit committee financial expert” under the rules of the SEC. No other directors attend audit committee meetings.

The Audit Committee operates under a written charter, which can be found under “Corporate Governance” on the Investor Relations page of our website at www.casualmaleXL.com.

The purpose of the Audit Committee is to (i) assist the Board of Directors in fulfilling its oversight responsibilities to the shareholders, potential shareholders and the investment community; (ii) oversee the audits of our financial statements and our relationship with our independent registered public accounting firm; (iii) promote and further the integrity of our financial statements and oversee the qualifications, independence and performance of our independent registered public accounting firm (including being solely responsible for appointing, determining the scope of, evaluating and, when necessary, terminating the relationship with the independent registered public accounting firm); and (iv) provide the Board of Directors and the independent registered public accounting firm, unfiltered access to each other on a regular basis. The Audit Committee meets at least quarterly and as often as it deems necessary in order to perform its responsibilities. During fiscal 2010, the Audit Committee met six times.

For additional information regarding the Audit Committee, see the “Report of the Audit Committee” included elsewhere in this Proxy Statement.

12

Compensation Committee

The primary purpose of the Compensation Committee is to discharge the Board’s responsibilities relating to executive compensation. The Compensation Committee also reviews and makes recommendations to the full Board, or independently approves, regarding all stock-based compensation awards to our executive officers under our equity incentive plans. The Compensation Committee met eleven times during fiscal 2010. The present members of the Compensation Committee are Messrs. Choper, Mooney and Porter, each of whom is “independent” under the rules of the Nasdaq. Although Mr. Holtzman is not a member of the Compensation Committee and is not entitled to vote or receive compensation for his participation, he routinely attends all meetings of the Compensation Committee, except those portions of meetings during which the Compensation Committee discusses the consulting agreement between the Company and Jewelcor Management, Inc. (“JMI”).

The Compensation Committee operates under a written charter, which can be found under “Corporate Governance” on the Investor Relations page of our website at www.casualmaleXL.com.

The Compensation Discussion and Analysis recommended by the Compensation Committee to be included in the Proxy Statement begins on page 17 of this Proxy Statement. Among other things, the Compensation Discussion and Analysis describes in greater detail the Compensation Committee’s role in the executive compensation process.

Board Leadership Structure

We currently separate the positions of Chief Executive Officer and Chairman of the Board of Directors. Since 2002, Mr. Holtzman has served as our Chairman. The responsibilities of the Chairman of the Board of Directors include setting the agenda for each meeting of the Board of Directors, in consultation with management; attending board of director sessions of the non-management directors; and facilitating communication with the Board of Directors, executive officers and stockholders.

Separating the positions of Chief Executive Officer and Chairman of the Board of Directors allows our Chief Executive Officer to focus on our day-to-day business, while allowing the Chairman of the Board of Directors to lead the Board of Directors in its fundamental role of providing independent advice to and oversight of management. Although Mr. Holtzman is not an independent director, the Board of Directors believes that having Mr. Holtzman serve as Chairman of the Board of Directors is the appropriate leadership structure for the Company, given his wealth of retail experience, his extensive knowledge of the Company and his history of innovative and strategic thinking.

In addition, as described above, our Board of Directors has three standing committees, each chairman and each member of which is an independent director. Our Board of Directors delegates substantial responsibility to each committee of the Board of Directors, which reports their activities and actions back to the full Board of Directors. We believe that the independent committees of our Board of Directors and their chairpersons are an important aspect of the leadership structure of our Board of Directors.

Risk Oversight

Our Board of Directors, as a whole and through its committees, has responsibility for the oversight of risk management. With the oversight of our full Board of Directors, our executive officers are responsible for the day-to-day management of the material risks we face. In its oversight role, our Board of Directors has the responsibility to satisfy itself that the risk management processes designed and implemented by our executive officers are adequate and functioning as designed. The involvement of the full Board of Directors in setting our business strategy is a key part of its oversight of risk management and in determining what constitutes an appropriate level of risk for us. The full Board of Directors receives updates from our executive officers and outside advisors regarding certain risks our Company faces, including various operating risks and corporate governance best practices.

13

In fiscal 2009, our Board of Directors engaged Ernst & Young LLP to assist our senior management team in completing a risk assessment. With Ernst & Young LLP as the facilitator, our senior management team identified, assessed and prioritized various risks throughout the Company. Management has identified compensating controls or recommendations for minimizing risk and meets periodically to review such controls and identify new potential risks. In the first quarter of fiscal 2011, Ernst & Young LLP met with senior management to re-assess and examine those risks and to review the current controls. The results of the risk reassessment were presented to the full Board of Directors.

In addition, our Board committees each oversee certain aspects of risk management. For example, our Audit Committee is responsible for overseeing the management of risks associated with the Company’s financial reporting, accounting and auditing matters; our Compensation Committee oversees risks associated with our compensation policies and programs; and our Nominating and Corporate Governance Committee oversees the management of risks associated with director independence, conflicts of interest, composition and organization of our Board of Directors, and director succession planning. Our Board committees report their findings to the full Board of Directors.

Our Chief Executive Officer and Chief Financial Officer attend all meetings of the Board of Directors, except executive sessions, and are available to address any questions or concerns raised by the Board of Directors on risk management-related and any other matters.

Director Compensation

The Compensation Committee is responsible for reviewing and making recommendations to our Board of Directors with respect to the compensation paid to our non-employee directors.

In January 2010, the Company established a Non-Employee Director Stock Purchase Plan to provide a convenient method for its non-employee directors to acquire shares of the Company’s common stock at fair market value by voluntarily electing to receive shares of common stock in lieu of cash for service as a director. The substance of this plan is now encompassed within the Casual Male Retail Group, Inc. Amended and Restated Non-Employee Director Compensation Plan (the “Non-Employee Director Compensation Plan”). As of January 29, 2011, there were 500,000 shares available for issuance under this plan for the sole purpose of satisfying elections to receive shares of common stock in lieu of cash for service as a director. The Non-Employee Director Compensation Plan is a stand-alone plan and is not a sub-plan under our 2006 Incentive Compensation Plan, as amended (the “2006 Plan”). Accordingly, shares issued under this plan do not reduce the shares available for issuance under the 2006 Plan.

In fiscal 2008, the Compensation Committee retained Sibson Consulting to review the compensation of directors at our peer companies. The following peer companies were used for this review:

•    Bebe	•    Citi Trends
•    The Buckle	•    Mothers Work
•    Cache	•    Hibbett
•    Cato Group	•    Hot Topic
•    Charlotte Russe	•    JoS A. Bank
•    Christopher Banks	•    The Wet Seal

The Compensation Committee has the authority to retain compensation consultants and other outside advisors, without Board or management approval, to assist it in carrying out its duties, including the evaluation of compensation for our non-employee directors. The Compensation Committee may accept, reject or modify any

14

recommendations by compensation consultants or other outside advisors. The purpose of the original study conducted by Sibson Consulting in fiscal 2008 was to assist the Compensation Committee in its goal of maintaining an appropriate level of compensation for our non-employee directors.

With respect to the compensation of our non-employee directors, the Compensation Committee’s goal is to maintain a level of compensation paid to our non-employee directors that is in the median of the companies within our peer group as well as similarly-sized companies. Pursuant to the Non-Employee Director Compensation Plan, non-employee directors receive an annual retainer of $20,000 which is paid in quarterly installments of $5,000. Each director receives $1,500 for participation in each meeting of the Board and its committees and $750 for participation in each telephonic meeting. In addition, the Chairmen of the Audit Committee, Compensation Committee and Nominating and Corporate Governance Committee receive an annual payment of $10,000, $5,000 and $5,000, respectively, which is paid quarterly. Upon each non-employee director’s re-election to the Board, each director will receive $82,250. Upon the initial election to the Board, a non-employee director will receive a stock option grant of 15,000 shares under the 2006 Plan. We believe that our Non-Employee Director Compensation Plan will support our ongoing efforts to attract and retain exceptional directors to provide strategic guidance to our Company. We believe that the total compensation that our non-employee directors receive is in line with our peer group.

Director Compensation Table

The following table sets forth the compensation paid to our Chairman and non-employee directors during fiscal 2010. David A. Levin is not included in the following table as he was a Named Executives Officer and, accordingly, received no compensation for his services as a director. The compensation received by Mr. Levin as an employee of our Company is shown below in the “Summary Compensation Table.”

2010 DIRECTOR COMPENSATION TABLE

Name	Fees Earned or Paid  in Cash ($)(1)		Stock Awards ($)(2)		Option Awards ($)(2)		Non-Equity Incentive Plan Compensation ($)		Change in Pension Value and Nonqualified Deferred Compensation Earnings ($)		All Other Compensation ($)			Total ($)
Seymour Holtzman, Chairman(3)	$	—		—		—		—		—	$	575,000	(3)	$	575,000
Alan S. Bernikow	$	124,750		—		—		—		—		—		$	124,750
Jesse Choper	$	150,250		—		—		—		—	$	32,813	(4)	$	183,063
John E. Kyees(5)	$	105,500		—		63,600		—		—		—		$	169,100
Ward K. Mooney	$	120,250		—		—		—		—		—		$	120,250
George T. Porter, Jr.	$	127,500		—		—		—		—		—		$	127,500
Mitchell S. Presser	$	118,750		—		—		—		—		—		$	118,750

(1) All non-employee directors, with the exception of Mr. Presser, elected to receive all fees paid during fiscal 2010 in cash. Mr. Presser elected to receive all fees paid during fiscal 2010 in unrestricted shares of our common stock. The number of shares issued as payment for an earned director fee is determined by taking the director fee earned and dividing by the closing price of our common stock on the grant date. For quarterly retainer fees, the grant date is the first business day of each respective quarter. For meetings, the grant date is the last business day of the month in which the meeting occurred and for a director’s re-election to the board, the grant date is the last business day of the month in which such re-election occurs. In addition to compensation received for board meetings and committee meetings, Mr. Choper also received a cash payment of $15,000 for serving as Chairman of both the Audit Committee and the Nominating and Corporate Governance Committee. Mr. Porter also received a cash payment of $5,000 for serving as Chairman of the Compensation Committee. Mr. Holtzman did not receive any payment for director meetings.

15

(2) Mr. Kyees was appointed a director of the Company on May 3, 2010 and, accordingly, received a stock option grant of 15,000 shares. The amount in the Option Awards column reflects the aggregate grant date fair value of this stock option award computed in accordance with Financial Accounting Standards Board (“FASB”) Accounting Standards Codification (“ASC”) Topic 718. The fair value of this stock option award is estimated as of the date of grant using a Black-Scholes valuation model. Additional information regarding the assumptions used to estimate the fair value of all stock option awards is included in Note A to Consolidated Financial Statements contained in our Annual Report on Form 10-K for the fiscal year ended January 29, 2011. There were no additional Stock Awards or Options Awards to any of the directors for fiscal 2010. Each director has the following number of stock options outstanding at January 29, 2011: Mr. Holtzman: 1,660,000; Mr. Bernikow: 70,000; Mr. Choper: 85,000; Mr. Kyees: 15,000; Mr. Mooney: 25,000; Mr. Porter: 55,000; and Mr. Presser: 25,000.

(3) During fiscal 2010, Mr. Holtzman received compensation from us both directly (as an employee of our Company) and indirectly (as a consultant pursuant to a consulting agreement we have with JMI). Mr. Holtzman is the president and chief executive officer and, together with his wife, indirectly, the majority shareholder of JMI. There were no services performed on behalf of the Company by JMI, other than those performed by Mr. Holtzman. See below for our discussion of the compensation paid to Mr. Holtzman under “Chairman Compensation.” All Other Compensation for Mr. Holtzman includes annual compensation paid to Mr. Holtzman of $551,000 pursuant to a consulting agreement with the Company and salary of $24,000.

(4) In August 2010, Mr. Choper was granted a cash award of $32,813 due to an oversight with respect to exercising his options and for his years of dedicated service to the Board in his capacity as a member of the Board.

(5) Mr. Kyees was appointed a director of our Company on May 3, 2010.

Chairman Compensation

Pursuant to an employment agreement as well as a consulting agreement with JMI, which was most recently amended in April 2011, Mr. Holtzman receives, in aggregate, annual employment and consulting compensation of $575,000, payable in cash. Mr. Holtzman is the president and chief executive officer and, together with his wife, indirectly, the majority shareholder of JMI. If we engage Mr. Holtzman’s services to assist us in a specific and significant corporate transaction or event, the Compensation Committee, at its discretion, has the right to grant Mr. Holtzman a bonus for his additional services. No such bonus was granted during fiscal 2010. As Chairman of the Board of Directors, in his ex officio capacity pursuant to the Company’s by-laws, Mr. Holtzman regularly attends meetings of the Compensation Committee and the Nominating and Corporate Governance Committee. Mr. Holtzman does not receive compensation for attending these meetings.

Compensation Committee Interlocks and Insider Participation

For fiscal 2010, the members of the Compensation Committee were Messrs. Choper, Mooney and Porter. Persons serving on the Compensation Committee had no relationships with our Company in fiscal 2010 other than their relationship to us as directors entitled to the receipt of standard compensation as directors and members of certain committees of the Board and their relationship to us as beneficial owners of shares of our common stock and options exercisable for shares of common stock. No person serving on the Compensation Committee or on the Board of Directors is an executive officer of another entity for which an executive officer of ours serves on such entity’s board of directors or compensation committee.

Code of Ethics

We have adopted a Code of Ethics which applies to our directors, Chief Executive Officer and Chief Financial Officer, as well as our other senior officers. The full text of the Code of Ethics can be found under “Corporate Governance” on the Investor Relations page of the our corporate web site, which is at www.casualmaleXL.com.

16

EXECUTIVE COMPENSATION

Compensation Discussion and Analysis

This Compensation Discussion and Analysis discusses the compensation paid to our key executives which includes our Chief Executive Officer and our Chief Financial Officer (“Key Executives”) as well as our other executive officers (the “Senior Executives”). For the purposes of this Proxy Statement, the Key Executives and our next three most highly-compensated Senior Executives for fiscal 2010 are collectively referred to herein as our “Named Executive Officers.”

The Named Executive Officers for fiscal 2010 are: David A. Levin, President and Chief Executive Officer, Dennis R. Hernreich, Executive Vice President, Chief Financial Officer, Chief Operating Officer, Treasurer and Secretary, Robert S. Molloy, Senior Vice President and General Counsel, Douglas Hearn, Former Senior Vice President, General Merchandise Manager of Casual Male XL and Global Sourcing, and Henry J. Metscher, Former Senior Vice President, General Merchandise Manager, Direct Businesses and President of Footwear. Messrs. Hearn and Metscher were Senior Executives while employed with us prior to their departures on April 25, 2011 and May 14, 2011, respectively.

Our Key Executives are Messrs. Levin and Hernreich. Our Senior Executives are listed above under “Proposal 1. Election of Directors—Non-Director Executive Officers,” with the exception of Dennis R. Hernreich who is a Key Executive.

The Compensation Committee of our Board of Directors determines the compensation for our Named Executive Officers. The responsibility of our Compensation Committee is to establish, implement and continually monitor adherence to our compensation philosophy, as well as ensure that the total compensation is fair, reasonable, competitive and consistent with the interests of the Company’s stockholders.

Executive Summary

We believe that our executive compensation policies and practices appropriately balance the interests of our executives with those of our stockholders. Performance is a key component of our philosophy for executive compensation. Accordingly, our executive’s compensation, specifically our compensation for our Key Executives, is heavily weighted toward “at risk” performance-based compensation. Our executives’ compensation for 2010 consisted of base salary (“guaranteed compensation”) and performance-based compensation (“at-risk compensation”). The performance-based compensation consisted of: (i) an annual cash incentive based on achievement of a specified performance goal for fiscal 2010 and (ii) a long-term incentive which is also based on achievement of specified performance goals for fiscal 2010, but which vests ratably over a three-year period starting on the first anniversary of the date of grant. See “Components of Executive Compensation” below for a full description of these two incentive plans.

During fiscal 2010, with the exception of the reinstatement of a 5% salary reduction that all of senior management took in fiscal 2009, base salary for our Key Executives and Senior Executives remained flat for fiscal 2010. Accordingly, the only opportunity for our Key Executives and Senior Executives to achieve compensation above 2009 levels was to exceed the performance targets established for our annual and long-term incentive plans to the same degree as in fiscal 2009.

17

The following charts show the breakdown of compensation for our Key Executives and Senior Executives, assuming performance under our annual and long-term incentive plans were achieved at “target” levels.

Summary of Fiscal 2010 Performance Compared to Fiscal 2009

We exceeded our original expectations for earnings in fiscal 2010, despite the slow economic recovery. With an essentially flat sales base in fiscal 2010, we were able to double our operating income from $8.0 million in fiscal 2009 to $16.2 million for fiscal 2010, resulting in diluted earnings per share of $0.32 per share as compared to $0.14 per share in fiscal 2009.

	% Change			For the fiscal year ending:
				January 29, 2011 (Fiscal 2010)			January 30, 2010 (Fiscal 2009)
	(in millions except per share and percentages)
Sales		(0.3	)%	$	393.6		$	395.2
Operating Income		102.5	%	$	16.2		$	8.0
Operating Margin Percentage		105.0	%		4.1	%		2.0	%
Net income		152.5	%	$	15.4		$	6.1
Adjusted EBITDA, a non-GAAP measure(1)		25.5	%	$	29.5		$	23.5
Earnings per share—diluted		128.6	%	$	0.32		$	0.14

(1) EBITDA, which is defined as income from continuing operations before interest, taxes, depreciation and amortization, is adjusted each period to exclude other income (which represents the deferred gain related to the sale of our loss prevention subsidiary in fiscal 2006) (“Adjusted EBITDA”). EBITDA and Adjusted EBITDA are non-GAAP measures and should not be considered superior to or as a substitute for net income derived in accordance with GAAP. We use Adjusted EBITDA because: (i) it measures performance over the periods in which executives can have significant impact, (ii) is directly linked to our long-term growth plan, and (iii) is a key metric used by management and the board to assess our operating performance. Adjusted EBITDA for fiscal 2010 is calculated as net income of $15.4 million, plus interest of $0.7 million, taxes of $0.7 million and depreciation and amortization of $13.2 million minus other income (related to the deferred gain) of $0.5 million. Adjusted EBITDA for fiscal 2009 is calculated as net income of $6.1 million, plus interest of $1.0 million, taxes of $1.5 million and depreciation and amortization of $15.5 million minus other income (related to the deferred gain) of $0.6 million.

18

Summary of Total Compensation Earned for Fiscal 2010 Compared to Fiscal 2009

The compensation earned by the Key Executives and Senior Executives recognized these financial achievements. However, on a comparative basis, the compensation earned by our Named Executive Officers for fiscal 2010 was slightly less than compensation earned for fiscal 2009. Because of the economic recession in fiscal 2009, the financial targets for fiscal 2009 did not include any sales growth and, instead, focused on strengthening gross margins and liquidity and reducing overhead. We exceeded these goals for fiscal 2009 by improving our gross margins through strong inventory control and negotiated rent reductions and by reducing our selling, general and administrative expenses by $30.0 million on an annualized basis. As a result, incentive payouts for fiscal 2009 under our annual and long-term incentive plans exceeded target levels. For fiscal 2010, the performance targets, which represented an increase of approximately 20% over fiscal 2009 actual results, were based on an expectation that retail sales would start to rebound. However, sales throughout the retail industry were much slower to recover than planned, resulting in an essentially flat sales base for us in fiscal 2010. Despite the sales shortfall against expectations, we were able to further improve our merchandise margins and control expenses. In addition, through improved sales productivity, we were able to improve our conversion rates in our stores (the number of people who visit a store and make a purchase) and average dollars per transaction metrics. These improvements helped us to offset the continued weak traffic flow. The combination of these achievements in fiscal 2010 enabled us to exceed slightly the performance targets established for fiscal 2010, but were insufficient to reach the level achieved in fiscal 2009. See the “Summary Compensation Table” on page 30 for a detail breakdown of compensation for each Named Executive Officer. Based on our results for fiscal 2010, we achieved our performance targets for our annual and long-term incentive plans for fiscal 2010.

The following table summarizes total compensation earned by each Named Executive Officer for fiscal 2010 as compared to fiscal 2009.

Named Executive Officer	% Change 2010 vs. 2009			January 29, 2011 (Fiscal 2010)(1)		January 30, 2010 (Fiscal 2009)(1)
David A. Levin, President and Chief Operating Officer		(1.2	)%	$	2,922,629	$	2,959,541
Dennis R. Hernreich Executive Vice President, Chief Financial Officer, Chief Operating Officer, Treasurer and Secretary		(13.5	)%	$	1,969,221	$	2,275,693
Robert S. Molloy Senior Vice President and General Counsel		(11.8	)%	$	697,952	$	791,122
Douglas Hearn Former Senior Vice President, General Merchandise Manager, Casual Male XL and Global Sourcing(2)		(9.1	)%	$	592,795	$	652,445
Henry J. Metscher Former Senior Vice President, General Merchandise Manager, Direct Businesses and President of Footwear(2)		(11.3	)%	$	644,711	$	726,977

(1) Includes performance-based awards, see table below for components of performance-based compensation.

(2) Unvested long-term incentive awards for Messrs. Hearn and Metscher were forfeited due to their employment terminations. See “Summary Compensation Table” for additional information.

19

Total Performance-Based Compensation Earned for Fiscal 2010 and Fiscal 2009

Named Executive Officer	Annual Incentive Plan– Cash				Long-Term Incentive Cash/Equity Subject to 3-yr vesting(1)				Total Performance-Based Compensation Earned
	2010		2009		2010		2009		2010		2009
David A. Levin	$	867,984	$	1,173,510	$	859,872	$	975,578	$	1,727,856	$	2,149,088
Dennis R. Hernreich	$	665,454	$	899,691	$	659,234	$	747,944	$	1,324,688	$	1,647,635
Robert S. Molloy	$	121,713	$	164,555	$	241,150	$	273,600	$	362,863	$	438,155
Douglas Hearn	$	102,987	$	134,405	$	204,050	$	223,471	$	307,037	$	357,876
Henry J. Metscher	$	112,350	$	151,897	$	222,600	$	252,553	$	334,950	$	404,450

(1) Represents awards granted under our long-term incentive plan, which were granted on March 16, 2011 for fiscal 2010 performance and on March 19, 2010 for fiscal 2009 performance. Each award vests over three years from the respective dates of grant. Messrs. Hearn and Metscher awards for fiscal 2010 were fully forfeited and for fiscal 2009, two-thirds of their award was forfeited due to their termination of employment.

Executive Compensation Philosophy and Objectives

Our Compensation Committee’s overarching compensation guiding principle is to reward our executives for the achievement of our primary business objectives: to grow our market share within the Big & Tall retail industry, to increase earnings and operating margins and, ultimately, to increase stockholder returns through increased stock price. Given the recent economic recession and its direct impact on our business and market capitalization, the Committee is challenged with providing competitive compensation opportunities to our executives, both short-term and long-term, during a period where our sales trends are expected to recover slowly.

The Compensation Committee believes that the most effective executive compensation program is one designed to:

• Attract, retain and engage the executive talent the Company needs to deliver on our performance expectations;

• Reward the achievement of specific annual, long-term and strategic goals through a combination of both cash and stock-based compensation;

• Align the executives’ interests with those of our stockholders; and

• Deliver a total compensation opportunity competitive with that available to similarly situated executives at our peer companies.

When reviewing compensation, the Compensation Committee evaluates the pay structure in two primary ways: “total cash compensation” and “total direct compensation.” The total cash compensation, which consists of the executive’s base salary and an annual performance-based cash incentive award, is tied to our annual performance targets, which is generally Adjusted EBITDA but can include other performance targets. The total direct compensation includes the total cash compensation plus target long-term incentive awards. Long-term incentives reward the achievement of our long-term objectives of increases in Adjusted EBITDA and operating margin percentage, which we believe is aligned with stockholder returns.

We balance the mix of long-term and annual compensation in order to ensure adequate base compensation and annual incentive opportunities to attract and retain executive talent, while providing meaningful incentives for them to create long-term sustained value for our Company and our stockholders.

With respect to the overall compensation paid to our Key Executives, the Compensation Committee’s objective is to aim toward a total direct compensation amount that is within our peer median if we are able to achieve financial targets. As such, the Compensation Committee’s objective is to allocate the Key Executive’s total direct compensation equally between base salary, short-term and long-term rewards.

20

For our Senior Executives, the Compensation Committee’s overall objective is to provide them with a competitive base salary that is within our peer median, while providing them with an opportunity for short-term and long-term compensation if our Company meets or exceeds its financial targets, such as Adjusted EBITDA and overall operating margins.

Over the years, the Compensation Committee has continued to fine-tune the pay for performance and shareholder alignment focus of our executive compensation programs. Prior to fiscal 2008, we provided cash compensation in the form of base salary to meet competitive salary norms and rewarded performance against specific short-term goals in the form of cash bonuses. Additionally, we provided stock options to reward superior performance against specific objectives and long-term strategic goals. In fiscal 2008, with the adoption of a Long-Term Incentive Plan, which was amended in June 2010, our management team was offered a compensation plan designed to award superior performance against short-term objectives. However, any award earned (which could be in the form of cash, stock options, restricted stock or any combination thereof based on the prior election of the executives) would be subject to a ratable three-year vesting period commencing on the first anniversary of the date of grant. Beginning in fiscal 2010, there were further refinements made with the intention of improving alignment of executive compensation with stockholders’ interest and the Company’s long-term strategic goals. Specifically, all participants in our long-term incentive plan, which includes our Key Executives and Senior Executives, are required to elect at least 50% of any award earned in equity, either through stock options or shares of restricted stock or a combination thereof, in order to be more aligned with our stockholders.

Risk Assessment

We believe that our compensation programs do not provide incentives for unnecessary risk taking by our employees. Our employment agreements with each of our executives include a “clawback” provision which permits us to demand full repayment of all amounts paid to the executive in the event we learn after the executive’s termination that the executive could have been terminated for “justifiable cause.” Our emphasis on performance-based annual and long-term incentive awards is also designed to align executives with preserving and enhancing shareholder value. Based on these considerations, among others, we do not believe that our compensation policies and practices create risks that are likely to have a material adverse effect on our Company.

Compensation Committee Process

-Key Executives

In consultation with Sibson Consulting, an independent firm which specializes in benefits and compensation, the Compensation Committee has developed a compensation program that is competitive within our peer group. The consultant analyzed compensation and benefits of our peer companies with respect to the compensation for our Named Executive Officers as well as compensation levels and practices for our Board of Directors.

The Compensation Committee has the authority to retain compensation consultants and other outside advisors, without Board or management approval, to assist it in carrying out its duties, including the evaluation of compensation to be paid to our Named Executive Officers. The Compensation Committee may accept, reject or modify any recommendations by compensation consultants or other outside advisors. Since 2006, the Compensation Committee has retained Sibson Consulting to analyze compensation and benefits of our peer companies with respect to our Key Executive compensation. The purpose of their study for fiscal 2010 compensation was to provide the Compensation Committee with current information on the competitiveness of our total compensation and employment package provided to our Named Executive Officers through their current employment agreements. This review also required Sibson Consulting to compare our current Annual Incentive Plan and Long-Term Incentive Plan with similar incentive programs of our peer group.

21

When reviewing the compensation program, the Compensation Committee used the following list of companies as our primary peer group when evaluating and assessing compensation levels for our Named Executive Officers. When determining peer companies, we chose public companies within the specialty retail apparel business with comparable sales and market capitalization:

•    Big 5 Sporting Goods	•    Coldwater Creek	•    JoS A. Bank
•    The Buckle	•    dELiA*s	•    Pacific Sunwear
•    Cache.	•    Destination Maternity	•    Rue 21
•    Cato Group	•    The Finish Line	•    Sport Chalet
•    Christopher Banks	•    Hibbett Sports	•    Wet Seal
•    Citi Trends	•    Hot Topic

The Committee’s overall goal is for our Key Executives’ total direct compensation to fall within the median of our peer group; however, this guideline may differ depending on an individual’s qualifications, role content and scope, past performance, the demand for individuals with the executive’s specific expertise and experience, achievement of our financial objectives and the executive’s contribution to such achievement and overall responsibility, among other criteria.

The Compensation Committee is directly responsible for determining the compensation paid to our Key Executives. For our Key Executives, the Compensation Committee, working with Sibson Consulting, compares each element of compensation to published survey data and proxy data from our peer group for executives with comparable positions and responsibilities. Based on that review for fiscal 2010, the Committee concluded that the total cash and total direct compensation for the two Key Executives was appropriate. This conclusion was based on the following observations:

• Our Company, unlike most of our peers, does not have a Chief Merchandising Officer and a separate Chief Financial Officer. The concentration of these key responsibilities reside with the two Key Executives.

• Mr. Levin’s total cash compensation and total direct compensation is competitive and within our peer median. (The Committee measures “competitive” as compensation that is between 85% to 115% of targeted market level among our peers).

• Mr. Hernreich’s total cash compensation and total direct compensation is above the market median for either a chief financial officer or chief operating officer when compared to our peers. However, our Company is organized differently from our peers. Mr. Hernreich serves in multiple roles as our Executive Vice President, Chief Financial Officer, Chief Operating Officer, Treasurer and Secretary. Because of Mr. Hernreich’s multiple responsibilities, the Committee believes that compensation above the median is appropriate recognition of Mr. Hernreich’s multi-faceted responsibilities. Further, the Committee believes Mr. Hernreich’s total compensation expense to the Company is far below the expense which would be incurred by adding additional executives to fill each of the many roles filled by Mr. Hernreich.

• With respect to incentive award opportunities, the Committee recognizes that the present value of Messrs. Levin’s and Hernreich’s long-term incentive awards granted prior to fiscal 2010 was substantially impacted by the decreased value in the Company’s stock resulting from the poor economy. The Committee believes, however, that as a result of actions taken by the Board of Directors and the Compensation Committee in fiscal 2008 in establishing the Annual Incentive Plan and Long-Term Incentive Plan (as discussed below), there can be significant upside to our Key Executives and Senior Executives if we are able to achieve our short-term and long-term objectives. Similarly, there is also potential upside present in the equity portions of their long-term awards if our objectives are achieved and the economy continues to improve. See “Outstanding Equity Award Value at Fiscal Year End” for a detailed listing of such awards outstanding for each executive.

22

• Furthermore, whereas most of our peers grant a significant portion of long-term awards that vest based on time (and the grant is not predicated on performance), we do not. Both the grant and the ultimate vested value of any long-term incentive award is based on Company performance.

-Senior Executives

The Key Executives, together with our Senior Vice President of Human Resources, are primarily responsible for determining the compensation paid to our Senior Executives, subject to ratification or approval by the Compensation Committee. For benchmarking purposes, the Key Executives utilize several published industry compensation surveys when determining compensation packages for our Senior Executives. Through our subscriptions with Salary.com and the National Retail Federation, we have access to the latest compensation data, which includes both base salary and total compensation, inclusive of incentives. While these sites do not identify the specific companies included in the survey, we are able to access information based on industry, size, such as sales volumes, and regional area, among others. In general, the Company benchmarks compensation against companies in the retail industry which are of similar size, based on comparative sales volumes. When recruiting a senior management position, we also will benchmark against larger or more complex business structures to ensure we retain the best talent to support future growth. A combination of performance, achievement of goals and survey data, among other criteria, is used to determine each Senior Executive’s total direct compensation opportunity. Like our Key Executives, the Senior Executives are provided with a competitive base salary within our retail industry and are provided with an opportunity to earn performance awards each year which are primarily driven by our Company’s overall financial targets. See “Components of Executive Compensation—Performance-based annual cash bonuses and—Long-term performance based awards.”

For our Senior Executives that are also Named Executive Officers for fiscal 2010, the Compensation Committee, with the assistance of Sibson Consulting, also reviewed the total cash compensation and total direct compensation paid to Messrs. Molloy, Hearn and Metscher. Based on a review of our peer group and compensation surveys, the Compensation Committee observed that the compensation paid to Messrs. Molloy, Hearn and Metscher is within the mid-range of our peer group. Further, the compensation to these Senior Executives is more heavily weighted toward our incentive programs than with our peer group. The Committee determined that the compensation opportunity for the Senior Executives who are also Named Executive Officers was appropriate given their specific roles with the Company and the intent of the Committee to emphasize rewards based on achievement of performance goals.

Components of Executive Compensation

The primary components of compensation for our Named Executive Officers include base salary, annual and long-term performance-based incentives. The components of compensation are described as follows:

• Base salary

The base salary represents the fixed component of an executive’s annual compensation. In order to be competitive in the marketplace and attract the top executive talent, we believe that it is important that our base salary be competitive, generally falling within the median of our industry peers.

Base salaries are reviewed annually and adjustments are influenced by our performance in the previous fiscal year and the executive’s contribution to that performance. The executive’s performance is measured by various factors, including, but not limited to, achievement of specific individual and department goals. Additionally, adjustments may consider the individual’s promotion that may have occurred during the fiscal year, and any modifications in the individual’s level of responsibility.

As mentioned above, the Compensation Committee reviews our Key Executives’ overall compensation. The Committee expects Key Executives’ base salary to generally fall in a range that is within the proxy peer median and that approximately one-third of a Key Executive’s total direct compensation will be in the form of base salary. In making base salary decisions for our Senior Executives, the Key Executives rely on published industry compensation surveys and target the market median range.

23

In fiscal 2009, the base salaries for our Key Executives and Senior Executives were reduced by 5.0% as part of our overall Company objectives of reducing expenses at that time. Based on the financial performance in fiscal 2009, on February 5, 2010, the Compensation Committee approved reinstatement of the 5.0% salary adjustment to all members of senior management, including our Key Executives and Senior Executives.

• Performance-based annual cash incentives

The Compensation Committee believes that a substantial portion of each executive’s compensation should tie directly to our financial performance. Our compensation program includes eligibility for an annual performance-based cash incentive opportunity for all executives as well as selected non-executive employees. The Committee believes that an annual cash incentive component of compensation is necessary to provide additional incentive to help achieve the annual goals, which ultimately benefit our stockholder value.

Annual Incentive Plan

The Casual Male Retail Group, Inc. Annual Incentive Plan, which was established in May 2008, is an annual performance-based cash incentive plan. The Plan was amended in August 2010 to, among other things, provide that the plan be considered a sub-plan under the 2006 Plan and not a stand-alone plan. As a result, any award granted under the Annual Incentive Plan is intended to qualify as “performance-based compensation” that is exempt from the deduction limitations imposed by Section 162(m) of the Internal Revenue Code.

The participants include both the Key Executives and the Senior Executives. The Annual Incentive Plan uses Adjusted EBITDA as the benchmark. The Compensation Committee believed it was important to establish a plan that awarded all of management, including our Key Executives and Senior Executives, using the same performance criteria. In addition, there is no personal performance component of the plan for Key Executives and Senior Executives. Thus, there is one single company-wide target. We use Adjusted EBITDA because: (i) it measures performance over the periods in which executives can have significant impact, (ii) is directly linked to our long-term growth plan, and (iii) is a key metric used by management and the board to assess our operating performance.

The target range of Adjusted EBITDA for fiscal 2010 was approved by our Board of Directors on April 9, 2010. Our executives were eligible to receive a cash incentive award ranging from 80% to 150% of their designated target opportunity (which is 100% of plan year base salary for the Key Executives and 35% of plan year base salary for the Senior Executives) if we achieved between 85% and 125% of an Adjusted EBITDA target of $28.5 million for fiscal 2010. Actual Adjusted EBITDA for fiscal 2010 was 103.5% of target; accordingly, on March 16, 2011 the Compensation Committee approved a cash bonus payout of 107% (which equates on the payout curve to 103.5% of target achievement).

The following table summarizes the overall target, performance achievement and payout ratio under the 2010 Annual Incentive Plan (dollar amounts are in millions):

Annual Incentive Performance Target	Target at 100%		Actual Performance (103.5% achieved)		Cash Incentive Payout %
2010 Adjusted EBITDA	$	28.5	$	29.5		107.0	%

The total cash award for the 2010 Annual Incentive Plan was $4.1 million, of which $1.9 million was paid to our Named Executive Officers. As a group, $2.7 million was paid to our Key Executives and Senior Executives.

24

The following table summarizes the incentive opportunity and actual cash award under the 2010 Annual Incentive Plan for each Named Executive Officer:

Named Executive Officer	Target % of Base Salary			Plan Year Base Salary		Individual Award at Target		Actual Payout At 107%
David A. Levin		100	%	$	811,200	$	811,200	$	867,984
Dennis R. Hernreich		100	%	$	621,920	$	621,920	$	665,454
Robert S. Molloy		35	%	$	325,000	$	113,750	$	121,713
Douglas Hearn		35	%	$	275,000	$	96,250	$	102,987
Henry J. Metscher		35	%	$	300,000	$	105,000	$	112,350

On January 27, 2011, the Compensation Committee approved the Adjusted EBITDA target range for fiscal 2011. If the bonuses were to be paid today based on salaries as of May 15, 2011, the cash bonuses would be based on salaries earned of $811,200 for Mr. Levin, $621,920 for Mr. Hernreich and $325,000 for Mr. Molloy. Assuming we achieve 100% of the Adjusted EBITDA target for fiscal 2011, we estimate that the total potential payout would be approximately $3.9 million, of which $2.5 million would be paid to our Key Executives and Senior Executives. The Compensation Committee believes this is a competitive level of compensation and provides incentives to all of our senior executives to achieve our annual financial goals.

The Compensation Committee believes that it is possible to meet or exceed the Adjusted EBITDA target set for fiscal 2011. The established Adjusted EBITDA target is intended to be achievable within an approximate 50% probability as a result of executing the Company’s operating plan. The target level is derived from our annual operating plan and budget for the fiscal year. The operating plan and budget set forth our internal goals and objectives for our growth and development and are expected to require substantial efforts by the entire Company to achieve. As a result, the likelihood of achieving the 2011 Adjusted EBITDA target reflects the challenges inherent in achieving the goals and objectives in the operating plan and budget. The Compensation Committee considered the likelihood of achieving the target levels when approving the target amount, including historical achievement by our executive officers.

• Long-term performance based plans

Our long-term incentives reward the achievement of meeting our current year performance objectives but the awards vest over a three-year period once earned, ultimately benefiting our stockholders while developing and retaining a strong management team.

Long-Term Incentive Plan

We have a Long-Term Incentive Plan, which was most recently amended in June 2010, for which certain key members of senior management, which includes both Key Executives and Senior Executives, are eligible to participate. Pursuant to the plan, the Compensation Committee can establish, annually, targeted performance goals, as provided for under the 2006 Plan. If the Company achieves the targeted performance goals, each participant in the plan will be entitled to receive an award (depending on the target level achieved) of the participant’s “target cash value.” Target cash value is defined as the participant’s actual annual base salary in effect at the commencement of a fiscal year multiplied by the long-term incentive program percentage in such participant’s employment agreement or otherwise on file with the Company (which is 100% for the Key Executives and 70% for the Senior Executives). For fiscal 2010, the targets were based on Adjusted EBITDA and operating margin percentage. For fiscal 2010, for a minimum threshold payout, the Company needed to achieve 85% of the financial targets in order for participants to receive an award of 80% of target cash value and, for the participants to receive a maximum award of 150% of target cash value, the Company needed to achieve 125% of the financial targets. Awards are payable based on the irrevocable election of each participant, which must be made no later than six months prior to the end of each fiscal year. The participant may irrevocably elect either cash, stock options and/or restricted shares or any combination thereof, provided that that Compensation

25

Committee, in its discretion, may require that all or any portion of a participant’s award be paid in shares of restricted stock and/or stock options, as opposed to cash.

For fiscal 2010, the Compensation Committee required each participant to elect to receive at least 50% of any award in either shares of restricted stock or stock options, or any combination thereof (the same requirement will apply in fiscal 2011). The awards will be granted by the Compensation Committee after the audited financial statements for the applicable fiscal year are available, which is expected to occur no later than 90 days after the close of the relevant fiscal year. Any awards granted under the plan vest ratably over a three-year period commencing on the one year anniversary of the date of grant and are subject to forfeiture.

For fiscal 2010, the Adjusted EBITDA target was $28.5 million and the operating margin percent target was 4.0%. As discussed above, participants were entitled to a payout of between 80% and 150% of target cash value if the Company achieved between 85% and 125% of its financial targets for fiscal 2010. In fiscal 2010, the Company achieved 103.5% of its Adjusted EBITDA target and 103.3% of its operating margin percent target, resulting in a weighted payout according to the payout curve of 106.0% of each participant’s target cash value; accordingly, on March 16, 2011, the Compensation Committee approved the grant of awards pursuant to the Long-Term Incentive Plan.

The following table summarizes the overall targets, performance achievements and payout ratios under the 2010 Long-Term Incentive Plan (dollar amounts in millions):

Long-Term Incentive Performance Target	Target at 100%			Actual Performance			Performance Achievement			Incentive Payout %
2010 Operating Margin Percent		4.0	%		4.1	%		103.5	%		107.0	%
2010 Adjusted EBITDA	$	28.5		$	29.5			103.3	%		104.9	%
Weighted Payout											106.0	%

Pursuant to the terms of the Long-Term Incentive Plan, together with the irrevocable elections of each participant, on March 16, 2011 we granted cash awards totaling $1.9 million and equity awards totaling $2.4 million, excluding potential forfeitures that may occur. The equity awards consisted of 538,661 shares of restricted stock and options to purchase 72,578 shares of common stock. Our Named Executive Officers, in aggregate, received cash awards of $0.9 million and equity awards of $1.3 million. Both the cash and equity awards will vest, based on each executive’s election, in three separate tranches, with the first tranche vesting on March 16, 2012. Except in the case of a change in control, our Senior Executives must still be employed by the Company on each annual vesting date to be entitled to the vested-portion of the award, otherwise the award is forfeited. In addition, for our Key Executives, if they are terminated for cause, the awards are forfeited, but if they are terminated without cause or they resign for “good reason” any unvested award would accelerate. The awards earned by Messrs. Hearn and Metscher for fiscal 2010 were fully forfeited due to their termination of employment. Messrs. Hearn and Metscher also forfeited two-thirds of their respective long-term incentive awards for fiscal 2009.

The following table summarizes the incentive opportunity and actual awards under the 2010 Long-Term Incentive Plan for each Named Executive Officer:

Named Executive Officer	Target % of Base Salary			Plan Year Base Salary		Individual Award at Target		Actual Payout, based on Executive Payout Elections
								Actual Payout Cash		Restricted Stock		Options		Total Value in Dollars
David A. Levin		100	%	$	811,200	$	811,200	$	429,936	$	429,936		—	$	859,872
Dennis R. Hernreich		100	%	$	621,920	$	621,920	$	164,808	$	494,426		—	$	659,234
Robert S. Molloy		70	%	$	325,000	$	227,500	$	120,575	$	100,479	$	20,096	$	241,150
Douglas Hearn(1)		70	%	$	275,000	$	192,500	$	102,025	$	102,025		—	$	204,050
Henry J. Metscher(1)		70	%	$	300,000	$	210,000	$	111,300	$	92,750	$	18,550	$	222,600

(1) Due to their respective terminations of employment, Messrs. Hearn’s and Metscher’s awards were fully forfeited.

26

On January 27, 2011, the Compensation Committee approved the targets for fiscal 2011 which, similar to fiscal 2010, will be based on Adjusted EBITDA and operating margin percentage. For fiscal 2011, for a minimum threshold payout, the Company needs to achieve 90% of the financial targets (an increase from the 85% needed in fiscal 2010) in order for participants to receive an award of 50% of target cash value (down from 80% in fiscal 2010) and, for the participants to receive a maximum award of 150% of target cash value, the Company needed to achieve 125% of the financial targets (no change from fiscal 2010). If the targets are achieved, each award will vest, based on each executive’s election, in three separate tranches, with the first tranche vesting on the first anniversary of the date of grant. Assuming we achieve 100% for each target for fiscal 2011, we estimate that the total potential payout will be approximately $3.8 million, which will be expensed over the vesting period of approximately forty-eight months, once it is determined that the achievement of these targets is probable. However, we cannot predict if and when such targets will be met or what the allocation of this award will be between cash, stock options and/or restricted shares; provided that at least 50% must be in the form of equity. Further, the fair market closing price of our common stock will not be known until the time of grant and, accordingly, the number of stock options and restricted shares ultimately issued will also not be known until the time of grant.

The Compensation Committee believes that there is an approximate 50% probability of meeting the Adjusted EBITDA and operating margin percentages set for fiscal 2011. We believe that our performance targets are established at levels that are intended to be difficult but attainable. The target levels of the performance objectives are derived from our annual operating plan and budget for a fiscal year. The operating plan and budget set forth our internal goals and objectives for our growth and development and are expected to require substantial efforts by the entire Company to achieve. As a result, the likelihood of achieving the 2011 performance targets reflects the challenges inherent in achieving the goals and objectives in the operating plan and budget. The Compensation Committee considered the likelihood of achieving the target levels when approving the target amounts, including historical achievement by our executive officers.

• Long-term equity incentives

Through fiscal 2007, as part of our long-term compensation program, our Compensation Committee had traditionally granted stock options to our Key Executives in order to align our Key Executives’ interests more closely with the interests of our stockholders. The amount of such awards was determined usually once in each fiscal year by the Compensation Committee. The Compensation Committee determined the amount of the stock-based awards granted to the Key Executives usually based on the impact of our financial and operational performance on our stock price but could also take into account other factors in determining the size of stock-based awards, including, but not limited to, special accomplishments during the preceding year.

In May 2008, Messrs. Levin and Hernreich were granted options to purchase an aggregate 500,000 and 400,000 shares of our common stock, respectively, with exercise prices ranging from $4.54-$4.55, the closing price of the Company’s common stock on the date of grant. Unlike previous stock option grants to our Key Executives, which were directly tied to our previous year’s annual stock price performance, these grants to Messrs. Levin and Hernreich represented retention awards. The awards were subject to a three-year cliff vest and expire in May 2012 (i.e., the ability to exercise these awards is limited to one year, between May 2011 when the awards vested and May 2012, when they expire).

With the adoption of the Long-Term Incentive Plan in 2008, the Compensation Committee has not made annual grants of stock options. Instead, the long-term component of compensation is directly tied to the achievement of our performance measures pursuant to our Long-Term Incentive Plan.

• Discretionary Cash and Equity Awards

On March 31, 2010, the Board of Directors granted to Mr. Levin an option to purchase 347,566 shares of our common stock. The exercise price of this option is $4.55 per share and the closing price of our stock on the date of grant was $3.86 per share. The grant date fair value of the award was $361,469. The

27

option cliff vested on May 6, 2011 and will expire on May 6, 2012. This grant to Mr. Levin represented a retention award. The Board of Directors believes that this award was appropriate given the equity holdings of CEOs in the peer companies and in order to ensure that Mr. Levin’s compensation is closely aligned with the shareholder’s long-term interest. To avoid any appearance of impropriety, the Compensation Committee determined that the vesting price and term of vesting should be identical to the terms which governed his previous option to purchase 500,000 shares which Mr. Levin voluntarily sold to the Company in January 2009.

• Other Compensation

In addition to our life insurance programs available to all of our employees, we also pay the insurance premiums for an additional $2.0 million life insurance policy for each Key Executive to the benefit of their designated beneficiary(ies). We also offer our Key Executives and Senior Executives supplemental Disability Insurance which is not offered to our other employees.

In addition to the foregoing, our executive officers receive benefits under certain group health, long-term disability and life insurance plans, which are generally available to all of our eligible employees.

After six months of service with us, executive officers are eligible to participate in the 401(k) Salaried Plan. Benefits under these plans are not tied to corporate performance. In particular circumstances, we also utilize cash signing bonuses and stock awards when certain employees join the Company. In May 2009, in connection with our cost reduction initiatives, the Company ceased any further employer contributions to the 401(k) Plans, effective May 31, 2009. In November 2010, the administrative committee voted to reinstate the “safe harbor” matching and vesting provision effective January 1, 2011. As a result, there were no employer contributions to the 401(k) Plans during fiscal 2010.

• Termination Based Compensation

We have employment agreements with all of our Key Executives and Senior Executives. Upon termination of employment, all of our Key Executives and Senior Executives are entitled to receive severance payments under their employment agreements in the event of terminations without justifiable cause. These employment agreements are discussed in detail below following the “Summary Compensation Table.” Our employment agreements do not contain any tax gross ups pursuant to Section 280(g) of the Internal Revenue Code.

• Tax Implications

Under Section 162(m) of the Internal Revenue Code, certain executive compensation in excess of $1 million paid to a principal executive officer and the three most highly compensated executive officers (other than the principal executive officer) at the end of any fiscal year is limited and is not deductible for federal income tax purposes unless the compensation qualifies as “performance-based compensation.” The Compensation Committee’s policy with respect to Section 162(m) is to make every reasonable effort to cause compensation to be deductible by us while simultaneously providing our executive officers with appropriate rewards for their performance.

On August 5, 2010, the stockholders of the Company approved a proposal to amend the 2006 Plan to increase the Section 162(m)-based individual limitations on the amount of certain awards and re-approved the material terms of the 2006 Plan in order to preserve tax deductibility under Section 162(m).

28

COMPENSATION COMMITTEE REPORT

We, the Compensation Committee of the Company, have reviewed and discussed the Compensation Discussion and Analysis required by Item 402(b) of Regulation S-K with management and, based on this review and discussion, recommend to the Board of Directors that the Compensation Discussion and Analysis be included in this Proxy Statement.

The Compensation Committee

George T. Porter, Jr., Chairman

Jesse Choper

Ward K. Mooney

29

Summary Compensation Table. The following Summary Compensation Table sets forth certain information regarding compensation paid or accrued by us with respect to the President and Chief Executive Officer, the Chief Financial Officer/Chief Operating Officer, the Senior Vice President and General Counsel, the former Senior Vice President, General Merchandise Manager, Casual Male XL and Global Sourcing and the former Senior Vice President, General Merchandise Manager, Direct Businesses and President of Footwear (collectively, the “Named Executive Officers”), for fiscal 2010.

SUMMARY COMPENSATION TABLE

Name and Principal Position	Year		Salary ($)		Bonus ($)(2)			Stock Awards ($)(1)(2)			Option Awards ($)(1)(2)			Non-Equity Incentive Plan Compensation ($)(2)		Change in Pension Value and Nonqualified Deferred Compensation Earnings ($)		All Other Compensation ($)(3)		Total ($)
David A. Levin		2010	$	810,420		—		$	429,936	(4)	$	361,469	(6)	$	1,297,920		—	$	22,904	$	2,922,649
President and Chief Executive Officer		2009	$	782,340		—		$	487,789	(5)		—		$	1,661,299		—	$	28,113	$	2,959,541
		2008	$	803,400	$	300,000	(7)		—		$	825,000			—		—	$	20,220	$	1,948,620
Dennis R. Hernreich		2010	$	621,322		—		$	494,426	(4)		—		$	830,262		—	$	23,211	$	1,969,221
Executive Vice President, Chief Financial Officer, Chief Operating Officer, Treasurer and Secretary		2009	$	599,794		—		$	373,972	(5)		—		$	1,273,663		—	$	28,264	$	2,275,693
		2008	$	615,940	$	240,000	(7)		—		$	660,000			—		—	$	25,323	$	1,541,263
Robert S. Molloy		2010	$	324,688		—		$	100,479	(4)	$	20,096	(4)	$	242,288			$	10,401	$	697,952
Senior Vice President and General Counsel		2009	$	313,438		—		$	70,400	(5)	$	22,800	(5)	$	369,755			$	14,729	$	791,122
		2008	$	321,875		—			—		$	16,600			—		—	$	10,631	$	349,106
Douglas Hearn		2010	$	274,760		—		$	102,025	(4)		—		$	205,012		—	$	10,998	$	592,795
Former Senior Vice President, General Merchandise Manager, Casual Male XL and Global Sourcing(8)		2009	$	256,010		—		$	24,800	(5)		—		$	357,876		—	$	13,760	$	652,446
		2008	$	159,615		—			—		$	17,700			—		—	$	4,597	$	181,912
Henry J. Metscher		2010	$	299,712		—		$	92,750	(4)	$	18,550	(4)	$	223,650		—	$	10,049	$	644,711
Former Senior Vice President, General Merchandise Manager, Direct Businesses and President of Footwear(8)		2009	$	289,327		—		$	24,800	(5)		—		$	404,450		—	$	8,400	$	726,977
		2008	$	293,750	$	10,746	(7)		—			—			—		—	$	8,100	$	312,596

(1) The amounts shown in the Stock Awards and Option Awards columns represent the aggregate grant date fair value of awards computed in accordance with FASB ASC Topic 718, not the actual amounts paid to or realized by the Named Executive Officers during the applicable fiscal year. The FASB ASC Topic 718 fair value amount as of the grant date for stock awards and stock options generally is spread over the number of months of service required for the grant to vest. An explanation of the vesting of restricted stock awards and option awards, as well as the methodology for performance share payouts, is discussed in the footnotes to the Grants of Plan Based Awards for 2010 and Outstanding Equity Awards at 2010 Fiscal Year End tables below.

(2) The table “Discretionary and Incentive-Based Awards” below provides a detailed breakdown with respect to awards earned.

In addition to salary, we also compensate our executives through a combination of discretionary and incentive-based awards. Since 2008, we have had two incentive-based programs, the Annual Incentive Plan and the Long-Term Incentive Plan. The Annual Incentive Plan is a cash-based program which awards certain members of management, which include our Named Executive Officers, if certain financial targets are achieved for the applicable fiscal year. The Long-Term Incentive Plan is a dollar-denominated program for certain members of management, including our Key Executives and Senior Executives, which permits each participant to choose, through an irrevocable election, to receive awards in cash, stock, and/or options or a combination thereof. For fiscal 2010, participants of the Long-Term Incentive Plan were required to elect to receive at least 50% of any award in equity, through shares of restricted stock or stock options or a combination thereof. Although the financial targets of the Long-Term Incentive Plan are annual, all awards earned pursuant to the Long-Term Incentive Plan are subject to a three-year vesting.

For Stock Awards and Option Awards, the amounts reflect the fair value, as of grant date, of both performance and non-performance awards computed in accordance with FASB ASC Topic 718. The fair value of each stock option award is estimated as of the date of grant using a Black-Scholes valuation model. Additional information regarding the assumptions used to estimate the fair value of all stock option awards is included in Note A to Consolidated Financial Statements contained in our Annual Report on Form 10-K for the fiscal year ended January 29, 2011.

(3) See table “All Other Compensation” below for a breakdown.

30

(4) Included in the above table is the actual grant date fair value of performance equity awards (restricted stock and stock options) issued on March 16, 2011, in accordance with each executive’s irrevocable election, pursuant to the Company’s Long-Term Incentive Plan (“LTIP”) for fiscal 2010 performance for Messrs. Levin, Hernreich, Molloy, Hearn and Metscher of $429,936, $494,426, $120,575 ($100,479 stock awards and $20,096 stock options), $102,025, and $111,300 ($92,750 stock awards and $18,550 stock options), respectively. The Compensation Committee approved the targets of the 2010 LTIP on April 9, 2010 (the service inception date), at which point the fair value of the performance equity awards, based upon the probable outcome of the performance condition on that date, approximated $324,480, $124,384, $91,000 ($75,833 stock awards and $15,167 stock options), $77,000 and $84,000 ($70,000 stock awards and $14,000 stock options) for Messrs. Levin, Hernreich, Molloy, Hearn and Metscher, respectively, excluding the effect of estimated forfeitures. If the highest level of performance were to be assumed, the fair value of the performance equity awards on April 9, 2010 would be $608,400, $233,220, $170,825 ($142,354 stock awards and $28,471 stock options), $144,375 and $157,500 ($131,250 stock awards and $26,250 stock options) for Messrs. Levin, Hernreich, Molloy, Hearn and Metscher, respectively, excluding the effect of estimated forfeitures.

(5) Included in the above table is the actual grant date fair value of performance equity awards (restricted stock and stock options) issued on March 19, 2010, pursuant to the Company’s LTIP for fiscal 2009 performance for Messrs. Levin, Hernreich and Molloy of $487,789, $373,972 and $68,400 ($45,600 stock awards and $22,800 stock options), respectively. The Compensation Committee approved the targets of the LTIP on April 29, 2009 (the service inception date), at which point the fair value of the performance equity awards, based upon the probable outcome of the performance condition on that date, approximated $312,936, $239,918 and $43,881($29,254 stock awards and $14,627 stock options) for Messrs. Levin, Hernreich and Molloy, respectively, excluding the effect of estimated forfeitures. If the highest level of performance were to be assumed the fair value of the performance equity awards on April 29, 2009 would be $586,755, $449,846 and $82,277 ($54,852 stock awards and $27,425 stock options), respectively, excluding the effect of estimated forfeitures.

(6) On March 31, 2010, the Board of Directors granted to Mr. Levin an option to purchase 347,566 shares of our common stock. The exercise price of this option is $4.55 per share and the closing price of our common stock on the date of grant was $3.86 per share. The option fully vested on May 6, 2011 and will expire on May 6, 2012.

(7) In May 2009, Messrs. Levin and Hernreich received discretionary bonuses of $300,000 and $240,000, respectively, for fiscal 2008. Mr. Metscher received a discretionary bonus $10,746 in fiscal 2008.

(8) On April 25, 2011, Mr. Hearn’s employment was terminated and on May 14, 2011, Mr. Metscher’s employment was terminated. Because the employment of Messrs. Hearn and Metscher terminated before the vesting of their 2010 LTIP award, the full value of the LITP award of $204,050 and $222,600, respectively was forfeited. In addition, the unvested portion of Messrs. Hearn’s and Metscher’s 2009 LTIP award of $148,981 and $168,369, respectively, was also forfeited.

The table set forth below is a supplement to the Summary Compensation Table and provides a breakdown of discretionary and incentive-based awards earned by each Named Executive Officer.

Discretionary and Incentive-Based Awards

			Discretionary Awards						Long-Term Incentive Plan All Awards Subject to Three Year Vesting(1)						Annual Incentive Plan		Totals per Summary Compensation Table
Name	Year		Bonus- Cash		Stock Awards		Option Awards		Non-Equity (Cash)		Stock Awards		Option Awards		Non-Equity (Cash)		Bonus		Stock Awards		Option Awards		Non-Equity Incentive Plan Compensation
David A. Levin		2010		—		—	$	361,469	$	429,936	$	429,936		—	$	867,984		—	$	429,936	$	361,469	$	1,297,920
		2009		—		—		—	$	487,789	$	487,789		—	$	1,173,510		—	$	487,789		—	$	1,661,299
		2008	$	300,000		—	$	825,000		—		—		—		—	$	300,000		—	$	825,000		—
Dennis R. Hernreich		2010		—		—		—	$	164,808	$	494,426		—	$	665,454		—	$	494,426		—	$	830,262
		2009		—		—		—	$	373,972	$	373,972		—	$	899,691		—	$	373,972		—	$	1,273,663
		2008	$	240,000		—	$	660,000		—		—		—		—	$	240,000		—	$	660,000		—
Robert S. Molloy		2010		—		—		—	$	120,575	$	100,479	$	20,096	$	121,713		—	$	100,479	$	20,096	$	242,288
		2009		—	$	24,800		—	$	205,200	$	45,600	$	22,800	$	164,555		—	$	70,400	$	22,800	$	369,755
		2008		—		—	$	16,600		—		—		—						—	$	16,600		—
Douglas Hearn		2010		—		—		—	$	102,025	$	102,025		—	$	102,987		—	$	102,025		—	$	205,012
		2009		—	$	24,800		—	$	223,471		—		—	$	134,405		—	$	24,800		—	$	357,876
		2008		—		—	$	17,700		—		—		—		—		—		—	$	17,700		—
Henry J. Metscher		2010		—		—		—	$	111,300	$	92,750	$	18,550	$	112,350		—	$	92,750	$	18,550	$	223,650
		2009		—	$	24,800		—	$	252,553		—		—	$	151,897		—	$	24,800		—	$	404,450
		2008	$	10,746		—		—		—		—		—		—	$	10,746		—		—		—

(1) The Company’s LTIP provides for dollar-based awards which, at the irrevocable election of the participant, can be awarded in cash, restricted stock, options or any combination thereof. For fiscal 2010, the Compensation Committee required that each participant elect to receive at least 50% of any award in either shares of restricted stock or stock options, or a combination thereof. Messrs. Levin and Hearn irrevocably elected to receive 50% of any award and Mr. Hernreich irrevocably elected to receive 75% of any award in shares of restricted stock with the remaining portion in cash. Messrs. Molly and Metscher elected to receive 50% of any award in cash, 41.7% in shares of restricted stock and 8.3% in stock options. Although the service inception date for these awards occurred in fiscal 2010, no payouts occurred until fiscal 2011, when it was determined that the performance targets were achieved. All awards vest, based on the executive’s election, in three separate tranches, with the first tranche vesting on March 16, 2012. Because Messrs. Hearn and Metscher’s employment with the Company terminated prior to any vesting of the LTIP award for 2010, their LTIP awards for fiscal 2010 were forfeited. In addition, for the 2009 LTIP award, Messrs. Hearn and Metscher forfeited the unvested portion of $148,981 and $168,369, respectively.

31

No payments were made to our Named Executive Officers pursuant to our LTIP or Annual Incentive Plan for fiscal 2008.

The table below sets forth the components of All Other Compensation listed above in the Summary Compensation Table.

All Other Compensation

Name	Year		Auto Allowance		401(k)/ Nonqualified Deferred Compensation Plan(1)(2)		Life Insurance Premiums		Supplemental Disability Insurance		Total All Other Compensation
David A. Levin		2010	$	10,000		—	$	2,527	$	10,377	$	22,904
		2009	$	9,696	$	6,125	$	1,915	$	10,377	$	28,113
		2008	$	9,600	$	8,050	$	2,570		—	$	20,220
Dennis R. Hernreich		2010	$	10,000		—	$	1,796	$	11,415	$	23,211
		2009	$	8,782	$	6,009	$	5,984	$	7,489	$	28,264
		2008	$	8,723	$	8,050	$	8,550		—	$	25,323
Robert S. Molloy		2010	$	8,100		—		—	$	2,301	$	10,401
		2009	$	7,200	$	5,228		—	$	2,301	$	14,729
		2008	$	7,131	$	3,500		—		—	$	10,631
Douglas Hearn		2010	$	8,400		—		—	$	2,598	$	10,998
		2009	$	7,915	$	3,247		—	$	2,598	$	13,760
		2008	$	4,597		—		—		—	$	4,597
Henry J. Metscher		2010	$	8,400					$	1,649	$	10,049
		2009	$	8,400		—		—		—	$	8,400
		2008	$	8,100		—		—		—	$	8,100

(1) In May 2009, in connection with its cost reduction initiatives, the Company ceased any further employer contributions to the 401(k) Plan, effective May 31, 2009. In November 2010, the Company decided to reinstate the “safe harbor” matching and vesting provision effective January 1, 2011.

(2) Through the end of fiscal 2008, we had a Non-Qualified Deferred Compensation Plan which was started in fiscal 2006 for the primary purpose of providing deferred compensation for a select group of management or highly compensated employees of the Company. Effective December 31, 2008, the plan was amended to freeze future contributions to the plan and balances were distributed to participants in fiscal 2009.

Employment Agreements

Key Executives

We have employment agreements, which were revised and restated as of November 5, 2009 (collectively, the “Employment Agreements”), with Mr. Levin and Mr. Hernreich. The initial three-year term of the Employment Agreements was January 1, 2009 through December 31, 2011. On January 1, 2011 and thereafter on each anniversary of the Employment Agreements’ commencement date, the term will be extended for a one-year period, which makes the Employment Agreements a rolling two-year agreement after the initial three-year term.

These Employment Agreements require each executive officer to devote substantially all of the executive officer’s time and attention to our business as necessary to fulfill his respective duties. The Employment Agreements provide that Messrs. Levin and Hernreich would be paid base salaries at annual rates of $811,200 and $621,921, respectively, with an annual automobile allowance of $10,000. During the period May 17, 2009 through February 1, 2010, the Compensation Committee approved a decrease for all employees. This included a 5% decrease in the base salaries for Messrs. Levin and Hernreich. The decreases were reinstated on February 1, 2010.

32

Messrs. Levin and Hernreich are eligible to participate in our Annual Incentive Plan at a target rate of 100% of each executive’s actual annual base salary, as defined in that plan, and in our Long-Term Incentive Plan (“LTIP”) at a target incentive rate of 100% of each executive’s combined actual annual base earnings for the incentive period. The Employment Agreements also provide for the payment of discretionary bonuses in such amounts as may be determined by the Compensation Committee or Board of Directors.

The Employment Agreements provide that in the event the executive officer’s employment is terminated by the Company at any time for any reason other than “justifiable cause” (as defined in the Employment Agreements), disability or death, or in the event the executive officer resigns with “good reason” (as defined in the Employment Agreement), we are required to pay the executive the following:

• A pro rata bonus under the Annual Incentive Plan and the LTIP;

• The accelerated vesting of LTIP awards; and

• A severance comprised as the sum of (1) the executive’s monthly base salary then in effect plus (2) a monthly amount calculated by dividing by twelve the average of the sum of (i) the annual incentive bonuses earned and (ii) the cash amounts paid to the executive pursuant to the LTIP or the cash value of the options or stock issued to the executive, during each of the two most recent fiscal years, with the monthly sum of (1) plus (2) payable for 24 consecutive months. This severance benefit is conditioned upon the executive officer’s execution of a general release.

The above-listed payments are not made if the executive officer is terminated with “justifiable cause,” the executive officer resigns and such resignation is not for “good reason”, or the executive officer dies or becomes disabled; provided, however, that if the executive officer’s employment terminates by reason of death, disability or retirement on or after age 65, the executives officer shall be entitled to a pro-rata bonus under the Annual Incentive Plan.

In the event the executive officer’s employment is terminated at any time within one (1) year following a Change of Control (as defined in the Employment Agreement) other than for “justifiable cause,” or if the executive resigns for “good reason”, the executive is entitled to receive a lump sum payment equal to the sum of two times the executive’s base salary, and two times the executive’s target annual incentive bonus for the fiscal year in which the Change of Control occurs. Payments made under this provision are to be reduced if and to the extent necessary to avoid any payments or benefits to executive being treated as “excess parachute payments” within the meaning of Internal Revenue Code Section 280G(b)(i). This payment also is conditioned upon the executive officer’s execution of a general release.

The Employment Agreements contain confidentiality provisions pursuant to which each executive officer agrees not to disclose confidential information regarding our Company. The Employment Agreements also contain covenants pursuant to which each executive officer agrees, during the term of his employment and for a one-year period following the termination of his employment, not to have any connection with any business which is a specialty retailer that primarily distributes, sells or markets so-called “big and tall” apparel of any kind for men or which utilizes the “big and tall” retail or wholesale marketing concept as part of its business. The Employment Agreements also contain a “clawback” provision which provides for remedies in the event we learn after the executive officer’s termination by us other than for “justifiable cause” that the executive officer could have been terminated for “justifiable cause”. Subject to the implementation of final rules by the SEC, the Dodd-Frank Act requires that we implement a policy providing for the recovery of erroneously paid incentive-based compensation following a required accounting restatement. Once the final rules are issued by the SEC, we will revise, in a timely manner, the current clawback provision of our employment agreements.

The Employment Agreements also provide that we will, during the term of employment, pay the insurance premiums under one or more life insurance policies on the executive officer’s life pursuant to an arrangement under which $2,000,000 of the death benefit under the policy or policies would be payable to the executive officer’s named beneficiary (with the executive officer making the election of the designated beneficiary) upon the executive officer’s death.

33

Senior Executives

We also have employment agreements with each of our Senior Executives (the “Sr. Exec. Employment Agreements”). The term of each employment agreement begins on the respective effective date and continues until terminated by either party. Pursuant to their respective employment agreements, our Senior Executives are eligible to participate in our Annual Incentive Plan at 35% of their respective average base salaries, as defined in the plan, depending on our performance (based on achieving annual Adjusted EBITDA targets). Each Senior Executive is also eligible to participate in our LTIP at 70% of their respective average base salaries, as defined in the plan, depending on our performance (based on annual performance goals). Each executive is entitled to vacation and to participate in and receive any other benefits customarily provided by us to our senior executives.

The Sr. Exec. Employment Agreements provide that in the event the executive officer’s employment is terminated by us at any time for any reason other than “justifiable cause” (as defined in the Employment Agreements), disability or death, we are required to pay the executive his then current base salary for five months after the effective date of such termination. This severance benefit is conditioned upon the senior executive’s execution of a general release.

The above-listed payments are not made if the senior executive is terminated with “justifiable cause,” the senior executive resigns, or the senior executive dies or becomes disabled.

In the event the senior executive’s employment is terminated at any time within one (1) year following a Change of Control (as defined in the Sr. Exec. Employment Agreement) other than for “justifiable cause,” or if the senior executive resigns for “good reason,” we shall pay the senior executive an amount equal to twelve months of executive’s highest base salary in effect at any time during the six (6) month period ending on the date of the Change of Control. This payment also is conditioned upon the senior executive’s execution of a general release. Payments made under this provision are to be reduced if and to the extent necessary to avoid any payments or benefits to senior executive being treated as “excess parachute payments” within the meaning of Internal Revenue Code Section 280G(b)(i).

The Sr. Exec. Employment Agreements contain confidentiality provisions pursuant to which each senior executive agrees not to disclose confidential information regarding our Company. The Sr. Exec. Employment Agreements also contain covenants pursuant to which each senior executive agrees, during the term of his employment and for a one-year period following the termination of his employment, not to have any connection with any business which is a specialty retailer that primarily distributes, sells or markets so-called “big and tall” apparel of any kind for men or which utilizes the “big and tall” retail or wholesale marketing concept as part of its business. The Sr. Exec. Employment Agreements also contain a “clawback” provision which provides for remedies in the event we learn after the senior executive’s termination by us other than for “justifiable cause” that the senior executive could have been terminated for “justifiable cause”. Subject to the implementation of final rules by the SEC, the Dodd-Frank Act requires that we implement a policy providing for the recovery of erroneously paid incentive-based compensation following a required accounting restatement. Once the final rules are issued by the SEC, we will revise, in a timely manner, the current clawback provision of our employment agreements.

34

Grants of Plan-Based Awards. The following table sets forth certain information with respect to plan-based awards granted to the Named Executive Officers in fiscal 2010.

2010 GRANTS OF PLAN-BASED AWARDS

Name	Grant Date	Approval Date (Service Inception Date)		Estimated Possible Payouts Under Non-Equity Incentive Plan Awards						Estimated Possible Payouts Under Equity Incentive Plan Awards						All  Other Stock Awards: Number of  Shares of Stock or Units (#)		All Other Option Awards: Number of Securities Underlying Options (#)		Exercise or  Base Price of Option Awards ($  / Sh)		Grant Date Fair Value of Stock and Option Awards(4)
				Threshold ($)		Target ($)		Maximum ($)		Threshold ($)(1)		Target ($)(1)		Maximum ($)(1)
David A. Levin
-Annual Incentive Plan(2)			4/9/10	$	648,960	$	811,200	$	1,216,800		—		—		—		—		—		—		—
-LTIP(3)			4/9/10	$	324,480	$	405,600	$	608,400	$	324,480	$	405,600	$	608,400		—		—		—		—
-Discretionary Stock Award(4)			3/31/10																347,566	$	4.55	$	361,469
Dennis R. Hernreich
-Annual Incentive Plan(2)			4/9/10	$	497,536	$	621,920	$	932,880		—		—		—		—		—		—		—
-LTIP(3)			4/9/10	$	124,384	$	155,480	$	233,220	$	373,152	$	466,440	$	699,660		—		—		—		—
Robert S. Molloy
-Annual Incentive Plan(2)			4/9/10	$	91,000	$	113,750	$	170,825		—		—		—		—		—		—		—
-LTIP(3)			4/9/10	$	91,000	$	113,750	$	170,825	$	91,000	$	113,750	$	170,825		—		—		—		—
Douglas Hearn
-Annual Incentive Plan(2)			4/9/10	$	77,000	$	96,250	$	144,375		—		—		—		—		—		—		—
-LTIP(3)(5)			4/9/10	$	77,000	$	96,250	$	144,375	$	77,000	$	96,250	$	144,375		—		—		—		—
Henry J. Metscher
-Annual Incentive Plan(2)			4/9/10	$	84,000	$	105,000	$	157,500		—		—		—
-LTIP(3)(5)			4/9/10	$	84,000	$	105,000	$	157,500	$	84,000	$	105,000	$	157,500		—		—		—		—

(1) The LTIP is denominated in dollars, but at the irrevocable election of each participant, the award may be granted in cash, restricted stock, options or any combination thereof. For fiscal 2010, the Compensation Committee required that all participants, through their irrevocable elections, elect to receive at least 50% of any award in equity, through either shares of restricted stock and/or stock options. Accordingly, the estimated possible payout of equity awards under the LTIP is based on potential dollar value of the award pursuant to each Named Executive Officer’s irrevocable election. The actual number of shares/options is determined on the grant date of the award based on the closing price of our common stock on that date. See footnote 3 for additional discussion.

(2) On April 9, 2010, the Compensation Committee approved the annual targets pursuant to our Annual Incentive Plan for fiscal 2010. The threshold, target and maximum payouts for each executive was estimated based on achieving 80%, 100% and 150% of the targets set by the Compensation Committee. On March 16, 2011, the Compensation Committee determined that the performance targets for fiscal 2010 had been achieved at the payout of 107%. The respective cash payment to each of the Named Executive Officers is included in the Summary Compensation Table (and the supplemental Discretionary and Incentive-Based Awards Table) for fiscal 2010.

(3) On April 9, 2010, the Compensation Committee approved the annual targets pursuant to our LTIP for fiscal 2010. Payments under the LTIP are based on each participant’s irrevocable election which permits their respective awards to be payable in cash, restricted stock, options or a combination thereof. Messrs. Levin and Hearn elected to receive 50% of any award in shares of restricted stock with the remaining 50% in cash. Mr. Hernreich elected to receive 75% of any award in shares of restricted stock with the remaining 25% in cash. Messrs. Molloy and Metscher elected to receive 8.3% of any reward in stock options and 41.7% in shares of restricted stock with the remaining 50% in cash. The estimated threshold, target and maximum payouts for each Named Executive Officer above was estimated based on those elections and on achieving 80%, 100% and 150% of the targets set by the Compensation Committee. On March 16, 2011, the Compensation Committee determined that the performance targets for fiscal 2010 had been achieved at a rate of 106%. Accordingly, the actual cash and equity payments awarded pursuant to the LTIP are included in the Summary Compensation Table (and the supplemental Discretionary and Incentive-Based Awards Table) for fiscal 2010.

(4) On March 31, 2010, the Board of Directors granted to Mr. Levin an option to purchase 347,566 shares of our common stock. The exercise price of this option is $4.55 per share and the closing price of our stock on the date of grant was $3.86 per share. The option cliff vested on May 6, 2011 and will expire on May 6, 2012. This grant represented a retention award to Mr. Levin. In accordance with FASB ASC Topic 718, the grant date fair value of the award was determined using a Black-Scholes valuation model. Additional information regarding the assumptions used to estimate the fair value of all stock option awards is included in Note A to Consolidated Financial Statements contained in our Annual Report on Form 10-K for the fiscal year ended January 29, 2011.

(5) Because the employment of Messrs. Hearn and Metscher was terminated prior to any vesting of the award, the full award of $204,050 and $222,600, respectively, was forfeited.

35

Outstanding Equity Awards at Fiscal Year-End. The following table sets forth certain information with respect to outstanding equity awards held by the Named Executive Officers at the end of fiscal 2010.

2010 OUTSTANDING EQUITY AWARDS AT FISCAL YEAR-END

	Option Awards											Stock Awards
	Number of Securities Underlying Unexercised Options (#)		Number of Securities Underlying Unexercised Options (#)			Equity Incentive Plan Awards: Number of Securities Underlying Unexercised Unearned Options (#)(1)		Option Exercise Price ($)		Option Expiration Date		Number of Shares or Units of Stock That Have Not Vested (#)			Market Value of Shares or Units of Stock That Have Not Vested ($)(2)		Equity Incentive Plan Awards: Number of Unearned Shares, Units or Other Rights That Have Not Vested (#)(1)		Equity Incentive Plan Awards: Market or Payout Value of Unearned Shares, Units or Other Rights That Have Not Vested ($)(2)
Name	Exercisable		Unexercisable
David A. Levin		125,000		—				$	3.88		5/25/2011
		75,000		—				$	4.58		5/01/2012
		75,000		—				$	5.01		7/01/2013
		75,000		—				$	6.24		5/24/2014
		—		347,566	(3)			$	4.55		5/06/2012
													152,434	(4)	$	631,077
																		102,610	$	424,805
Dennis R. Hernreich		1,255		—				$	4.58		5/01/2012
		27,520		—				$	5.01		7/01/2013
		21,988		—				$	6.24		5/24/2014
		—		120,000	(3)			$	4.54		5/01/2012
		—		80,000	(3)			$	4.55		5/06/2012
													116,866	(4)	$	483,825
																		118,001	$	488,524
Robert S. Molloy		6,667		3,333	(6)			$	4.13		2/08/2018
				20,606	(4)			$	3.19		3/19/2020
							13,955	$	4.19		3/16/2021
													13,333	(5)	$	55,199
													14,250	(4)	$	58,995
																		23,981	$	99,281
Douglas Hearn		6,667		3,333	(6)			$	4.30		6/13/2018
													13,333	(5)	$	55,199
																		24,349	$	100,805
Henry J. Metscher		10,000		—				$	7.52		10/22/2017
							12,881	$	4.19		3/16/2021
													13,333	(5)	$	55,199
																		22,136	$	91,643

(1) While these options and shares were not actually outstanding at the end of fiscal 2010, in connection with the payout under our LTIP for fiscal 2010, on March 16, 2011 each of the Named Executive Officers was granted options and/or shares of restricted stock in accordance with their respective irrevocable elections. These irrevocable elections required that at least 50% of any award granted be in the form of equity. Each award vests in three tranches with the first tranche vesting on March 16, 2012.

Messrs. Levin and Hearn elected to receive 50% of each tranche in shares of restricted stock and Mr. Hernreich elected to receive 75% of each tranche in shares of restricted stock; therefore, one-third of each executive’s equity award will vest each year with the first one-third vesting on March 16, 2012. Due to Mr. Hearn’s termination of employment, his award was forfeited.

For the first tranche, that will vest March 16, 2012, Messrs. Molloy and Metscher each elected to receive 25% in stock options and 25% in shares of restricted stock. Accordingly, for Messrs. Molloy and Metscher stock options of 13,955 and 12,881, respectively, and 4,796 and 4,428 shares of restricted stock, respectively, will vest on March 16, 2012. For the second and third tranches, Messrs. Molloy and Metscher elected to receive 50% of their awards in shares of restricted stock. Accordingly, 19,184 and 17,708 shares of restricted stock for Messrs. Molloy and Metscher, respectively, will vest in two equal tranches on March 16, 2013 and March 16, 2014. Due to Mr. Metscher’s termination of employment, his award was forfeited.

(2) The value of shares was calculated using the closing price of our common stock of $4.14 on January 28, 2011.

(3) These options fully vest on May 1, 2011.

(4) These equity awards represent awards granted on March 19, 2010 in connection with the payout under our LTIP for fiscal 2009 and were in accordance with each executive’s irrevocable election. Each award vests in three tranches with the first tranche vesting on the first anniversary of the grant date. Mr. Levin elected to receive 50% of each tranche in shares of restricted stock; therefore, one-third of his award vests each year with the first one-third vesting on March 19, 2011. Mr. Hernreich elected to receive 50% of his second tranche and 100% of his third tranche in shares of restricted stock; therefore, one-third of his award will vest on the second anniversary of the date of grant, March 19, 2012, and the remaining two-thirds will vest on the third anniversary of the date of grant, March 19, 2013. Mr. Molloy elected to receive 25% of the first

36

tranche in stock options and 25% of the second and third tranches in shares of restricted stock; therefore, the 20,606 stock options fully vested on the first anniversary of the date of grant, March 19, 2011, and 50% of the 14,250 shares of restricted stock will vest on March 19, 2012 with the remaining 50% vesting on March 19, 2013.

(5) These shares of restricted stock were granted on May 13, 2009. The remaining shares vest ratably over the next two years, with 6,666 of the shares vesting on May 13, 2011 and 6,667 shares vesting on May 13, 2012. Messrs. Hearn and Metscher were entitled to the restricted shares that vested on May 13, 2011; however, due to their employment terminations, 6,667 shares of restricted stock will be forfeited by each.

(6) Mr. Molloy’s stock options vest ratably over three years, commencing on February 8, 2009, the one-year anniversary from the date of grant. Mr. Hearn’s stock options vest ratably over three years, commencing on June 13, 2009, the one-year anniversary from the date of grant. Due to his employment termination, Mr. Hearn forfeited the 3,333 unvested stock options.

Option Exercises and Stock Vested Table. The following table sets forth information for the Named Executive Officers with respect to the exercise of option awards and the vesting of stock awards during fiscal 2010.

2010 OPTION EXERCISES AND STOCK VESTED

Name	Option Awards				Stock Awards
	Number of Shares Acquired  on Exercise (#)		Value Realized on Exercise ($)(1)		Number of Shares Acquired  on Vesting (#)		Value Realized on Vesting ($)(1)
David A. Levin		225,000	$	452,813		—		—
Dennis R. Hernreich		5,000	$	12,275		—		—
Robert S. Molloy		—		—		6,667	$	26,935
Douglas Hearn		—		—		6,667	$	26,935
Henry J. Metscher		—		—		6,667	$	26,935

(1) Represents the difference between the exercise price and fair market value of the common stock on the date of exercise.

Pension Benefits

As part of the Casual Male acquisition, we assumed Casual Male Corp.’s obligations under its Pension Plan and its Supplemental Plan. The following table shows the aggregate annual benefits payable under both the Pension Plan and Supplemental Plan to persons in specified compensation and years of service classifications, based on a straight line annuity form of retirement income:

Average of Highest Five Years of Compensation	Representative Years of Service
	10yrs		20yrs		30yrs (maximum)
$  50,000	$	4,977	$	9,953	$	14,930
100,000		11,977		23,953		35,930
150,000		18,977		37,953		56,930
200,000		25,977		51,953		77,930
250,000*		32,977		65,953		98,930
267,326*		35,402		70,805		106,207
300,000*		39,977		79,953		119,930

(*) The maximum compensation that may be used as of December 31, 1998 to calculate benefits under the Pension and Supplemental Plans is $267,326.

In December 1993, Casual Male Corp. established the Supplemental Plan to provide benefits attributable to compensation in excess of the qualified plan limit, which was $160,000 but less than $267,326. The benefits provided by the Pension Plan and the Supplemental Plan are equal to (i) the sum of 0.75% of the executive’s highest consecutive five year average annual compensation plus 0.65% of the excess of the executive’s highest consecutive five year average annual compensation over the average of the Social Security taxable wage bases, multiplied by (ii) the executive’s years of “benefit service” with Casual Male Corp. (not to exceed 30 years).

37

Effective February 1, 1995, compensation for such purposes was defined as all compensation reported on Form W-2 (excluding such items as bonus and stock options) up to a maximum of $267,326 for the calendar year ended December 31, 1998. Annual benefits are payable under the Pension Plan and Supplemental Plan for retirees at age 65, prior to the offset, if any, for benefits accrued under the retirement plan of Casual Male Corp. and for Social Security benefits.

On May 3, 1997, Casual Male Corp.’s board of directors voted to amend the Pension Plan to cease all benefits accruals under the Pension Plan as of that date. Effective December 31, 1998, the Casual Male Corp.’s board of directors voted to cease all benefit accruals under its Supplemental Plan.

None of our Named Executive Officers was a participant in the plan and, therefore, have no accumulated benefits.

EQUITY COMPENSATION PLAN INFORMATION

The following is a summary of information with respect to our equity compensation plans as of January 29, 2011:

Plan category	Number of securities to be issued upon exercise of outstanding options, warrants and rights (a)		Weighted-average exercise price of outstanding options, warrants and rights (b)		Number of securities remaining available for future issuance under equity compensation plans (excluding securities reflected in column (a))(2) (c)
Equity compensation plans approved by security holders(1)		2,939,097	$	5.42		3,736,329	(3),(4)
Equity compensation plans not approved by security holders(2)		730,000	$	3.63		500,000
Total		3,669,097	$	5.06		4,236,329	(3),(4)

(1) During fiscal 2010, we had one stockholder-approved equity compensation plan, our 2006 Plan. Pursuant to the 2006 Plan, at January 29, 2011, we had a total of 3.7 million shares authorized for issuance of which 2.8 million shares remained available for future issuances. At January 29, 2011, we had 1.0 million options outstanding. We also had 1.9 million options still outstanding as part of our 1992 Stock Incentive Plan (the “1992 Plan”). No further grants can be made under the 1992 Plan.

(2) Prior to 2006, our Board of Directors and the Compensation Committee had the reasonable discretion to issue shares outside of the 1992 Plan, subject to the rules of Nasdaq. These outstanding options to purchase up to 730,000 shares of our common stock, with exercise prices ranging from $3.15 to $5.01 per share, were granted between fiscal 2001 and fiscal 2002. These options represent grants to consultants and also options granted to our executives that were in excess of our plans’ annual maximums during those fiscal years. At January 29, 2011, all of these options were exercisable. In addition, pursuant to our Non-Employee Director Compensation Plan, we have a total 500,000 shares available for stock issuances in lieu of cash director fees, of which all remained available at January 29, 2011.

(3) Our 2006 Plan includes a limitation of 3,750,000 shares that may be issued in the form of restricted stock, deferred stock, performance awards or other stock-based awards. At January 29, 2011, 2,779,501 shares remained subject to this limitation. Subsequent to January 29, 2011, on March 16, 2011, we granted a total of 538,661 shares of unvested restricted stock to certain members of our executive group in connection with achieving targets under our LTIP for fiscal 2010.

(4) Subsequent to January 29, 2011, on March 16, 2011, we granted options to purchase 72,576 shares of common stock to certain members of our executive group in connection with achieving targets under our LTIP for fiscal 2010.

38

401(k) Plans

We have two defined contribution plans, the Casual Male Retail Group, Inc. 401(k) Salaried Savings Plan and the Casual Male Retail Group, Inc. 401(k) Hourly Savings Plan (the “401(k) Plans”). The 401(k) Plans cover all eligible employees who are at least 21 years of age and have completed the required months of service, which is six months for the 401(k) Salaried Savings Plan and 1000 hours or twelve months for the 401(k) Hourly Savings Plan. Effective January 1, 2008, we adopted the safe harbor matching and vesting provisions permitted under the Pension Protection Act of 2006 for its 401(k) Plans. Accordingly, beginning in fiscal 2008, we matched 100% of the first 1% of deferred compensation and 50% of the next 5% (with a maximum contribution of 3.5% of eligible compensation).

In May 2009, in connection with our cost reduction initiatives, we ceased any further employer contributions to the 401(k) Plans, effective May 31, 2009. In November 2010, the administrative committee voted to reinstate the “safe harbor” matching and vesting provision effective January 1, 2011.

Key Man Insurance

We have key man life insurance policies on the lives of Messrs. Levin and Hernreich, each in the amount of $2,000,000.

Certain Relationships and Related Transactions

Seymour Holtzman/Jewelcor Management, Inc.

During fiscal 2010, Seymour Holtzman, our Chairman of the Board of Directors, was compensated by us both directly (as an employee of our Company) and indirectly (as the president and chief executive officer and, together with his wife, indirectly, the majority shareholder of JMI). A complete summary of all compensation and consulting fees paid to Mr. Holtzman is described above under “Corporate Governance—2010 Director Compensation Table” and “Chairman Compensation.”

Review, Approval or Ratification of Transactions with Related Persons

Through JMI, Mr. Holtzman receives consulting compensation from the Company pursuant to a consulting agreement originally entered into with JMI in October 1999 and most recently amended April 2011. Due to Mr. Holtzman’s role as our Chairman of the Board of Directors and the relevance of the services he provides on a consulting basis, our Compensation Committee has had the primary responsibility for reviewing and approving all amendments to the consulting agreement since March 2000. Our Audit Committee Charter, which was subsequently adopted in June 2003, provides that our Audit Committee shall review all related party transactions on an ongoing basis and all such transactions must be approved by the Audit Committee, to the extent required by the Sarbanes—Oxley Act of 2002, the SEC and Nasdaq. Because Nasdaq provides that such oversight can be conducted by either a company’s audit committee or another independent body of the board of directors, the Audit Committee determined that due to the nature of the consulting agreement with JMI the review and approval of all transactions pursuant to this arrangement should continue to be the primary responsibility of the Compensation Committee. Over the years, the Compensation Committee, at its discretion, has at times taken its recommendations, as they relate to the JMI consulting agreement, to the Board of Directors for ratification (without participation from Seymour Holtzman). For all other related party transactions, the review and approval of such transactions is the responsibility of our Audit Committee.

39

Section 16(a) Beneficial Ownership Reporting Compliance

Section 16(a) of the Exchange Act, requires our executive officers and directors, and persons who own more than 10% of a registered class of our equity securities (collectively, the “Reporting Persons”), to file reports of ownership and changes in ownership with the SEC. The Reporting Persons are required to furnish us with copies of all Section 16(a) reports they file. Based solely upon a review of Forms 3, 4 and 5 and amendments thereto furnished to us during fiscal 2010, we believe that the current Reporting Persons complied with all applicable Section 16(a) reporting requirements and that all required reports were filed in a timely manner, with the following exception. In April 2010, Mr. Levin filed a late Form 4 for stock compensation received in connection with our 2009 LTIP on March 19, 2010.

40

PROPOSAL 2

ADVISORY VOTE ON NAMED EXECUTIVE OFFICER COMPENSATION

The Dodd-Frank Wall Street Reform and Consumer Protection Act added Section 14A to the Exchange Act, which requires that we provide our stockholders with the opportunity to vote to approve, on a non-binding, advisory basis, the compensation of our Named Executive Officers as disclosed in this Proxy Statement in accordance with the compensation disclosure rules of the SEC.

We seek to align the interests of our named executive officers with the interests of our stockholders. Our Compensation Committee’s overarching compensation guiding principle is to reward our executives for the achievement of our primary business objectives: to grow our market share within the Big & Tall retail industry, to increase earnings and operating margins and, ultimately, to increase stockholder returns through increased stock price.

This total direct compensation is directly in line with our total stockholder return over the past two fiscal years. Similar to most public companies, we were negatively impacted by the volatile economic environment during fiscal 2008. Over the past two years, we have taken the steps necessary to be profitable and cash flow positive given the weakened economy. This financial performance is a key component of our executive compensation for fiscal 2010. The following chart is a summary of our financial performance over the past two years:

	Fiscal 2008			Fiscal 2009			Fiscal 2010
Total indebtedness at fiscal year-end (in millions)	$	51.2		$	11.1			—
Net income (in millions)	$	(109.3	)(1)	$	6.1		$	15.4
Adjusted EBITDA, a non-GAAP measure(2) (in millions)	$	(59.9	)	$	23.5		$	29.5
Operating Margin Percentage		(17.3	)%		2.0	%		4.1	%
Earnings Per Share—diluted	$	(2.64	)	$	0.14		$	0.32
Stock Price at fiscal year end(3)	$	0.38		$	2.80		$	4.14
Total CMRG Shareholder Return (“TSR”)(4)		(92.4	)%		636.8	%		47.9	%
S&P 500 Composite Retail Index(5)		(39.0	)%		53.7	%		25.5	%

(1) In fiscal 2008, as a result of the decreases in our market capitalization caused by the volatility in the financial markets, we recorded substantial non-cash charges totaling $100.0 million, or $2.42 per diluted share, related primarily to the impairment of goodwill and trademarks as well as a valuation allowance against our deferred tax assets.

(2) Adjusted EBITDA is a non-GAAP measure and is defined as income from continuing operations before interest, taxes, depreciation and amortization and is adjusted each period to exclude other income.

(3) Represents the closing market price of our common stock on the last business day of the respective fiscal year.

(4) TSR is calculated as the change in stock price over the preceding fiscal year.

(5) Represents the annual return percentage of the S&P 500 Composite Retail, a broad market index.

The “Executive Compensation” section of this Proxy Statement, including the “Compensation Discussion and Analysis” section, describes in detail our executive compensation programs and the decisions made by the Compensation Committee with respect to fiscal 2010. Highlights from our executive compensation program include the following:

• Performance is a key component of our philosophy for executive compensation, representing approximately 67% of our Key Executives’ annual target total direct compensation and approximately 51% of our Senior Executives’ annual target direct compensation;

• The performance-based compensation for fiscal 2010 consisted of:

• an annual cash incentive based on achievement of a specified performance goal for fiscal 2010

41

• a long-term incentive which is also based on achievement of specified performance goals for fiscal 2010, but which vests ratably over a three-year period starting on the first anniversary of the date of grant; and,

• Based on the Company’s financial performance for fiscal 2010, the Named Executive Officers earned an annual cash incentive payout of approximately 107% of their target and a long-term incentive payout of approximately 106% of target.

• For fiscal 2010, Mr. Levin, President and Chief Executive Officer, earned total compensation of $2.9 million, of which $1.7 million was performance-based compensation. Of the $1.7 million in performance-based compensation, approximately $0.9 million is subject to a three-year vesting. Mr. Levin’s total compensation for fiscal 2010 of $2.9 million decreased 1.2% from total compensation earned in fiscal 2009 of $3.0 million. (Mr. Levin’s compensation for fiscal 2010 includes a retention award of $361,469. This amount represents the fair value of an option to purchase 347,566 shares of our common stock, which was granted to Mr. Levin on March 31, 2010. The exercise price of the option is $4.55 per share and the closing price of our stock on the date of grant was $3.86 per share. The option became fully vested on May 6, 2011 and will expire on May 6, 2012).

• For fiscal 2010, Mr. Hernreich, Executive Vice President, Chief Operating Officer, Chief Financial Officer, Treasurer and Secretary, earned total compensation of $2.0 million, of which $1.3 million was performance-based compensation. Of the $1.3 million in performance-based compensation, approximately $0.7 million is subject to a three-year vesting. Mr. Hernreich’s total compensation for fiscal 2010 of $2.0 million decreased 13.5% from total compensation earned in fiscal 2009 of $2.3 million.

• Given his widespread expertise and the multiple roles that Mr. Hernreich fulfills, including but not limited to, serving as our Chief Financial Officer as well as Chief Operating Officer, the Compensation Committee believes that Mr. Hernreich’s compensation is appropriate and the expense to the Company is far below the expense which would be incurred by adding additional executives to fill the many roles filled by Mr. Hernreich.

• Unlike most of our peers, we do not have a Chief Merchandising Officer and a separate Chief Financial Officer. The concentration of these key responsibilities reside with Messrs. Levin and Hernreich.

• For fiscal 2010, the three other Named Executive Officers earned, in aggregate, a total of $1.9 million, of which $1.0 million was performance-based compensation. Of the $1.0 million in performance-based compensation, approximately $0.7 million is subject to a three-year vesting. Because of the departure of two of the three Named Executive Officers, $0.4 million of fiscal 2010 long-term awards was forfeited.

We encourage you to read our “Compensation Discussion and Analysis” for a complete discussion and analysis of our executive compensation program, including detailed information about the fiscal 2010 compensation of the Named Executive Officers.

Our Board is asking stockholders to approve the compensation of our Named Executive Officers as disclosed pursuant to the SEC’s compensation disclosure rules (which includes the Compensation and Discussion Analysis, the compensation tables and the narrative disclosures). As an advisory vote, this proposal is not binding upon the Company. The Compensation Committee values the opinions expressed by our stockholders in their vote on this proposal and will consider the outcome of the vote when making future compensation decisions for named executive officers. Accordingly, we ask our stockholders to vote on the following resolution at the Annual Meeting:

“RESOLVED, that the compensation paid to the Company’s Named Executive Officers, as disclosed in this Proxy Statement pursuant to the compensation rules of the SEC, including Compensation Discussion and Analysis, compensation tables and narrative discussion, is hereby APPROVED.”

Our Board of Directors recommends that you vote FOR the approval of the compensation of our Named Executive Officers as disclosed in this Proxy Statement.

42

PROPOSAL 3

ADVISORY VOTE ON THE FREQUENCY OF HOLDING

ADVISORY VOTES ON NAMED EXECUTIVE OFFICER COMPENSATION

The Dodd-Frank Wall Street Reform and Consumer Protection Act added Section 14A to the Exchange Act, which requires that we provide our stockholders with the opportunity to vote, on a non-binding, advisory basis, for their preference as to how frequently to vote on future advisory votes on the compensation of our named executive officers as disclosed in accordance with the disclosure rules of the SEC.

Shareholders may indicate whether they would prefer that we conduct future advisory votes on named executive officer compensation once every one, two, or three years. Stockholders also may abstain from casting a vote on this proposal.

The Board of Directors has determined that an advisory vote on named executive officer compensation that occurs each year is the most appropriate alternative for the Company and therefore the Board recommends that you vote for the “one year” option with respect to the advisory vote on named executive officer compensation. The Board of Directors believes that an annual advisory vote on named executive officer compensation is the optimal interval for conducting and responding to a “Say-on-Pay” vote. The Board of Directors also believes this frequency is in alignment with our executive compensation practices, as we review the core elements of our executive compensation program annually. By providing an advisory vote on named executive officer compensation on an annual basis, our stockholders will be able to provide us with direct input on our compensation philosophy, policies and practices as disclosed in the Proxy Statement every year.

The proxy card provides stockholders with the opportunity to choose among four options (holding the vote every one, two or three years, or abstain from voting) and, therefore, stockholders will not be voting to approve or disapprove the recommendation of the Board of Directors. Generally, the affirmative vote of the majority of the votes cast is necessary to approve any matter being submitted to stockholders at the Annual Meeting. However, because this vote is advisory and non-binding, if none of the three frequency options receives the vote of the holders of a majority of the votes cast, the Company will consider the frequency option (one year, two years or three years) receiving the highest number of votes cast by stockholders to be the frequency that has been recommended by stockholders. In addition, because this vote is advisory and non-binding, the Board of Directors may decide that it is in the best interest of our stockholders and the Company to hold future advisory votes on named executive officer compensation more or less frequently.

The Board of Directors recommends that you vote for the option of “one-year” as the preferred frequency for advisory votes on named executive officer compensation.

43

PROPOSAL 4

RATIFICATION OF APPOINTMENT OF

INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

Subject to ratification by our stockholders, the Audit Committee has appointed Ernst & Young LLP as our independent registered public accounting firm for the fiscal year ending January 28, 2012. The agreed upon audit fee for the fiscal year ending January 28, 2012 is approximately $450,000 plus reasonable out-of-pocket expenses. Ernst & Young LLP has served as our auditors since October 2000. Stockholder ratification of our independent registered public accounting firm is not required by our bylaws or otherwise. However, we are submitting the selection of Ernst & Young LLP to our stockholders for ratification as a matter of good corporate practice. If stockholders fail to ratify the appointment of such auditors, the Audit Committee will reconsider the selection. Even if the selection is ratified, the Audit Committee may, in its discretion, direct the appointment of a different independent auditor at any time during the year if it determines that such a change would be in our best interest.

Audit Fees

Ernst & Young LLP billed us an aggregate of $629,000 and $638,750 in fees for fiscal years 2010 and 2009, respectively, for professional services rendered in connection with the audits of our financial statements for the fiscal years ended January 29, 2011 and January 30, 2010 included in our Annual Reports on Form 10-K, for services performed related to compliance with Section 404 of the Sarbanes-Oxley Act of 2002 and the reviews of the financial statements included in each of our Quarterly Reports on Form 10-Q. Fiscal 2009 also included $10,000 in fees related to registration statements.

Audit—Related Fees

For fiscal 2009, Ernst & Young billed $40,000 in fees related to risk management. No such fees were billed by Ernst & Young for fiscal 2010.

Tax Fees

No fees were billed by Ernst & Young for fiscal 2010 and 2009 for tax services.

All Other Fees

For fiscal 2010 and fiscal 2009, Ernst & Young billed us $1,970 and $1,940, respectively, for fees related to an online accounting research tool.

Pre-Approval of Services by Independent Auditors

The Audit Committee has adopted a policy governing the provision of audit and non-audit services by our independent registered public accounting firm. Pursuant to this policy, the Audit Committee will consider annually and, if appropriate, approve the provision of audit services (including audit review and attest services) by its independent registered public accounting firm and consider and, if appropriate, pre-approve the provision of certain defined permitted non-audit services within a specified dollar limit. It will also consider on a case-by-case basis and, if appropriate, approve specific engagements that do not fit within the definition of pre-approved services or established fee limits.

The policy provides that any proposed engagement that does not fit within the definition of a pre-approved service or is not within the fee limits must be presented to the Audit Committee for consideration at its next regular meeting or to the Chairman of the Audit Committee in time sensitive cases. The Audit Committee will regularly review summary reports detailing all services (and related fees and expenses) being provided to us by the independent registered public accounting firm.

44

All of the services provided under Audit Fees, Audit-Related Fees, Tax Fees and All Other Fees are approved by the Audit Committee.

Representatives of Ernst & Young LLP are expected to be available by telephone at the Annual Meeting and will have the opportunity to make a statement if they desire to do so and to respond to appropriate questions.

Vote Needed for Approval

The affirmative vote of a majority of the votes properly cast is required for the ratification of the appointment of Ernst & Young LLP as our independent registered public accounting firm. Any abstentions and broker non-votes will not be counted as votes cast on this proposal and, accordingly, will have no effect.

Recommendation

The Audit Committee and the Board of Directors recommend that you vote FOR the ratification of the appointment of Ernst & Young LLP as our independent registered public accounting firm.

45

REPORT OF THE AUDIT COMMITTEE

The Audit Committee oversees our financial reporting process on behalf of the Board of Directors. Our management has the primary responsibility for the financial statements, for maintaining effective internal control over financial reporting and for assessing the effectiveness of internal control over financial reporting. In fulfilling its oversight responsibilities, the Audit Committee reviewed and discussed the audited consolidated financial statements and related schedule in the Annual Report with management, including a discussion of the quality, not just the acceptability, of the accounting principles, the reasonableness of significant judgments, and the clarity of disclosures in the financial statements. The Audit Committee also oversees the review and assessment process of our internal control over financial reporting, including the framework used to evaluate the effectiveness of such internal controls.

The Audit Committee reviewed and discussed with Ernst & Young LLP, our independent registered public accounting firm, which is responsible for expressing an opinion on the conformity of our audited consolidated financial statements for the fiscal year ended January 29, 2011 with U.S. generally accepted accounting principles, their judgments as to the quality, not just the acceptability, of our accounting principles and such other matters as are required to be discussed with the Audit Committee by Statement on Auditing Standards No. 61, as amended by Statement on Auditing Standards No. 90 (Communication with Audit Committees), as adopted by the Public Company Accounting Oversight Board. In addition, the Committee has discussed with Ernst & Young LLP the firm’s independence from our management and our Company, including the matters in the letter from Ernst & Young LLP required by applicable requirements of the Public Company Accounting Oversight Board regarding Ernst & Young LLP’s communications with the Audit Committee concerning independence. The Audit Committee also considered the compatibility of non-audit services with Ernst & Young LLP’s independence.

The Audit Committee discussed with Ernst & Young LLP the overall scope and plans for their audit. The Audit Committee meets with the independent registered public accounting firm, with and without management present, to discuss the results of their examinations, their evaluations of our internal controls (including internal control over financial reporting) and the overall quality of our financial reporting. The Audit Committee held 6 meetings during the fiscal year ended January 29, 2011.

In reliance on the reviews and discussions referred to above, the Audit Committee recommended to the Board of Directors (and the Board has approved) that the audited consolidated financial statements be included in the Annual Report on Form 10-K for the fiscal year ended January 29, 2011 for filing with the SEC.

Pursuant to Section 404 of the Sarbanes-Oxley Act, management was required to prepare as part of our Annual Report on Form 10-K for the year ended January 29, 2011 a report by management on our assessment of our internal control over financial reporting, including management’s assessment of the effectiveness of such internal control. Ernst & Young LLP has issued an audit report relative to our internal control over financial reporting at January 29, 2011. During the course of fiscal 2010, management regularly discussed the internal control review and assessment process with the Audit Committee, including the framework used to evaluate the effectiveness of such internal controls, and at regular intervals updated the Audit Committee on the status of this process and actions taken by management to respond to issues identified during this process. The Audit Committee also discussed this process with Ernst & Young LLP. Management’s assessment report and Ernst & Young LLP’s audit report on our internal control over financial reporting are included as part of our Annual Report on Form 10-K for the year ended January 29, 2011.

The Audit Committee is governed by a written charter, which can be found under “Corporate Governance” of the Investor Relations page of our website at www.casualmaleXL.com. The members of the Audit Committee are considered independent because they satisfy the independence requirements for Board members prescribed by the Nasdaq listing standards and Rule 10A-3 of the Exchange Act.

THE AUDIT COMMITTEE

Jesse Choper, Chairman of the Audit Committee

Alan S. Bernikow

John E. Kyees

46

WHERE YOU CAN FIND MORE INFORMATION

We file annual, quarterly and special reports, proxy statements and other information with the SEC under the Exchange Act. You may read and copy these reports and other information filed by us at the Public Reference Section of the SEC, 100 F Street, N.E., Room 1580, Washington, D.C. 20549, at prescribed rates. You may obtain information on the operation of the Public Reference Room by calling the SEC at 1-800-SEC-0330.

The SEC also maintains a website that contains reports, proxy statements and other information about issuers, like us, who file electronically with the SEC through the Electronic Data Gathering, Analysis and Retrieval (EDGAR) system. The address of this website is http://www.sec.gov.

Access to this information as well as other information on our Company is also available on our website at http://www.casualmaleXL.com and clicking on “Investor Relations.”

SOLICITATION

We will bear the cost of solicitation of proxies. In addition to the use of the mails, proxies may be solicited by certain of our officers, directors and employees without extra compensation, by telephone, facsimile or personal interview. We have retained D.F. King & Company, Inc. for a fee not to exceed $6,000 to aid in solicitation of proxies.

DELIVERY OF DOCUMENTS TO STOCKHOLDERS SHARING AN ADDRESS

Only one copy of the Proxy Statement is being delivered to multiple stockholders sharing an address unless we have received contrary instructions from one or more of the stockholders in question. If you are one of a number of stockholders sharing a single address and would like to receive a separate copy of the Proxy Statement or if you would like to request that we send you a separate copy of annual reports or proxy statements, as applicable, in the future, please contact us at 555 Turnpike Street, Canton Massachusetts 02021. We will send you a copy of the Proxy Statement promptly after we receive your written or oral request.

STOCKHOLDER PROPOSALS

Under the rules of the SEC, in order for any stockholder proposal to be included in our proxy statement and proxy card for presentation at the 2011 Annual Meeting of Stockholders, the proposal must be received by the Secretary of our Company at our principal executive offices by February 22, 2012 (120 days before the anniversary of the date this Proxy Statement is being mailed to our stockholders).

Our By-Laws provide that for business to be properly brought before an Annual Meeting of Stockholders (or any Special Meeting in lieu of Annual Meeting of Stockholders), a stockholder must: (i) give timely written notice to the Secretary of our Company describing any proposal to be brought before such meeting; and (ii) be present at such Annual Meeting, either in person or by a representative. Such procedural requirements are fully set forth in Section 3.14 of our By-Laws. A stockholder’s notice will be timely if delivered to, or mailed to and received by, us not less than 75 days nor more than 120 days prior to the anniversary date of the immediately preceding Annual Meeting (the “Anniversary Date”). To bring an item of business before the 2012 Annual Meeting, a stockholder must deliver the requisite notice of such item to the Secretary of our Company not before April 6, 2012 or after May 21, 2012. In the event the Annual Meeting is scheduled to be held on a date more than 30 days before the Anniversary Date or more than 60 days after the Anniversary Date, however, a stockholder’s notice will be timely delivered to, or mailed to, and received by, us not later than the close of business on the later of (a) the 75th day prior to the scheduled date of such Annual Meeting or (b) the 15th day following the day on which public announcement of the date of such Annual Meeting is first made by us.

47

STOCKHOLDER COMMUNICATIONS WITH THE BOARD OF DIRECTORS

Our Board of Directors maintains a process for stockholders to communicate with them. Stockholders wishing to communicate with our Board should direct their communications to: Secretary of the Company, Casual Male Retail Group, Inc., 555 Turnpike Street, Canton, Massachusetts 02021. Any such communication must state the number of shares beneficially owned by the stockholder sending the communication. The Secretary will forward such communication to all of the members of the Board of Directors or to any individual director or directors to whom the communication is directed; provided, however, that if the communication is unduly hostile, profane, threatening, illegal or otherwise inappropriate, then the Secretary has the authority to discard the communication or take appropriate legal action in response to the communication.

OTHER MATTERS

As of this date, our management knows of no business, which may properly come before the Annual Meeting other than that stated in the Notice of Annual Meeting of Stockholders. Should any other business arise, proxies given in the accompanying form will be voted in accordance with the discretion of the person or persons voting them.

48

CASUAL MALE RETAIL GROUP, INC.

Notice of 2011 Annual Meeting of

Stockholders and Proxy Statement

Thursday, August 4, 2011

9:00 A.M. EDT

Casual Male Retail Group, Inc.

555 Turnpike Street

Canton, Massachusetts 02021

Please sign your proxy and

return it in the enclosed

postage-paid envelope so

that you may be represented

at the Annual Meeting.

CASUAL MALE RETAIL GROUP, INC.

555 Turnpike Street

Canton, Massachusetts 02021

This Proxy Is Solicited On Behalf Of The Board Of Directors

For The Annual Meeting Of Stockholders To Be Held On August 4, 2011

The undersigned stockholder of Casual Male Retail Group, Inc. (the “Company”), hereby appoints Seymour Holtzman and David A. Levin, and each of them, proxies, with full power of substitution to each and to each substitute appointed pursuant to such power, to vote all shares of Common Stock of the Company which the undersigned would be entitled to vote if personally present at the Annual Meeting of Stockholders of the Company to be held on Thursday, August 4, 2011, at 9:00 A.M. local time, at the corporate offices of the Company, 555 Turnpike Street, Canton, Massachusetts, and at any adjournment thereof, with all powers the undersigned would possess if personally present, as set forth on the reverse hereof, upon the matters set forth thereon and more fully described in the Notice and Proxy Statement for such Annual Meeting, and, in their discretion, upon all such other matters as may properly come before the Annual Meeting. The undersigned hereby revokes all proxies, if any, hitherto given by the undersigned for such Annual Meeting.

(Continued and to be signed on the reverse side)

ANNUAL MEETING OF STOCKHOLDERS OF

CASUAL MALE RETAIL GROUP, INC.

August 4, 2011

Important Notice Regarding Internet Availability of Proxy Materials for the

Annual Meeting of Stockholders to be Held on August 4, 2011:

The Proxy Statement and 2011 Annual Report to Stockholders are available at

www.casualmale.com/proxymaterials

Please sign, date and mail

your proxy card in the

envelope provided as soon

as possible.

i   Please detach along perforated line and mail in  the envelope provided.  i

n      20830003003000001000     4	080510
THE BOARD OF DIRECTORS RECOMMENDS A VOTE “FOR” THE ELECTION OF DIRECTORS AND PROPOSALS 2 AND 4. THE BOARD OF DIRECTORS RECOMMENDS “1 YEAR” FOR PROPOSAL 3. PLEASE SIGN, DATE AND RETURN PROMPTLY IN THE ENCLOSED ENVELOPE. PLEASE MARK YOUR VOTE IN BLUE OR BLACK INK AS SHOWN HERE  x

1. Election of Directors: THE BOARD OF DIRECTORS RECOMMENDS A VOTE “FOR” ALL NOMINEES NAMED BELOW.

	NOMINEES:
¨       FOR ALL NOMINEES	O  Seymour Holtzman O  David A. Levin O  Alan S. Bernikow O  Jesse H. Choper O  John E. Kyees O  Ward K. Mooney O  George T. Porter, Jr. O  Mitchell S. Presser
¨        WITHHOLD AUTHORITY FOR ALL NOMINEES
¨       FOR ALL  EXCEPT (See instructions below)

INSTRUCTIONS: To withhold authority to vote for any individual nominee(s), mark “FOR ALL EXCEPT” and fill in the circle next to each nominee you wish to withhold, as shown here:

2. To approve, on an advisory basis, the compensation of the Company’s named executive officers. THE BOARD OF DIRECTORS RECOMMENDS A VOTE “FOR” PROPOSAL 2.

¨    FOR ¨    AGAINST ¨    ABSTAIN

3. To approve, on an advisory basis, the frequency of holding advisory votes on the compensation of the Company’s named executive officers. THE BOARD OF DIRECTORS RECOMMENDS “1 YEAR” FOR PROPOSAL 3.

¨    1 YEAR ¨    2 YEARS ¨    3 YEARS ¨    ABSTAIN

4. Ratification of the appointment of Ernst & Young LLP as the Company’s independent registered public accounting firm for fiscal year ending January 28, 2012. THE BOARD OF DIRECTORS RECOMMENDS A VOTE “FOR” PROPOSAL 4.

¨    FOR ¨    AGAINST ¨    ABSTAIN

IF THIS PROXY IS PROPERLY EXECUTED AND RETURNED IT WILL BE VOTED AS SPECIFIED HEREIN. IF NO SPECIFIC DIRECTION IS GIVEN, IT WILL BE VOTED “FOR” EACH OF THE DIRECTOR NOMINEES; “FOR” PROPOSALS 2 AND 4; AND FOR “1 YEAR” ON PROPOSAL 3. RECEIPT IS HEREBY ACKNOWLEDGED OF THE NOTICE OF ANNUAL MEETING AND PROXY STATEMENT OF CASUAL MALE RETAIL GROUP, INC. DATED JUNE 21, 2011.

l
	MARK “X” HERE IF YOU PLAN TO ATTEND THE MEETING   ¨

To change the address on your account, please check the box at right and indicate your new address in the address space above. Please note that changes to the registered name(s) on the account may not be submitted via this method. ¨

Signature of Shareholder Date:  Signature of Shareholder  Date:

n Note: Please sign exactly as your name or names appear on this Proxy. When shares are held jointly, each holder should sign. When signing as executor, administrator, attorney, trustee or guardian, please give full title as such. If the signer is a corporation, please sign full corporate name by duly authorized officer, giving full title as such. If signer is a partnership, please sign in partnership name by authorized person.