Exhibit 23.1

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in Registration Statements Nos. 333-81741, 333-114933, 333-133846, and 333-152043 on Forms S-8 of our reports dated April 9, 2009, relating to the consolidated financial statements of Perry Ellis International, Inc., and management’s report on the effectiveness of internal control over financial reporting, appearing in this Annual Report on Form 10-K of Perry Ellis International, Inc. for the year ended January 31, 2009.

/s/ Deloitte & Touche LLP

Miami, Florida

April 9, 2009