Exhibit 23.1

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in Registration Statement Nos. 333-81741, 333-114933, 333-133846, and 333-152043 on Form S-8 of our reports dated April 13, 2010, relating to the consolidated financial statements of Perry Ellis International, Inc., and the effectiveness of Perry Ellis International Inc.’s internal control over financial reporting, appearing in this Annual Report on Form 10-K of Perry Ellis International, Inc. for the year ended January 30, 2010.

/s/ Deloitte & Touche LLP

Miami, Florida

April 13, 2010