Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement (Form S-8 No. 333-81741) pertaining to the 1993 Stock Option Plan, as Amended, of Perry Ellis International, Inc. of our report dated November 19, 2004, except for Note 16, as to which the date is May 9, 2005, with respect to the consolidated financial statements of Tropical Sportswear Int’l Corporation for the year ended October 2, 2004.

Tampa, Florida May 9, 2005 /s/ Ernst & Young LLP Certified Public Accountants