EXHIBIT 23.1

CONSENT OF ERNST & YOUNG LLP, INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

The Board of Directors
Delta Apparel, Inc.

We consent to the incorporation by reference in the Registration Statement (Form S-8 No. 333-61190) pertaining to the Delta Apparel, Inc. 2001 Stock Option Plan and Delta Apparel, Inc. 2001 Incentive Stock Award Plan of our report dated August 26, 2005, with respect to the consolidated financial statements and schedule of Delta Apparel, Inc. included in the Annual Report on Form 10-K for the year ended July 2, 2005.

/s/ Ernst & Young LLP  

Atlanta, Georgia
September 21, 2005