Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

We consent to the reference to our firm under the caption “Experts” in this Registration Statement (Form S-3) and related Prospectus of Delta Apparel, Inc. and subsidiaries for the registration of common stock, preferred stock, debt securities, warrants, and units and to the incorporation by reference therein of our report dated August 29, 2013, with respect to the consolidated statements of operations, comprehensive income, shareholders’ equity, cash flows and related financial schedule of Delta Apparel, Inc. and subsidiaries for the year ended June 29, 2013 included in its Annual Report (Form 10-K) for the year ended October 3, 2015, filed with the Securities and Exchange Commission.

/s/ Ernst & Young LLP

Atlanta, Georgia

November 23, 2016