Exhibit 23

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement on Form S-8 (File No. 333-171813) pertaining to the Harley-Davidson, Inc. Employee Stock Purchase Plan of our report dated March 28, 2012, with respect to the financial statements of the Harley-Davidson, Inc. Employee Stock Purchase Plan included in this Annual Report (Form 11-K) for the period February 14, 2011 to December 31, 2011 and the period then ended.

/s/ Ernst & Young LLP

Milwaukee, Wisconsin

March 28, 2012