Exhibit 23

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

We consent to the incorporation by reference in the Registration Statements (Form S-8 Nos. 333-106861 and 333-132925) pertaining to the 2003 and 2005 Stock Incentive Plans of Cavco Industries, Inc. of our reports dated June 11, 2012, with respect to the consolidated financial statements of Cavco Industries, Inc. and subsidiaries and the effectiveness of internal control over financial reporting of Cavco Industries, Inc., included in this Annual Report (Form 10-K) for the year ended March 31, 2012.

/s/ Ernst & Young LLP

Phoenix, Arizona

June 11, 2012