Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement (Form S-8 No. 333-131259) pertaining to the Caribou Coffee Company, Inc. 1994 Stock Awards Plan, 2001 Stock Incentive Plan, and 2005 Equity Incentive Plan, in the Registration Statement (Form S-8 No. 333-174237) pertaining to the Caribou Coffee Company, Inc. Amended and Restated Equity Incentive Plan and in the Registration Statement (Form S-3 No. 333-170634) of our reports dated March 16, 2012, with respect to the consolidated financial statements and schedule of Caribou Coffee Company, Inc. and Affiliates and the effectiveness of internal control over financial reporting of Caribou Coffee Company, Inc. and Affiliates, included in this Annual Report (Form 10-K) for the year ended January 1, 2012.

/s/ Ernst & Young, LLP

Minneapolis, Minnesota

March 16, 2012