Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

The Board of Directors

Ruth’s Hospitality Group, Inc.:

We consent to the incorporation by reference in the registration statements on Form S-3 (No. 333-171485 and 333-160231) and Form S-8 (No. 333-127681) of Ruth’s Hospitality Group, Inc. of our reports dated March 2, 2012 with respect to the consolidated balance sheets of Ruth’s Hospitality Group, Inc. and subsidiaries as of December 26, 2010 and December 25, 2011, and the related consolidated statements of income, stockholders’ equity and cash flows for the fifty-two weeks ended December 27, 2009, December 26, 2010, and December 25, 2011, and the effectiveness of internal control over financial reporting as of December 25, 2011, which reports appear in the December 25, 2011 annual report on Form 10-K of Ruth’s Hospitality Group, Inc.

/s/ KPMG LLP

Orlando, Florida

March 2, 2012

Certified Public Accountants