Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

The Board of Directors

Ruth’s Hospitality Group, Inc.:

We consent to the incorporation by reference in the registration statements on Forms S-3 (No. 333-171485 and 333-160231) and Forms S-8 (No. 333-127681, 333-182845 and 333-196836) of Ruth’s Hospitality Group, Inc. of our reports dated March 4, 2016 with respect to the consolidated balance sheets of Ruth’s Hospitality Group, Inc. and subsidiaries as of December 27, 2015 and December 28, 2014, and the related consolidated statements of income, shareholders’ equity and cash flows for each of the years in the three-year period ended December 27, 2015, and the effectiveness of internal control over financial reporting as of December 27, 2015, which reports appear in the December 27, 2015 annual report on Form 10-K of Ruth’s Hospitality Group, Inc.

Our report refers to a change in the method of accounting for the recognition of gift card breakage income.

/s/ KPMG LLP

Orlando, Florida

March 4, 2016

Certified Public Accountants