Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the registration statement (No. 333-174207) on Form S-3 of Rick’s Cabaret International, Inc. of (i) our report dated December 14, 2012, with respect to the consolidated balance sheets of Rick’s Cabaret International, Inc. and subsidiaries, as of September 30, 2012 and 2011, and the related consolidated statements of operations, changes in permanent stockholders’ equity, and cash flows for each of the years in the three-year period ended September 30, 2012, which report appears in the Annual Report on Form 10-K of Rick’s Cabaret International, Inc. for the fiscal year ended September 30, 2012, and (ii) our report dated December 14, 2012, with respect to the effectiveness of internal control over financial reporting for the fiscal year ended September 30, 2012.

/s/ Whitley Penn LLP

Dallas, Texas

December 14, 2012