Exhibit 23

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement (Form S-8 Nos. 333-81606, 333-81608, 333-70315, and 333-55140) pertaining to the Luby's, Inc. Amended and Restated Nonemployee Director Stock Option Plan, miscellaneous employee benefit plans, Luby's Incentive Stock Plan, and Luby's, Inc. Nonemployee Director Phantom Stock Plan of our report dated October 13, 2004, with respect to the consolidated financial statements of Luby's, Inc. included in the Annual Report (Form 10-K) for the year ended August 25, 2004.

/s/Ernst & Young

San Antonio, Texas

November 5, 2004