Exhibit 23

Consent of Independent Auditors

We consent to the incorporation by reference in the following Registration Statements and Post-Effective Amendments of our report dated February 25, 2004, with respect to the consolidated financial statements of Oglebay Norton Company included in this Annual Report (Form 10-K) for the year ended December 31, 2003:

Registration Statement Number 33-58819 on Form S-8 dated April 26, 1995, pertaining to the Oglebay Norton Company Director Stock Plan;

Post-Effective Amendment Number 1 to Registration Statement Number 33-58819 on Form S-8 dated February 22, 2000, pertaining to the Oglebay Norton Company Director Stock Plan;

Post-Effective Amendment Number 2 to Registration Statement Number 33-58819 on Form S-8 dated May 11, 2001, pertaining to the Oglebay Norton Company Director Stock Plan;

Registration Statement Number 333-02781 on Form S-8 dated April 24, 1996, pertaining to the Oglebay Norton Company Long-Term Incentive Plan;

Post-Effective Amendment Number 1 to Registration Statement Number 333-02781 on Form S-8 dated August 3, 1999, pertaining to the Oglebay Norton Company Long-Term Incentive Plan;

Post-Effective Amendment Number 2 to Registration Statement Number 333-02781 on Form S-8 dated May 11, 2001, pertaining to the Oglebay Norton Company Long-Term Incentive Plan;

Registration Statement Number 333-84243 on Form S-8 dated August 2, 1999, pertaining to the Oglebay Norton Company 1999 Long-Term Incentive Plan and Option Agreements with Two Executive Officers;

Post-Effective Amendment Number 1 to Registration Statement Number 333-84243 on Form S-8 dated May 11, 2001, pertaining to the Oglebay Norton Company 1999 Long-Term Incentive Plan and Option Agreements with Two Executive Officers;

Registration Statement Number 333-75225 on Form S-8 dated March 29, 1999, pertaining to the Oglebay Norton Company Performance Option Agreement;

Post-Effective Amendment Number 1 to Registration Statement Number 333-75225 on Form S-8 dated May 11, 2001, pertaining to the Oglebay Norton Company Performance Option Agreement;

Registration Statement Number 333-75223 on Form S-8 dated March 29, 1999, pertaining to the Oglebay Norton Company Director Fee Deferral Plan;

Post-Effective Amendment Number 1 to Registration Statement Number 333-75223 on Form S-8 dated May 11, 2001, pertaining to the Oglebay Norton Company Director Fee Deferral Plan;

Registration Statement Number 333-75221 on Form S-8 dated March 29, 1999, pertaining to the Oglebay Norton Incentive Savings and Stock Ownership Plan;

Post-Effective Amendment Number 1 to Registration Statement Number 333-75221 on Form S-8 dated May 11, 2001, pertaining to the Oglebay Norton Incentive Savings and Stock Ownership Plan;

Registration Statement Number 333-60844 on Form S-8 dated May 14, 2001, pertaining to the Oglebay Norton Incentive Savings and Stock Ownership Plan;

Registration Statement Number 333-53534 on Form S-4 dated January 11, 2001, pertaining to ON Minerals Company, Inc. (now known as Oglebay Norton Company, an Ohio corporation);

Amendment Number 1 to Registration Statement Number 333-53534 on Form S-4 dated March 8, 2001, pertaining to ON Minerals Company, Inc.;

Registration Statement Number 333-98157 on Form S-8 dated August 15, 2002, pertaining to the Oglebay Norton Company 2002 Stock Option Plan; and

Registration Statement on Form S-8 dated February 28, 2003, pertaining to the Restricted Stock Award to John N. Lauer.

/S/    ERNST & YOUNG LLP

Cleveland, Ohio

March 26, 2004