EXHIBIT 23.4

Consent of Independent Auditors

The Board of Directors

GBC Bancorp:

We consent to the incorporation by reference in the registration statement on Form S-4 of Cathay Bancorp, Inc. of our report dated January 17, 2003, relating to the consolidated balance sheets of GBC Bancorp as of December 31, 2002 and 2001, and the related consolidated statements of operations, changes in stockholders’ equity and comprehensive income, and cash flows for each of the years then ended, which report appears in the December 31, 2002 annual report on Form 10-K of GBC Bancorp, Inc. and to the reference to our firm under the heading “Experts” in the joint proxy statement/prospectus.

As discussed in note 1 to the consolidated financial statements, the Company changed its method of accounting for derivative instruments and hedging activities in 2001.

KPMG LLP

/s/    KPMG LLP

Los Angeles, California

June 25, 2003