EXHIBIT 23.1

Independent Auditors’ Consent

The Board of Directors

Charter Financial Corporation:

We consent to the incorporation by reference in the registration statements (No. 333-67402 and No. 333-98161) on Form S-8 of Charter Financial Corporation and subsidiaries of our report dated November 15, 2002, with respect to the consolidated balance sheets of Charter Financial Corporation and subsidiaries as of September 30, 2002 and 2001, and the related consolidated statements of income, stockholders’ equity and comprehensive income (loss), and cash flows, for each of the years in the three-year period ended September 30, 2002 which report appears in the September 30, 2002, annual report on Form 10-K of Charter Financial Corporation.

                                                                                                                            /s/    KPMG LLP

December 26, 2002