Exhibit 23.2

CONSENT OF INDEPENDENT REGISTERED PUBLIC ACCOUNTING FIRM

Board of Directors

Charter Financial Corporation

West Point, Georgia

We consent to the use of our report dated December 23, 2009, except for Note 22 as to which the date is June 18, 2010, with respect to the consolidated statements of financial condition of Charter Financial Corporation and subsidiaries as of September 30, 2009 and 2008, and the related consolidated statements of income, stockholders’ equity and comprehensive income (loss), and cash flows for the years then ended, and we also consent to the use of our report dated June 18, 2010, with respect to the statement of assets acquired and liabilities assumed by CharterBank (a wholly-owned subsidiary of Charter Financial Corporation) as of March 26, 2010, included in the prospectus in the Form S-1 of Charter Financial Corporation and to the references to our Firm under the heading “Experts” in the prospectus.

Atlanta, Georgia

August 12, 2010