Exhibit 23.2

Consent of Independent Registered Public Accounting Firm

We consent to the reference to our firm under the caption “Experts” in this Registration Statement (Form S-3) and related Prospectus of Capital One Financial Corporation for the registration of senior and subordinated debt securities, preferred stock, depositary shares representing preferred stock, common stock, purchase contracts, warrants, and units of Capital One Financial Corporation and to the incorporation by reference therein of our report dated March 5, 2012, with respect to the consolidated financial statements of ING Bank, fsb and Subsidiaries included in the Current Report on Form 8-K of Capital One Financial Corporation dated March 14, 2012.

/s/ Ernst & Young LLP

McLean, VA

April 30, 2012