Exhibit 23.2

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement (Form S-8) pertaining to the 2004 Stock Incentive Plan of Capital One Financial Corporation of our report dated February 15, 2006, except for Note 27, as to which the date is February 27, 2006, with respect to the consolidated financial statements of Capital One Financial Corporation, and our report dated February 15, 2006, with respect to Capital One Financial Corporation management’s assessment of the effectiveness of internal control over financial reporting, and the effectiveness of internal control over financial reporting of Capital One Financial Corporation, included in its Annual Report (Form 10-K) for the year ended December 31, 2005 filed with the Securities and Exchange Commission.

/s/ ERNST & YOUNG LLP

McLean, Virginia

July 31, 2006