EXHIBIT 23

Consent of Independent Registered Public Accounting Firm

The Board of Directors

Ambac Financial Group, Inc.:

We consent to the incorporation by reference in the Registration Statements (Nos. 333-110145 and 333-52449) on Forms S-8 of Ambac Financial Group, Inc. of our report dated June 14, 2005, relating to the financial statements of Ambac Financial Group, Inc. Savings Incentive Plan as of and for the years ended December 31, 2004 and 2003, and related supplemental schedules, which report appears in the December 31, 2004 annual report on Form 11-K of Ambac Financial Group, Inc.

/s/ KPMG LLP

New York, New York

June 28, 2005