EXHIBIT 23

Consent of Independent Registered Public Accounting Firm

The Board of Directors

Ambac Financial Group, Inc.:

We consent to the incorporation by reference in the Registration Statements (Nos. 333-110145 and 333-52449) on Form S-8 of Ambac Financial Group, Inc. of our report dated June 26, 2006, relating to the statements of net assets available for plan benefits of the Ambac Financial Group, Inc. Savings Incentive Plan as of December 31, 2005 and 2004 and related statements of changes in net assets available for plan benefits for the years then ended, and the related supplemental schedule H, line 4i- schedule of assets (held at end of year) as of December 31, 2005 and schedule H, line 4j- schedule of reportable transactions for the year ended December 31, 2005, which report appears in the December 31, 2005 annual report on Form 11-K of the Ambac Financial Group, Inc. Savings Incentive Plan.

/s/ KPMG LLP

New York, New York

June 26, 2006