QuickLinks -- Click here to rapidly navigate through this document

Exhibit (a)(1)(D)

IMPAC MORTGAGE HOLDINGS, INC.

Offer to Purchase

Offer to purchase all outstanding shares of

9.375% Series B Cumulative Redeemable Preferred Stock (CUSIP No. 45254P300)
9.125% Series C Cumulative Redeemable Preferred Stock (CUSIP No. 45254P409)

and Consent Solicitation

THE OFFER TO PURCHASE AND CONSENT SOLICITATION WILL EXPIRE AT 9:00 A.M., EASTERN DAYLIGHT TIME, ON JUNE 26, 2009, UNLESS EXTENDED OR EARLIER TERMINATED (SUCH TIME AND DATE, AS THE SAME MAY BE EXTENDED, THE "EXPIRATION DATE"). TENDERS OF PREFERRED STOCK MAY BE WITHDRAWN AT ANY TIME PRIOR TO 9:00 A.M., EASTERN DAYLIGHT TIME, ON THE EXPIRATION DATE.

To Brokers, Dealers, Commercial Banks, Trust Companies and Other Nominees:

        Impac Mortgage Holdings, Inc., a Maryland corporation (the "Company"), is making an offer to purchase all outstanding shares of 9.375% Series B Cumulative Redeemable Preferred Stock, $0.01 par value per share ("Series B Preferred Stock") and 9.125% Series C Cumulative Redeemable Preferred Stock, $0.01 par value per share, ("Series C Preferred Stock," and collectively with the Series B Preferred Stock, the "Preferred Stock"), upon the terms and subject to the conditions specified in the Offering Circular, dated May 29, 2009 (the "Offering Circular"), and the related Letter of Transmittal and Consent (which together, as amended, supplemented or otherwise modified from time to time, collectively constitute the "Offer to Purchase").

        We are also soliciting consents (the "Consent Solicitation") from holders of the Preferred Stock (each, a "Holder" and collectively, the "Holders") as of the expiration of the Offer to Purchase and Consent Solicitation, to amend our charter (the "Charter") to modify the terms of each series of Preferred Stock to: (i) make future dividends non-cumulative; (ii) eliminate the provisions prohibiting the payment of dividends on junior stock and prohibiting the purchase or redemption of junior or parity stock if full cumulative dividends for all past dividend periods are not paid or declared and set apart for payment; (iii) eliminate any premiums payable upon the liquidation, dissolution or winding up of the Company; (iv) eliminate the provision prohibiting the Company from electing to redeem Preferred Stock prior to the fifth year anniversary of the issuance of such Preferred Stock; (v) eliminate the provision prohibiting the Company from redeeming less than all of the outstanding Preferred Stock if full cumulative dividends for all past dividend periods have not been paid or declared and set apart for payment; (vi) eliminate the right of holders of Preferred Stock to elect two directors if dividends are in arrears for six quarterly periods; and (vii) eliminate the right of holders of Preferred Stock to consent to or approve the authorization or issuance of preferred stock senior to the Preferred Stock (the "Proposed Amendments").

        The elimination of the restrictions described above would allow us to declare and pay dividends on shares of common stock or shares of any other class or series of our capital stock, except any class or series of parity stock, or redeem, repurchase or otherwise acquire shares of any class or series of our capital stock, including our common stock and any other series of preferred stock, without paying or setting apart for payment any dividends on shares of any series of preferred stock. The foregoing is only a summary of the Proposed Amendments, and is qualified by reference to the amended text of the affected provisions of our Charter reflecting the Proposed Amendments, set forth as Annex A and Annex B to the Offering Circular.

        In connection with the Offer to Purchase and Consent Solicitation, we are requesting that you contact your clients for whom you hold Preferred Stock registered in your name or in the name of your nominee, or who hold Preferred Stock registered in their own names.

        For forwarding to your clients, we are enclosing the following documents:

1.

Offering Circular, dated May 29, 2009;

2.

Letters of Transmittal and Consent for the information of your clients;



3.

A form of notice of guaranteed delivery; and



4.

A form letter, along with Instructions of Beneficial Owner form, which may be sent to your clients for whose account you hold Preferred Stock registered in your name or the name of your nominee, to obtain such clients' instructions with regard to the Offer to Purchase and Consent Solicitation.

        YOUR PROMPT ACTION IS REQUESTED. THE OFFER TO PURCHASE AND CONSENT SOLICITATION WILL EXPIRE AT 9:00 A.M., EASTERN DAYLIGHT TIME, ON JUNE 26, 2009, UNLESS EXTENDED OR EARLIER TERMINATED (SUCH TIME AND DATE, AS THE SAME MAY BE EXTENDED OR EARLIER TERMINATED, THE "EXPIRATION DATE"). TENDERS OF PREFERRED STOCK MAY BE WITHDRAWN AT ANY TIME PRIOR TO 9:00 A.M., EASTERN DAYLIGHT TIME, ON THE EXPIRATION DATE.

        Brokers, banks of other nominees that are tendering by book-entry transfer to the Depositary's Agent's account at DTC may tender Preferred Stock through the DTC ATOP, for which the Offer to Purchase and Consent Solicitation will be eligible. DTC ATOP instructions should include the name and taxpayer identification number of any beneficial owner shares. DTC will then send an Agent's Message to the Depositary. The term "Agent's Message" means a message transmitted by DTC, received by the Company and forming part of the book-entry confirmation, to the effect that: (i) DTC has received an express acknowledgment from a participant in ATOP that it is tendering shares and submitting authorizations to consent to the Proposed Amendments with respect to the Preferred Stock that is the subject of such book-entry confirmation; (ii) such participant has received and agrees to be bound by the terms of the Offer to Purchase and Consent Solicitation for the Preferred Stock being tendered; and (iii) the agreement may be enforced against such participant. Accordingly, the Letter of Transmittal and Consent need not be completed when tendering through ATOP. However, the beneficial holder will be bound by the terms of the Offer to Purchase and Consent Solicitation. Delivery of documents to DTC does not constitute delivery to the Depositary.

        Brokers, banks and other nominees that hold shares of Preferred Stock on behalf of a beneficial holder desiring to tender Preferred Stock prior to the Expiration Date through ATOP should allow sufficient time for completion of the ATOP procedures during the normal business hours of DTC prior to such date, subject to the guaranteed delivery procedures described below. Tenders via the DTC ATOP system shall be deemed made only when timely confirmed by DTC. In all cases, the Holder should allow sufficient time to ensure timely processing of the Holder's tender. If you cannot complete the procedures for book-entry transfer on a timely basis you must tender shares of Preferred Stock pursuant to the guaranteed delivery procedure. Pursuant to such procedure: (i) you must make your tender by or through an eligible institution; (ii) a properly completed and duly executed notice of guaranteed delivery substantially in the form made available by the Company must be received by the Depositary prior to the Expiration Date; and (iii) a confirmation of a book-entry transfer of the Preferred Stock into the Depositary's account at DTC, as well as an Agent's Message and any other documents required by this letter, must be received by the Depositary within three NYSE trading days after the date of execution of such notice of guaranteed delivery.

        In order to accept the Offer to Purchase and Consent Solicitation and consent to the Proposed Amendments an Agent's Message (as defined in the Offering Circular) in connection with a book-entry delivery of Preferred Stock and any other required documents must be received by the Depositary by 9:00 A.M., Eastern Daylight Time, on June 26, 2009.

        The Company will not pay any fees or commissions to any broker, dealer or other person for soliciting tenders of Preferred Stock pursuant to the Offer to Purchase and Consent Solicitation. The Company will, however, upon request, reimburse brokers, dealers, banks and trust companies for customary clerical and mailing expenses incurred by them in forwarding materials to their customers. The Company will pay all stock transfer taxes applicable to its repurchase of Preferred Stock pursuant

to the Offer to Purchase and Consent Solicitation, subject to Instruction 8 of the Letter of Transmittal and Consent.

        Any inquiries you may have with respect to the Offer to Purchase and Consent Solicitation should be addressed to, and additional copies of the enclosed materials may be obtained from, the Information Agent at the address and telephone numbers set forth on the back cover of the Offering Circular.

Sincerely,

IMPAC MORTGAGE HOLDINGS, INC.

NOTHING CONTAINED HEREIN OR IN THE ENCLOSED DOCUMENTS SHALL RENDER YOU THE AGENT OF THE COMPANY, THE INFORMATION AGENT OR THE DEPOSITARY, OR AUTHORIZE YOU OR ANY OTHER PERSON TO USE ANY DOCUMENT OR MAKE ANY STATEMENT ON BEHALF OF ANY OF THEM IN CONNECTION WITH THE OFFER TO PURCHASE AND CONSENT SOLICITATION NOT CONTAINED IN THE OFFERING CIRCULAR, THE LETTER OF TRANSMITTAL AND CONSENT OR THE NOTICE OF GUARANTEED DELIVERY.

QuickLinks

Exhibit (a)(1)(D)