EXHIBIT 5.1

[LETTERHEAD OF MCKEE NELSON LLP]

May 7, 2004

Impac Mortgage Holdings, Inc.

1401 Dove Street

Newport Beach, California 92660

Ladies and Gentlemen:

This opinion is furnished in connection with the registration statement on Form S-3, as amended (File No. 333-111517) (the “Registration Statement”) filed with the Securities and Exchange Commission (the “Commission”) under the Securities Act of 1933, as amended (the “Securities Act”), relating to the offering by Impac Mortgage Holdings, Inc., a Maryland corporation (the “Company”), of up to 1,725,000 shares of its common stock, par value $0.01 per share (the “Shares”).

As Maryland counsel to the Company, we have examined and relied on the originals or copies, certified or otherwise identified to our satisfaction, of such instruments, certificates, records, and other documents and have made such examination of law as we have deemed necessary or appropriate for the purpose of this opinion. In our examination, we have assumed the legal capacity of all natural persons, the genuineness of all signatures, the authenticity of all documents submitted to us as originals, the conformity to original documents of all documents submitted to us as certified or photostatic copies or by facsimile or other means of electronic transmission or which we obtained from the Commission’s Electronic Data Gathering, Analysis and Retrieval System (“EDGAR”) or FreeEDGAR.com, and the authenticity of the originals of such latter documents. As to facts relevant to the opinions expressed herein and other statements made herein, we have relied without independent investigation upon certificates and oral or written statements and representations of public officials, officers, and other representatives of the Company.

We are attorneys admitted to practice in the State of Maryland. We express no opinion concerning the laws of any of the jurisdictions other than the laws of the United States of America and the State of Maryland.

Based upon the foregoing, we are of the opinion that the Shares have been duly authorized by the Company and, when issued and paid for as contemplated by the Registration Statement, will be duly and validly issued, fully paid, and non-assessable.

This opinion is being furnished to you for use in connection with the Registration Statement and may be relied upon by Kirkpatrick & Lockhart LLP. We hereby consent to the

Impac Mortgage Holdings, Inc.

May 7, 2004

Page 2

filing of this opinion as Exhibit 5.1 to the Form 8-K and the reference to our firm under the caption “Legal Matters” in the prospectus or any supplement thereto. In giving this consent, we do not admit that we are within the category of persons whose consent is required under Section 7 of the Securities Act or the rules and regulations of the Commission promulgated thereunder.

Very truly yours,

/s/ McKee Nelson LLP