Exhibit 23.1

CONSENT OF INDEPENDENT AUDITOR

We consent to the incorporation by reference in the Registration Statements on Form S-8 (No. 333-169316, No. 333-185195 and No. 333-193489) of Impac Mortgage Holdings, Inc. of our report dated March 31, 2015 with respect to the financial statements of the Mortgage Division of CashCall, Inc. as of and for the years ended December 31, 2014 and 2013.

/s/ Squar, Milner, Peterson, Miranda & Williamson, LLP

May 11, 2015