Exhibit 23.3

Consent of Independent Registered Public Accounting Firm

We consent to the reference to our firm under the caption “Experts” and to the use of our report dated April 29, 2013, in the Registration Statement (Form S-11 No. 333-188214) and related Prospectus of Cherry Hill Mortgage Investment Corporation for the registration of shares of its Common Stock.

/s/ Ernst & Young LLP

New York, NY

May 28, 2013