Exhibit 5.1

March 31, 2021   VPC Impact Acquisition Holdings 150 North Riverside Plaza, Ste. 5200 Chicago, IL 60606

Ladies and Gentlemen:

We have acted as New York counsel to VPC Impact Acquisition Holdings, a Cayman Islands exempted company (“VIH” or the “Company”), in connection with the preparation and filing by VIH of a registration statement on Form S-4 (the “Registration Statement,” which term does not include any other document or agreement whether or not specifically referred to or incorporated by reference therein or attached as an exhibit or schedule thereto), including the proxy statement/prospectus forming a part thereof, with the U.S. Securities and Exchange Commission (the “Commission”) relating to, among other things, (1) the merger of Pylon Merger Company LLC, a Delaware limited liability company and direct wholly owned subsidiary of VIH, with and into Bakkt Holdings, LLC (which will be renamed Bakkt Opco Holdings, LLC prior to the consummation of the Business Combination (as defined below), “Bakkt”), a Delaware limited liability company (the “Merger”, and, together with the Domestication (as defined below), the “Business Combination”), with Bakkt surviving the Merger as a subsidiary of Bakkt Pubco (as defined below), pursuant to the terms of the Agreement and Plan of Merger, dated as of January 11, 2021, by and among VIH, Pylon Merger Company LLC and Bakkt (as amended from time to time, the “Merger Agreement”) and (2) the other transactions contemplated by the Merger Agreement and documents related thereto. In connection with the Business Combination, VIH will change its name to “Bakkt Holdings, Inc.” Unless otherwise indicated, each capitalized term used herein has the meaning ascribed to it in the Registration Statement.

Immediately prior to the consummation of the Merger, VIH intends to effect a deregistration under the Cayman Islands Companies Act (2020 Revision) and a domestication under Section 388 of the Delaware General Corporation Law (“DGCL”), pursuant to which VIH’s jurisdiction of incorporation will be changed from the Cayman Islands to the State of Delaware (the “Domestication”) and, in connection therewith, the Company will file the Certificate of Domestication (as defined below) simultaneously with the Certificate of Incorporation (as defined below), in each case, in respect of the Company with the Secretary of State of the State of Delaware (the “Delaware Secretary of State”). In this opinion, we refer to the Company following effectiveness of the Domestication as “Bakkt Pubco.”

Upon the Certificate of Domestication and the Certificate of Incorporation becoming effective under Section 103 of the DGCL (the “Effective Time”), among other things, pursuant to the Plan of Domestication (as defined below): (a) each VIH Unit then issued and outstanding as of immediately prior to the Domestication will automatically be separated into the underlying VIH Class A Ordinary Share and one-half of a VIH Warrant; (b) each VIH Class A Ordinary Share issued and outstanding immediately prior to the Domestication will remain outstanding and will automatically convert into one Bakkt Pubco Class A Share (provided that each VIH Class A Ordinary Share owned by Public Shareholders who have validly elected to redeem their VIH Class A Ordinary Shares will be redeemed for cash in an amount equal to the Redemption Price); (c) each VIH Class B Ordinary Share issued and outstanding immediately prior to the Domestication will be automatically converted into one Bakkt Pubco Class A Share; (d) each VIH Warrant will be automatically converted into a redeemable warrant exercisable for one Bakkt Pubco Class A Share on the same terms as the VIH Warrants; and (e) each Private Placement Warrant issued and outstanding prior to the Domestication will be automatically converted into a warrant exercisable for one Bakkt Pubco Class A Share on the terms and subject to the conditions set forth in the applicable warrant agreement.

March 31, 2021

This opinion letter is rendered in accordance with the requirements of Item 601(b)(5) of Regulation S–K under the Securities Act of 1933, as amended (the “Securities Act”), and no opinion is expressed herein as to any other matter pertaining to the contents of the Registration Statement or related proxy statement/prospectus, any proxy statement/prospectus filed pursuant to Rule 424(b) with respect thereto, other than as expressly stated herein with respect to the issue of the VIH Shares, the VIH Warrants and the Private Placement Warrants.

In connection with our opinions expressed below, we have examined originals or copies certified to our satisfaction of the following documents and such other documents, certificates and other statements of government officials and corporate officers of VIH as we deemed necessary for the purposes of the opinions set forth in this opinion letter:^,

(a) the Registration Statement;

(b) the Merger Agreement, filed as Exhibit 2.1 to the Registration Statement, as amended;

(c) the form of certificate of incorporation of the Company to become effective upon consummation of the Business Combination, filed as Exhibit 3.2 (the “Certificate of Incorporation”);

(d) the form of bylaws of the Company to become effective upon consummation of the Business Combination, filed as Exhibit 3.3 (the “Bylaws”);

(e) the Form of Certificate of Domestication, to be filed with the Delaware Secretary of State (the “Certificate of Domestication”);

(f) the form of Warrant certificate (included in the Warrant Agreement);

(g) an executed copy of the Warrant Agreement, dated September 22, 2020, between the Company and Continental Stock Transfer & Trust Company, as warrant agent; and

(h) resolutions of the Board of Directors of the Company, dated January 8, 2021 and March 30, 2021, relating to, among other things, the Registration Statement, the Merger and the Domestication.

We have relied, to the extent we deem such reliance proper, upon such certificates or comparable documents of officers and representatives of the Company and of public officials and upon statements and information furnished by officers and representatives of the Company with respect to the accuracy of material factual matters contained therein which were not independently established by us. In rendering the opinions expressed below, we have assumed, without independent investigation or verification of any kind, the genuineness of all signatures on documents we have reviewed, the legal capacity and competency of all natural persons signing all such documents, the authenticity and completeness of all documents submitted to us as originals, the conformity to authentic, complete original documents of all documents submitted to us as copies, the truthfulness, completeness and correctness of all factual representations and statements contained in all documents we have reviewed, the accuracy and completeness of all public records examined by us, and the accuracy of all statements in certificates of officers of the Company that we reviewed.

2

March 31, 2021

In addition to the foregoing, for the purpose of rendering our opinions as expressed herein, we have assumed that:

1.    Prior to effecting the Domestication: (i) the Registration Statement, as finally amended (including all necessary post-effective amendments), will have become effective under the Securities Act; (ii) the shareholders of the Company will have approved, among other things, the Merger Agreement and the Domestication, the Business Combination, including the Certificate of Incorporation and Bylaws, and that all conditions precedent to the closing of the Business Combination have been satisfied or will have been satisfied or otherwise waived; and (iii) all other necessary action will have been taken under the applicable laws of the Cayman Islands to authorize, approve and permit the Domestication, and any and all consents, approvals and authorizations from applicable Cayman Islands and other governmental and regulatory authorities required to authorize and permit the Domestication will have been obtained;

2.    The Certificate of Domestication, in the form attached as Exhibit 3.4 to the Registration Statement, without alteration or amendment (other than identifying the appropriate date), will be duly authorized and executed and thereafter be duly filed with the Delaware Secretary of State in accordance with Sections 103 and 388 of the DGCL, that no other certificate or document, other than the Certificate of Incorporation, has been, or prior to the filing of the Certificate of Domestication will be, filed by or in respect of the Company with the Delaware Secretary of State and that the Company will pay any fees and other charges required to be paid in connection with the filing of the Certificate of Domestication;

3.    The Certificate of Incorporation, in the form filed as Exhibit 3.2 to the Registration Statement, without alteration or amendment (other than identifying the appropriate date), will be duly authorized and executed and thereafter be duly filed with the Delaware Secretary of State and have become effective in accordance with Sections 103 and 388 of the DGCL, that no other certificate or document, other than the Certificate of Domestication, has been, or prior to the filing of the Certificate of Incorporation will be, filed by or in respect of the Company with the Delaware Secretary of State and that the Company will pay any fees and other charges required to be paid in connection with the filing of the Certificate of Incorporation; and

4.    The Bylaws, in the form attached as Exhibit 3.3 to the Registration Statement, without alteration or amendment (other than identifying the appropriate date), will become effective upon the Effective Time.

Based upon the foregoing assumptions and the assumptions set forth below, and subject to the qualifications and limitations stated herein, having considered such questions of law as we have deemed necessary as a basis for the opinions expressed below, we are of the opinion that:

1.    Upon the Effective Time, pursuant to the Merger Agreement, each issued and outstanding Class A ordinary share of VIH will convert automatically into one share of Bakkt Pubco Class A common stock that will have been duly authorized by all requisite corporate action on the part of Bakkt Pubco under the DGCL and that will be validly issued, fully paid and nonassessable.

3

March 31, 2021

2.    Upon the Effective Time, pursuant to the Merger Agreement, each issued and outstanding Class B ordinary share of VIH will convert automatically into one share of Bakkt Pubco Class A common stock that will have been duly authorized by all requisite corporate action on the part of Bakkt Pubco under the DGCL and that will be validly issued, fully paid and nonassessable.

3.    Upon the Effective Time, pursuant to the Merger Agreement, each issued and outstanding VIH Warrant will become one Bakkt Pubco Warrant that will have been duly authorized by all requisite corporate action on the part of Bakkt Pubco under the DGCL and will constitute valid and binding obligations of Bakkt Pubco, enforceable against Bakkt Pubco in accordance with their terms, subject to (i) applicable bankruptcy, insolvency, receivership, conservatorship, liquidation, reorganization, moratorium, fraudulent transfer and other laws affecting the enforcement of creditors’ rights generally, and (ii) the application of general principles of equity (whether applied by a court in equity or at law).

4.    Upon the Effective Time, pursuant to the Merger Agreement, each issued and outstanding Private Placement Warrant of VIH will become one Bakkt Pubco warrant that will have been duly authorized by all requisite corporate action on the part of Bakkt Pubco under the DGCL and will constitute valid and binding obligations of Bakkt Pubco, enforceable against Bakkt Pubco in accordance with their terms, subject to (i) applicable bankruptcy, insolvency, receivership, conservatorship, liquidation, reorganization, moratorium, fraudulent transfer and other laws affecting the enforcement of creditors’ rights generally, and (ii) the application of general principles of equity (whether applied by a court in equity or at law).

5.    Upon the Effective Time, the shares of Bakkt Pubco Class A common stock issuable upon exercise of the Bakkt Pubco Warrants that will have been duly authorized by all requisite corporate action on the part of Bakkt Pubco under the DGCL, will be duly validly issued, fully paid and non-assessable.

The opinions stated herein are subject to the following qualifications:

(a)    we do not express any opinion with respect to any law, rule or regulation that is applicable to any party to the Merger Agreement or any Warrant certificate or Warrant Agreement or the transactions contemplated thereby solely because such law, rule or regulation is part of a regulatory regime applicable to any such party or any of its affiliates as a result of the specific assets or business operations of such party or such affiliates;

(b)    we do not express any opinion with respect to the enforceability of any provision contained in any Warrant certificate or Warrant Agreement relating to any indemnification, contribution, non-reliance, exculpation, release, limitation or exclusion of remedies, waiver or other provisions having similar effect that may be contrary to public policy or violative of federal or state securities laws, rules or regulations, or to the extent any such provision purports to, or has the effect of waiving or altering any statute of limitations; and

(c)    we have assumed that Continental Stock Transfer & Trust Company has the power, corporate or other, to enter into and perform all obligations under the Warrant Agreement and have also assumed due authorization by all requisite action, corporate or other, and the execution and delivery by Continental Stock Transfer & Trust Company of the Warrant Agreement and that the Warrant Agreement constitutes the valid and binding obligation of Continental Stock Transfer & Trust Company, enforceable against Continental Stock Transfer & Trust Company in accordance with its terms.

4

March 31, 2021

VIH is a Cayman Islands exempted company that will be redomesticated as a Delaware corporation pursuant to the Domestication, and we have not considered, and we express no opinion as to, any law other than the DGCL.

This opinion letter is for your benefit in connection with the Registration Statement and may be relied upon by you and by persons entitled to rely upon it pursuant to the applicable provisions of the Securities Act. This opinion letter is provided solely in connection with the issuance of the common stock and warrants registered pursuant to the Registration Statement, and is not to be relied upon for any other purpose.

The opinions expressed above are as of the date hereof only, and we express no opinion as to, and assume no responsibility for, the effect of any fact or circumstance occurring, or of which we learn, subsequent to the date of this opinion letter, including, without limitation, legislative and other changes in the law or changes in circumstances affecting any party. We assume no responsibility to update this opinion letter for, or to advise you of, any such facts or circumstances of which we become aware, regardless of whether or not they affect the opinions expressed in this opinion letter.

We hereby consent to the filing of this opinion letter as Exhibit 5.1 to the Registration Statement and to the reference to our firm as counsel for the Company that has passed on the validity of the securities to be issued in connection with the Domestication, appearing under the caption “Legal Matters” in the prospectus forming part of the Registration Statement or any prospectus filed pursuant to Rule 424(b) with respect thereto. In giving this consent, we do not thereby admit that we are within the category of persons whose consent is required under Section 7 of the Securities Act or the rules and regulations of the Securities and Exchange Commission thereunder.

Very truly yours,

White & Case LLP

/s/ White & Case LLP

JR:EMS:EA:SB:BM:PD

5