Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

The Board of Directors

Destination XL Group, Inc.:

We consent to the incorporation by reference in the registration statements (Nos. 333-118966 and 333-90742) on Form S-3 and (Nos. 333-136890, 333-163245, 333-164618, 333-170708, 333-170764, 333-194627 and 333-213311) on Form S-8 of Destination XL Group, Inc. of our reports dated March 22, 2019, with respect to the consolidated balance sheets of Destination XL Group, Inc. and subsidiaries as of February 2, 2019 and February 3, 2018, and the related consolidated statements of operations, comprehensive income (loss), changes in stockholders’ equity, and cash flows for each of the years in the three-year period ended February 2, 2019, and the related notes (collectively, the consolidated financial statements), and the effectiveness of internal control over financial reporting as of February 2, 2019, which reports appear in the February 2, 2019 annual report on Form 10-K of Destination XL Group, Inc.

/s/ KPMG LLP

Boston, Massachusetts

March 22, 2019