Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

The Board of Directors

Castle Biosciences, Inc.:

We consent to the incorporation by reference in the registration statement (No. 333-232884) on Form S-8 of Castle Biosciences, Inc. of our report dated March 10, 2020, with respect to the balance sheets of Castle Biosciences, Inc. as of December 31, 2019 and 2018, the related statements of operations and comprehensive income (loss), convertible preferred stock and stockholders’ equity (deficit), and cash flows for each of the years in the two-year period ended December 31, 2019, and the related notes, which report appears in the December 31, 2019 annual report on Form 10-K of Castle Biosciences, Inc.

/s/ KPMG LLP

San Diego, California

March 10, 2020