A “Keep Out” Sign for AI Scrapers: Steps to Help Protect Your Art on Shopify

If you sell art online, or if you're like us and you work with a lot of artists and creatives, wanting more control over how that work is collected and used is entirely reasonable.

tl;dr: Scroll to the header that says "How to Add a Crawler Rule to a Shopify Store"

Why This Matters to Us, and to Artists & Creators

We love sharing the things we print and make for others on our socials to spread awareness of the artists and creatives we work with. We want them to succeed, find new audiences, and their work deserves to be seen!

From a business standpoint, if we printed it we want to show it off. If we make you look good, then we look good, and then we get to work with more artists as a result. But now we need to be incredibly careful about what we share, and *how* we show off what we make. We are going to be making some changes to how we use social media to better protect your work, and ours. 

Our first step is doing what we are able to do to keep Meta's AI scraping bot off of our site. Part of their recent changes to their Commercial Terms included some specific wording about AI and data collection. This caught the attention of creators, which led to some changes in their wording (launch pushed back to Nov 20th as of writing this because of creator backlash, so keep speaking up!). The original phrasing of the policy involved consent to crawl and scrape any links we put on our business page.

Following the criticism, Meta removed “train” from the disputed Commercial Terms clause and narrowed its stated purposes to advertising and business products. Meta says that users can update the robots.txt file on their to block their AI crawler.

Read the AI policies on all sites where your creative work is posted.

Meta is not the only one doing this, they're just the one we're hearing about right now.

There’s no simple switch that makes a public website scrape-proof or immune to AI. But you can take practical steps to state your preferences and make casual crawling less likely moving forward.

Crawling is when an automated program (called a crawler or bot) visits a website and reads their pages. It may look at your product pages, your text, and your images, then it will save that information so a search engine can list your store and push it to the right people (ex: Google) or, in some cases, so a company can collect (scrape) material for other purposes.

Scraping is when software automatically collects information from websites or online services, like product names, prices, descriptions, or images, and then saves or processes it elsewhere. Some scraping is ordinary and usually authorized, such as a search engine (Google) collecting page information to index. Other types of scraping may violate a site’s rules, or a person’s wishes (which is what many feel Meta's AI policy changes are about to do.)

 

So... that robots.txt file

On your website, you'll have a robots.txt file which tells web crawlers that follow the rules which parts of a site they should avoid. It doesn’t prevent a determined scraper from requesting access to public pages, and it can’t retrieve work that has already been copied. Shopify describes these rules as advisory, and says that not all crawlers are guaranteed to follow them. [Link to this info on Shopify]

Think of disallowing certain crawlers in your robots.txt as a Keep Out sign, not a locked door. A polite bot that follows the rules will respect the request to leave. Not all crawlers are guaranteed to follow them, just as a Keep Out sign won't stop a determined thief from walking through an unlocked door. 

So there are rules for bots but they don't necessarily have to follow them? 

Yeah, kind of. Meta provides objection mechanisms in the EU and UK for certain uses of personal information in AI training. Those mechanisms are separate from the website crawler controls discussed here. US users are not offered the same opportunity to object. When I submitted a complaint to Meta to make it clear I don't want our work, posts, reels, comments, etc training AI, the response I received was that they will follow the laws where I live. Our actual informed consent seems to matter less to them than what they can legally get away with.

Many of us who make art or have a business with a social presence have huge audiences on Facebook and Instagram; stepping away can feel impossible. Start building a presence elsewhere (check their AI policies first), post there first, direct people there for the most recent updates. Call your legislators. Submit complaints to Meta. Run for office. We need regulation.

How to Add a Crawler Rule to a Shopify Store:

Shopify creates a default robots.txt file for every store. To customize it, Shopify’s current instructions are:

1. Before making changes, save a copy of your current robots.txt output.
2. In Shopify admin, go to Online Store → Themes.
3. On your published theme, choose Edit code.
4. Open templates/robots.txt.liquid, or create a robots.txt.liquid template if one doesn’t exist.
5. Add the crawler rules shown below, keeping the default-rule structure intact. If the template already contains custom rules, preserve those rather than replacing them blindly.
6. Save, then visit yourdomain.com/robots.txt to check the published output.
7. Compare the output with your saved copy. Confirm that the intended crawler rules appear and that you haven’t unintentionally removed existing rules

If you are doing this, as an extra safety step, before publishing any changes, save a copy of your current robots.txt if there was one and make sure your custom version retains Shopify’s default protections and SEO rules. (Link to a How to on Shopify)

Some quick liability language: I'm here to share some info, changing or adding anything to your website is your choice. Shopify warns that replacing the default rules wholesale can create problems, and recommends working with a Shopify Partner or someone experienced with code and SEO. Basically, if you're not sure what you're doing, ask a professional or do more research to learn more before making big changes.

That being said, here is what I put in my robots.txt.liquid file:

{% for group in robots.default_groups %}

  {{- group.user_agent }}

  {% for rule in group.rules %}

    {{- rule }}

  {% endfor %}

  {%- if group.sitemap != blank %}

    {{ group.sitemap }}

  {% endif %}

{% endfor %}

User-agent: Meta-ExternalAgent

Disallow: /

User-agent: Meta-WebIndexer

Disallow: /

User-agent: Meta-ExternalFetcher

Disallow: /

These rules ask these Meta crawlers not to access any path on our site (Disallow: /) 

• Meta-ExternalAgent : Meta describes this one as crawling public pages for uses such as training AI models or improving products by indexing content. This is the one they're publicly telling people to add to robots.txt if you don't want your stuff crawled and scraped.
  
  
• Meta-WebIndexer : This one gathers information to improve Meta AI search results. Blocking it is likely to reduce the chance your pages are cited or linked in Meta AI answers.
  
  
• Meta-ExternalFetcher : This fetches a specific link when someone asks a Meta AI product to visit it, including for AI assistant features. Meta says this user-requested fetcher may not follow robots.txt rules in the same way as its regular crawlers. This is because the request comes from a person (something like "find me a blue vase, gift, small, ships US")
  
  

By blocking these three (which is not a complete list of Meta crawlers), I essentially asked Meta not to use our site for training/indexing, Meta AI search, or user-requested fetching. Blocking Meta-WebIndexer may reduce the chance that your products appear as links in Meta AI search results. That could matter if your customers use Meta AI like a search engine to discover businesses or products, though it won’t determine whether they can find you through other routes.  Again, this step tells the named crawlers not to crawl the site. It does not 100% definitively prevent access.

Important: Blocking open-web crawlers is not necessarily the same as blocking product information shared through other channels. Crawler rules don’t control product data sent through Shopify Catalog to agentic storefronts that you’ve activated. Ironically, Shopify's AI "sidekick" is very helpful and can walk you through all of this.

Other Reasons this Change Poses Privacy Concerns:

The updated terms grant Meta permission to crawl, index, and analyze the domains, web pages, or URLs, and any content available through them, that are associated with your products or services.

1. The "Link in Bio" (Profile Links)
   If your Creator or Business account lists your business domain in your profile description or uses a landing page tool (like Linktree), Meta’s bots can crawl that destination.
   
   
2. Links Posted Anywhere on Your Professional Page
   Because the Commercial Terms govern your entire use of Meta’s professional dashboards, the scraping rights may extend to external links you share. Because it is worded so broadly, this could include things like:
   - Links placed inside your Feed posts.
   - Links dropped into Captions (on Facebook or threads).
   - Links pasted inside the Comments section of your own posts.
   - Links shared in Instagram Story Link Stickers.
   
   
3. Off-Platform Connections
   The policy is broadly worded, which raises concerns that this is not limited to text links you physically type out. "Associated with" may also cover websites linked to your account in other ways:
   - Your site if it has the Meta Pixel installed.
   - Websites feeding into your Meta Product Catalog or Instagram Shopping.
   - Websites connected to any active Meta Ad Campaigns.

Glaze and Nightshade for images

You may have heard of these! These tools are said to "poison" your images for AI. 

Glaze and Nightshade are image-processing tools from the University of Chicago. Glaze was designed to interfere with AI's ability to imitate an artist’s style. Nightshade alters image data to disrupt training if an AI model uses the image. They both work on the image file itself, but they don’t stop a bot from visiting your website or downloading an image.

The smart people on the internet suggest that you keep your original files untouched, process a copy with these tools, and check the results before posting. Use the processed copy as your public web image, and retain the original high-resolution artwork privately for prints/production. The changes may be visibly noticeable, and processing can take time, so this isn't a quick plug-and-play step. The Glaze team itself says its tool is not a permanent solution. (Read what the creators say at University of Chicago) 

These tools are best understood as an additional precaution, not a guarantee. For example, researchers found that a tool called LightShed could detect and remove some protections from images, so image-level defenses shouldn’t be treated as foolproof. The thief is intent on getting into the house. We can't lock every door, and we can't delete the house, so we make our stuff harder or less interesting to steal.

More guides and background

• Shopify Help: Editing robots.txt.liquid - Shopify’s setup steps and cautions.
• Shopify Developer docs: Customize robots.txt - technical details on editing the template.
• Glaze: What it does and its limitations
  
• Nightshade project site
  

No perfect shield

It’s possible that work posted publicly has already been collected or copied; a new setting can’t undo that. Going forward, you can combine safeguards: tell compliant crawlers your preference, publish lower-resolution previews where practical, keep your original files offline, and use image-protection tools if they fit your process and workflow.