Menu
PDF Tools
Image Tools
Converters & Utilities
Seller & Print Tools
Accounting & Business
Calculators
Text & Writing Tools
Developer & Web Tools
Creator & Social Tools
Browser Utilities & Games
Document & Design Makers
Text & Developer Tools
Exam & Form Tools

AI Bot Blocker robots.txt

Choose which AI crawlers to block and copy a ready robots.txt. You can block training crawlers only, add AI search crawlers, or type your own bot names.

How to use AI Bot Blocker robots.txt online

  1. Pick a mode. Training only is the narrowest.
  2. Add your own names. Optional.
  3. Add a sitemap. Optional.
  4. Copy robots.txt. Upload it to your site root.

Why use this free AI Bot Blocker robots.txt

  • Bots grouped by kind
  • Custom bot names
  • Sitemap line
  • Bot table with explanations
  • Honest note on limits

What robots.txt does

A robots.txt file sits at the top of your site and tells crawlers which paths they may fetch. Well-behaved crawlers read it first and follow it. It is a polite request, not a lock, so a crawler that ignores the rules can still fetch your pages.

Kinds of AI bot

Some bots collect pages that may be used to train AI models. Some find pages so an AI answer engine can show them. Some only fetch a page when a person asks an assistant to read it. Because the goals differ, the page groups the bots by kind, so you can block training but still be found by search.

The modes

Training crawlers only is the narrow choice. Adding AI search crawlers also removes you from those answer engines. Blocking all listed bots includes the ones that fetch a page for a single person. The last choice uses only the names you type yourself.

Keep the list current

Vendors add and rename bots. The names on this page are those the vendors have published, written from memory of their pages and not re-checked on a date here. Check each vendor's own bot page, and remember that some entries, such as Google-Extended, are control tokens and not crawlers you will see in your logs.

Where the file goes

Save the text as robots.txt in the root of your site, so it can be opened at your address followed by /robots.txt. You may add a sitemap line. Nothing you type leaves your browser.

Using the tool

As you type in the AI Bot Blocker robots.txt, the result appears at once. Undo and redo step back through your earlier text.

AI Bot Blocker robots.txt: frequently asked questions

Will this stop every AI company?
No. robots.txt is a request. Crawlers that respect it will follow it and others may not.
Does blocking Google-Extended hurt my search ranking?
It is a separate control token for AI use, and it is not the same as the normal Google crawler. Check Google's own documentation.
Can I add bots that are not in the list?
Yes. Type their names, one per line.
Where do I put the file?
In the root folder of your site, named robots.txt.
Are the bot names verified?
They are the published names as the page knows them, not re-checked on a date. Check each vendor page.
Is the AI Bot Blocker robots.txt private?
Your text stays on this device when you use the AI Bot Blocker robots.txt. The editor may keep the last text locally and offers to bring it back; you can dismiss it.
Is there a way to reuse the AI Bot Blocker robots.txt result?
Copy the AI Bot Blocker robots.txt result, or save it as a text file from the result box.

Related searches for AI Bot Blocker robots.txt

  • AI Bot Blocker robots.txt online
  • free AI Bot Blocker robots.txt
  • AI Bot Blocker robots.txt online free
  • AI Bot Blocker robots.txt for mobile
  • AI Bot Blocker robots.txt without sign up
  • AI Bot Blocker robots.txt – Developer & Web Tools

AI Bot Blocker robots.txt is a free online tool on MAITBOX, in the Developer & Web Tools group. It runs in your browser, so what you type is not sent to a server. It works on Android and iPhone, and on Windows, Mac and Linux, with no download and no account.

More tools in Developer & Web Tools →

Search tools