AI Bot Blocker robots.txt
Choose which AI crawlers to block and copy a ready robots.txt. You can block training crawlers only, add AI search crawlers, or type your own bot names.
- Free, no sign-up
- Nothing is sent to a server
- Worldwide
How to use AI Bot Blocker robots.txt online
- Pick a mode. Training only is the narrowest.
- Add your own names. Optional.
- Add a sitemap. Optional.
- Copy robots.txt. Upload it to your site root.
Why use this free AI Bot Blocker robots.txt
- Bots grouped by kind
- Custom bot names
- Sitemap line
- Bot table with explanations
- Honest note on limits
What robots.txt does
A robots.txt file sits at the top of your site and tells crawlers which paths they may fetch. Well-behaved crawlers read it first and follow it. It is a polite request, not a lock, so a crawler that ignores the rules can still fetch your pages.
Kinds of AI bot
Some bots collect pages that may be used to train AI models. Some find pages so an AI answer engine can show them. Some only fetch a page when a person asks an assistant to read it. Because the goals differ, the page groups the bots by kind, so you can block training but still be found by search.
The modes
Training crawlers only is the narrow choice. Adding AI search crawlers also removes you from those answer engines. Blocking all listed bots includes the ones that fetch a page for a single person. The last choice uses only the names you type yourself.
Keep the list current
Vendors add and rename bots. The names on this page are those the vendors have published, written from memory of their pages and not re-checked on a date here. Check each vendor's own bot page, and remember that some entries, such as Google-Extended, are control tokens and not crawlers you will see in your logs.
Where the file goes
Save the text as robots.txt in the root of your site, so it can be opened at your address followed by /robots.txt. You may add a sitemap line. Nothing you type leaves your browser.
Using the tool
As you type in the AI Bot Blocker robots.txt, the result appears at once. Undo and redo step back through your earlier text.
AI Bot Blocker robots.txt: frequently asked questions
Will this stop every AI company?
Does blocking Google-Extended hurt my search ranking?
Can I add bots that are not in the list?
Where do I put the file?
Are the bot names verified?
Is the AI Bot Blocker robots.txt private?
Is there a way to reuse the AI Bot Blocker robots.txt result?
Related searches for AI Bot Blocker robots.txt
- AI Bot Blocker robots.txt online
- free AI Bot Blocker robots.txt
- AI Bot Blocker robots.txt online free
- AI Bot Blocker robots.txt for mobile
- AI Bot Blocker robots.txt without sign up
- AI Bot Blocker robots.txt – Developer & Web Tools
AI Bot Blocker robots.txt is a free online tool on MAITBOX, in the Developer & Web Tools group. It runs in your browser, so what you type is not sent to a server. It works on Android and iPhone, and on Windows, Mac and Linux, with no download and no account.