Generate a clean robots.txt: WordPress standard, sitemap line, allow or block AI crawlers and your own disallow paths. There's a one-click blocker for staging. Everything is generated in your browser.
# robots.txt — erzeugt mit forge12.com/robots-txt-generator
User-agent: *
Disallow: /wp-admin/
Disallow: /?s=
Disallow: /search/
Disallow: /readme.html
Disallow: /license.txt
Allow: /wp-admin/admin-ajax.php
Sitemap: https://deine-domain.de/sitemap.xml
Place it in the domain root (reachable at your-domain.com/robots.txt). WordPress otherwise serves a virtual robots.txt — a real file takes precedence.
Pick the mode and rules. Enter your sitemap URL (usually your-domain.com/sitemap.xml). The code on the right updates live.
Save the content as a file named “robots.txt” in your domain root (via FTP/SFTP). A real file takes precedence over WordPress's virtual robots.txt.
Open your-domain.com/robots.txt and test it in the Google Search Console robots.txt report.
A text file in the domain root that tells search-engine crawlers which areas they may crawl and which not. It controls crawling — not indexing directly.
No, not reliably. A path blocked via robots.txt can still end up in the index (without a snippet) if it's linked. To keep a page out of the index, use the “noindex” meta tag or password protection — not robots.txt.
In the root (where wp-config.php lives), reachable at your-domain.com/robots.txt. Without a real file WordPress serves a virtual robots.txt; once a real file exists, it takes precedence.
Yes — but allow admin-ajax.php, because many themes/plugins use it on the front end. That's exactly what the WordPress standard preset does here.
It's a trade-off. Blocking GPTBot, ClaudeBot & co. prevents training on your content — but also costs visibility in ChatGPT, Perplexity and AI Overviews. For most business sites, visibility is the bigger lever.
No. The tool runs entirely in your browser — no server endpoint, nothing transmitted or stored.