Robots.txt Generator

Create SEO-friendly robots.txt files for your website. Control search engine crawling and improve your site's indexation with our free robots.txt generator.

Website URL

Default Directive for All Search Engines

Allow All (Index entire website)

Disallow All (Block entire website)

Custom (Recommended for WordPress/CMS)

Crawl Delay (seconds) - Optional Delay between requests from crawlers (1-86400 seconds)

Sitemap URL - Optional

Custom Rules - Optional

📝 WordPress Template

Standard robots.txt for WordPress sites with common restrictions for admin areas and sensitive files.

🛒 E-commerce Template

Optimized for online stores, blocking cart, checkout, and account pages while allowing product pages.

🌐 Standard Website

Basic template for regular websites with minimal restrictions and good crawling permissions.

🔒 Private Website

Block all search engines from indexing your website completely.

How to Use Robots.txt

Generate your robots.txt using the form above
Copy the generated content and save it as "robots.txt" (without quotes)
Upload the file to your website's root directory (e.g., https://yoursite.com/robots.txt)
Test your robots.txt by visiting yoursite.com/robots.txt in a browser
Submit to Google Search Console for validation

Important Notes:

Robots.txt is publicly accessible and visible to everyone
It's a guideline, not a guarantee - malicious bots may ignore it
Always test your robots.txt file before deploying
Include your sitemap URL to help search engines find your content