Robots.txt Generator

Generate robots.txt with the most useful rules to make sure search engines are crawling and indexing your website.

Options

Sitemaps

Must include http(s)://. Separate each sitemap by line. Leave empty if none.

Apply following rules to all crawlers

By default, crawlers should be

Disallow individual paths

Absolute paths to root of website (start with /). Separate each path by line. Leave empty if none.

Code

Loading code...

Privacy

We participate in the Cloudways Affiliate Program, which means that if you click on a link to Cloudways and sign up for their services, we may earn a commission at no extra cost to you. Our recommendations are based on our genuine experience and expertise with their platform.