Robots.txt Generator

Build a valid robots.txt file in seconds.

Crawling

Paths to disallow (one per line)

Sitemap URL (optional)

robots.txt

User-agent: *
Allow: /
Disallow: /admin
Disallow: /private

About this tool

Free robots.txt generator. Choose which crawlers and paths to allow or block and generate a valid robots.txt file for your website instantly.