Build a valid robots.txt file in seconds: choose which bots may crawl which folders and add your sitemap.
A robots.txt file sits at the root of your site and tells search engine crawlers which areas they may visit. Pick a default rule, set crawl delay, add your sitemap URL and block folders such as /admin/ or /cart/. Upload the generated file as robots.txt to your website's root folder.
Upload it to the root folder of your website so it opens at yourdomain.com/robots.txt. Search engines only read it from that exact address.
No. robots.txt only stops crawling; a blocked page can still appear in results if other sites link to it. To keep a page out of Google, use a noindex tag and let it be crawled.
Free tool by Blue Mint · Updated November 2026
Blue Mint writes your titles and descriptions, fixes technical SEO and grows your Google traffic month by month.
See our SEO service