How to Block Bots using Robots.txt File in cPanel?

Robot.txt file is a simple text file created on a root directory of your website which allows the search engine bots to crawl your website to index it. robot .txt file also controls how and which files the bots should access so configuring it properly is an essential thing. You can use the robot.txt file to prevent bots from indexing your website in search engines.

In this article, we will show you how to block bots by editing your robot.txt file. Follow the below steps to get started.

1. Login to your cPanel account.
   
   
2. Open file manager.
   
   
   
   
3. Create one file name robot.txt and add the below code to block the bot. This code will block the bot from the Baidu search engine.
   
   

   RewriteEngine On
   RewriteCond %{HTTP_USER_AGENT} Baiduspider [NC]
   RewriteRule .* – [F,L]
   

   
   
   
   
   
   
4. This is it. Now, Baidu will not be able to bot your website anymore.