How to Block Bots using Robots.txt File in cPanel?

Robot.txt file is a simple text file created on a root directory of your website which allows the search engine bots to crawl your website to index it. robot .txt file also controls how and which files the bots should access so configuring it properly is an essential thing. You can use the robot.txt file to prevent bots from indexing your website in search engines.

In this article, we will show you how to block bots by editing your robot.txt file. Follow the below steps to get started.

  1. Login to your cPanel account.

  2. Open file manager.

    File Manager

  3. Create one file name robot.txt and add the below code to block the bot. This code will block the bot from the Baidu search engine.

    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} Baiduspider [NC]
    RewriteRule .* – [F,L]
    


    Open the file


    edit the file
  4. This is it. Now, Baidu will not be able to bot your website anymore.

  • 0 Users Found This Useful

Was this answer helpful?

Related Articles

How to change max_input_vars in cPanel?

PHP collects the input variables from the cookies enabled on-page from the GET and POST requests....

How to use SFTP to connect a cPanel account?

FTP defined as File Transfer Protocol. You can tranfer files between two remote servers/computers...