How to Block Bots using Robots.txt File in cPanel?

Robot.txt file is a simple text file created on a root directory of your website which allows the search engine bots to crawl your website to index it. robot .txt file also controls how and which files the bots should access so configuring it properly is an essential thing. You can use the robot.txt file to prevent bots from indexing your website in search engines.

In this article, we will show you how to block bots by editing your robot.txt file. Follow the below steps to get started.

  1. Login to your cPanel account.

  2. Open file manager.

    File Manager

  3. Create one file name robot.txt and add the below code to block the bot. This code will block the bot from the Baidu search engine.

    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} Baiduspider [NC]
    RewriteRule .* – [F,L]

    Open the file

    edit the file
  4. This is it. Now, Baidu will not be able to bot your website anymore.

Related Articles

How to Enable Spam Filters From cPanel to Reduce Incoming spam?

In order to filter out unwanted messages/spams, you can activate the Spam Filters anti-spam tool...

How to Create Website from RVSiteBuilder in cPanel?

There are few site builders available in the market which can help the user to build their...

  • 0 Users Found This Useful

Was this answer helpful?