How to Block Bots using Robots.txt File in cPanel?

Robot.txt file is a simple text file created on a root directory of your website which allows the search engine bots to crawl your website to index it. robot .txt file also controls how and which files the bots should access so configuring it properly is an essential thing. You can use the robot.txt file to prevent bots from indexing your website in search engines.

In this article, we will show you how to block bots by editing your robot.txt file. Follow the below steps to get started.

  1. Login to your cPanel account.

  2. Open file manager.

    File Manager

  3. Create one file name robot.txt and add the below code to block the bot. This code will block the bot from the Baidu search engine.

    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} Baiduspider [NC]
    RewriteRule .* – [F,L]
    


    Open the file


    edit the file
  4. This is it. Now, Baidu will not be able to bot your website anymore.

Related Articles

How to Create Website from RVSiteBuilder in cPanel?

There are few site builders available in the market which can help the user to build their...

How to Block Emails by Country or Domain in cPanel?

In the latest version of cPanel, they have added new features to block the emails from any...

How to Fix: check_mysql Has Determined That There Are Corrupted Database Tables (eximstats)

In this article, we will discuss how to resolve the error: check_mysql Has Determined That There...

  • 0 Users Found This Useful

Was this answer helpful?