Overview
This article explains how to use the traffic tab in the panel to view and interpret bot metrics.
Background
Bots are automated programs designed to perform specific, repetitive tasks. These tasks can range from beneficial activities like indexing the internet for improved SEO, exemplified by Google's "crawlers", to malicious activities such as spamming, data theft, or executing Distributed Denial of Service (DDoS) attacks.
Detecting bots
DreamHost utilizes a combination of IP addresses and user agent information to detect bots on your website. Here's how:
- IP Address: Gives an idea of where the visitor is coming from. A large number of requests from a single IP is often an indicator of a bot.
- User Agent Information: Helps to identify the software the visitor is using to access the website. Bots often use different software than human visitors, providing another clue to their identity.
By using these two pieces of information, DreamHost can more accurately determine whether a visitor is a bot or a human, helping to ensure the security and performance of your website.
Understanding different types of bots
This section provides information on common bot types.
Social bots
-
Social bots can be seen as automated tools that can interact with your website's content and social media platforms, mimicking human behavior to execute specific tasks. These tasks can range from marketing and promoting content to engaging with users, spreading awareness about products or services.
-
They can interact by posting, commenting, liking, sharing, or messaging to achieve specific objectives, which can include marketing initiatives, promotional activities, and even disinformation campaigns. A common example would be Facebook bots.
AI bots
-
AI bots can browse a website to extract data. They can fetch various information like web pages, metadata, or specific content, and process it for subsequent uses such as forming replies or developing training datasets.
-
For instance, ChatGPT may gather content from diverse news websites to provide customized news summaries to users.
Search engine bots
-
Search engine bots are automated software utilized by search engines to explore, analyze, and index online content. They examine web pages to assist in delivering pertinent search results.
-
Their primary functions include making web content searchable, ranking pages considering relevance and SEO parameters, and maintaining the freshness of search engine databases. A good example are Google bots
Viewing Bots traffic page
Use the following steps to view the Bots Traffic graphs:
- Navigate to the Manage Websites page.
- Click the Manage button to open the Domain Settings page, which allows you to adjust various settings for your site.
- Click the Traffic tab.
-
The Traffic page opens and automatically displays a visitors graph. To switch to Bots graphs, click on Visitors at the top of the page and select Bot.
-
Analyzing Bots traffic graphs
This section provides details on the different bot traffic graphs available.
Recent Traffic
-
Displays unique and total bot counts over time. For an extended view, click View More. Once on this page, you can choose to view the 'Last Month', 'Last 7 days', or 'Last 24 hours' reports. Click on the desired option located in the top right corner of the page.
Bot Type Traffic
-
This graph allows you to view and rank bot traffic based on type. By default, the system displays the top 5 Search Engine Bots. However, you can modify the view by selecting either 'Social Bots' or 'AI Bots' from the dropdown menu. For an expanded list of the top 10, click on View More.
Page Views
-
This graph allows you to analyze bot visit trends on your most viewed pages. You can switch between list and graphic modes. Details for specific pages are displayed when you hover over the graphic. It initially shows the top 8 page visits, but you can view up to the top 100 by clicking on View More.
IP Addresses
-
This graph provides a list of recognized bot IP addresses along with their corresponding services or organizations. By default, the top 8 IP addresses are displayed. For a complete list of the top 100, click View More.
-
To block or control the frequency of bot visits to your site, utilize a robots.txt file. For details to control bots, spiders, and crawlers, see this article.
Visitors by Location
-
This graph provides information on the global distribution of bot traffic and the main source countries. For an in-depth analysis, including region-specific data and an extended list of countries, click View More.