Maybe someone told you that your website gets thousands of visits a month.
Maybe they showed you a graph that goes up, with the line rising from left to right. But then you look at the quote requests you’ve received, the leads, and the orders—and the numbers tell a different story.
We don’t know your information, and we don’t know your company. There’s only one thing we can tell you with reasonable certainty: a significant portion of that traffic is generated by programs, not by people.
We know this because we’ve measured it on our website, and we publish the full set of numbers.
From August 15 to 18, 2026, we collected every single request that came in to our server, saved it, and classified it one by one: 27,789 automated requests in just under three days, of which 14,162 came from outside the network.
This article explains what was included and how you can perform the same check on your website.
What percentage of a website’s traffic is bot traffic?
On our website, over three days of continuous monitoring, there were 27,789 requests generated by automated programs.
In the same window, our hosting provider’s security panel classified 9.5% of all requests as likely to be from humans.
The proportions vary from site to site; the general trend—with the majority of traffic generated by software—is the norm.
The next question is more interesting: Why isn’t this visible in Google Analytics?
Because Analytics only tracks users who run JavaScript and accept cookies—in other words, basically only those who use a browser.
Automated programs do neither, so they don’t exist for Analytics. The server log, on the other hand, records them all, because it logs requests before anything else comes into play.
Three words that make everything else clear
If you don’t work on SEO every day, these three points should suffice. If you’re already familiar with them, you can skip to the next paragraph.
Access Log
Every time someone—whether a person or a program—requests a page from your site, the server writes a line to a file: when it happened, from which address, which page, and with what result.
It’s the most comprehensive log of what happens on your site, and almost no one reads it because it’s unreadable at a glance: thousands of lines of densely packed text every day.
Crawler
A crawler is a program that automatically visits web pages. The best-known crawler is Google’s, which scans a website, reads the pages, and adds them to the search engine’s index.
It’s one of dozens, and on our website we’ve counted 37 different ones in three days.
User agent
The calling card that every program presents when it knocks: that’s where it reveals who it is. The tricky part—which we’ll come back to later—is that it writes that calling card itself.
Claiming to be Google’s crawler and actually being it are two different things.
Who Was Actually on Our Website?
Before we get to the table, a few words of explanation are needed, because without them the numbers would give the wrong impression.
Of the 27,789 automated requests, we generated 13,627 of them: 49%.
Of these:
- 7,249 come from the plugin that preloads pages to keep the site fast
- 4,420 from a subscription-based SEO analysis tool
- 1,958 from an analytics platform we developed in-house.
It’s worth keeping this in mind before getting alarmed by a large number: part of that number could be your own website operating on its own.
For this reason, the table below is based solely on the 14,162 requests received from external sources—approximately 5,000 per day.
| Category | Requests | Quota |
| AI Dataset Collection | 5.928 | 41,9% |
| Search Engines | 3.803 | 26,9% |
| Third-Party SEO Tools | 2.289 | 16,2% |
| AI assistants that read to respond | 1.163 | 8,2% |
| Unidentified Programs | 729 | 5,1% |
| Social Media Link Previews | 250 | 1,8% |
The largest source of external traffic isn’t search engines—it’s the collection of content used to train artificial intelligence models, with 5,928 requests compared to 3,803 from all the world’s search engines combined. That’s a ratio of about one and a half to one.
Of those 5,928, the most active program is called img2dataset, which made 3,816 requests—almost all of them for image files—concentrated over a ten-hour period between the night of August 15 and the morning of August 16.
Then he disappeared. His job is exactly what the name implies: building image databases used to train generative models.
He downloaded a communications agency’s photo archive, photo by photo, without notifying them or obtaining a court order. By comparison, during the same time period, Google’s crawler made 545 requests.
AI crawlers that can cite you and those that just take your content
There is a distinction here that is worth keeping in mind, because it is the one that leads to wrong decisions. AI-powered crawlers perform two different functions.
Who Collects for Training
- OpenAI’s GPTBot with 140 requests
- the Meta agent with 970
- ByteDance’s, with 689
- the Common Crawl public archive with 246
- the previously mentioned img2dataset with 3,816.
They take the content, carry it away, and that content ends up mixed in with billions of others inside a model.
No citations, no links, no return visits.
Someone who reads in order to respond to another person
The clearest example is ChatGPT-User, which has 578 requests.
It’s the most active in its category: it’s not an indexing crawler—it’s ChatGPT that opens a page because a user asked a question at that moment.
Each visit corresponds to an actual conversation in which content from the site was used to respond to someone.
Along with him, they behave the same way:
- ClaudeBot with 229 requests
- PerplexityBot with 183
- OpenAI’s search crawler has 97, while You.com ‘s has 45.
The group also includes recent additions such as DuckDuckGo’s DuckAssistBot, ExaSearchBot, and Reflectionbot, a sign that the category is expanding. When they mention them, they include a link.
The practical consequence is that blocking AI crawlers across the board is a decision best avoided: it means you’ll also disappear from the search results that would otherwise mention you.
It’s a good idea to allow GPTBot access to your website
This is the question entrepreneurs ask AI assistants most often on this topic, and the answer depends on which of the two games you want to play.
Allowing training crawlers to run means contributing to models without immediate returns, betting on their presence in future responses.
Allowing those who read the post to respond, on the other hand, helps you get mentioned today, and that’s the approach we recommend sticking with in any case.
One in three AI assistant requests leads to a page that doesn’t exist
This is the data that yields immediate results. Looking at how many requests for each program result in errors, this is the picture that emerges.
| Program | Invalid Requests |
| OpenAI’s Search Crawler | 30,9% |
| GPTBot (OpenAI) | 22,9% |
| Meta Agent | 21,0% |
| Google Crawler | 18,9% |
| PerplexityBot | 13,7% |
| ChatGPT User | 13,1% |
| ClaudeBot (Anthropic) | 12,7% |
The reason is almost always the same: the models are using outdated URLs. A page that was moved two years ago, an archived article, or a URL that was changed when the site was redesigned.
Google, which constantly reviews information and doesn’t want to waste time, updates itself quickly; generative models have a slower and longer-lasting memory.
The solution is technically simple: they’re redirects—instructions that tell visitors that the page is now located elsewhere. Almost no one sets them up, because almost no one looks at that data.
It’s not in Google Analytics, and it’s not in Search Console either: Search Console reports how many errors Google’s crawler encounters, not how many ChatGPT encounters.
Someone pretended to be Google for eighty-three seconds
Among the programs featured on our site, there was one that claimed to be called Google-Extended. Three things didn’t add up.
- First: Google-Extended is not a crawler. It is a label that Google provides to websites to indicate that their content should not be used to train Gemini. No Google program goes by that name.
- Second: All 67 of its requests were sent within 83 seconds, between 01:55:56 and 01:57:19 on August 16, from a single address:
During the same time period, Google’s authentic crawler accessed the site from dozens of different IP addresses, all within the range that Google uses for its own programs. - Third: 47.8% of those requests resulted in errors. It was requesting random pages.
That’s why a business card, on its own, doesn’t tell you much. Anyone can write whatever they want on it. The only reliable way to verify it is by checking the sender’s address, and that can only be done by accessing the server log.
How to Perform the Same Traffic Analysis for Your Website
The first check, without tools
If you have managed hosting, the access log is already there: you can download it from the control panel or via FTP.
Open it in a text editor and search for the words GPTBot, ClaudeBot, PerplexityBot, and Googlebot. The number of results already tells you a lot, and the difference between that number and the sessions you see in Analytics tells you even more.
The second check—the one that really matters
There’s one important limitation: logs are retained for only a few days—four, in our case—and via the API, you can retrieve them for about twenty-three hours. Anything you don’t look at today will be lost in a week.
It’s the opposite of Analytics, which collects data on its own and waits for you. To build a time series, you need to collect and categorize data every day— that’s the work we’ve automated to arrive at the numbers in this article.
What These Data Don’t Tell Us
It would be inaccurate to present these numbers without mentioning their limitations. There are four of them, and they matter.
- It’s three days, not a year. Organic traffic fluctuates quite a bit: on our website, it went from 5,989 visits on August 15 to 12,885 on the 17th. These figures reflect a specific period, and the measurement needs to be repeated.
- Half of the recorded traffic is ours, and the reason is that we constantly analyze this site. That’s the least replicable part of these numbers: the share generated by analytics tools indicates how much someone is measuring that site.
If that share is close to zero in a company’s logs, the meaning is simple: no one is looking at it. It’s worth asking yourself this before anything else. - Some of the traffic does not go through the log. Our hosting service has a distributed cache that handles some requests before they reach the server, and those requests are not counted.
- Our example is a B2B website with about 1,600 pages. An e-commerce site, a newspaper, or a local website would have different proportions. The logic behind this approach applies everywhere; the percentages are specific to our case.
Frequently Asked Questions About Bot Traffic
Bot traffic includes all automated programs, including useful ones such as search engines. Fake traffic is the portion that simulates human behavior to inflate statistics or to search for vulnerabilities. The former must be understood and managed; the latter must be filtered out.
In the vast majority of cases, no. Analytics tracks users who run JavaScript and accept cookies, while crawlers do something else. That’s why a website can have a huge amount of automated traffic yet still show a perfectly normal Analytics graph.
It’s important to make a distinction. Crawlers that read content to answer a question can mention you and drive traffic your way: letting them through is an advantage. For those that collect data solely for training purposes, the decision is a strategic one and should be made with a clear understanding of who they are—not by chance.
It depends on how much of the page is rebuilt from scratch with each request instead of being served from a cached copy. This is a metric measured in the server log that affects response time—a parameter that Google monitors.
Check the source address to verify that it actually falls within the ranges specified by the search engine. The name the program assigns itself is information that is generated automatically, so it should be considered a clue rather than proof.
The Point
The number of visits, on its own, doesn’t tell us much. It tells us much more to know who those visitors are, how many of them might become leads, and how many are actually working for someone else. It’s an analysis you do once, and it changes the way you interpret every subsequent report.
If you want to understand what happens on your website when no one is looking, we’d be happy to discuss it with you. Request a free consultation.