Verified crawl
A user agent string is free to set. At scmc.ch, 95 percent of requests carrying a crawler user agent were forged. How to tell the real ones apart, and why it only works with server logs.
Read the term →Glossary
AI visibility, term by term
What the terms around AI visibility actually mean, each one backed by a measurement from our own estate. No entry without evidence, no evidence without a date.
AI crawler
Programs run by AI providers to fetch pages so a model can answer from them. Across eleven measured hosts they came 17,290 times last month, the classic search bots only 7,976.
Read the term →JavaScript shell
If the text only arrives after JavaScript runs, an AI crawler gets an empty frame. Vercel measured this across its own network in December 2024, and for us it affects 17,290 requests a month.
Read the term →llms.txt
The proposal promises a machine-readable summary of your site for language models. In 30 days on scmc.ch: 417 fetches of robots.txt by AI crawlers, zero fetches of llms.txt.
Read the term →Scanner
35 percent of all requests to our eleven hosts come from programs hunting for weaknesses. On small sites they outnumber humans twenty to one.
Read the term →Query fan-out
AI search breaks a question into many searches. In the Search Console of k-i-soft.ch, 14 of 149 queries were whole prompts, one of them word for word on 31 of 35 days. What that means, measured.
Read the term →User-triggered fetch
ChatGPT-User, Claude-User and Perplexity-User fetch a page while an answer is being written. On k-i-soft.ch, 907 requests carried the ChatGPT-User identifier in 31 days, and 88 of them were genuine.
Read the term →Discovered, not indexed
A page in the sitemap is not yet a page in the index. On 1 October 2026, 30 of 86 addresses of k-i-soft.ch were not indexed, the most important offer page unknown to Google three weeks after launch.
Read the term →