AI crawlers are different
ChatGPT, Claude and Perplexity each send their own bots. Many hosting providers block them by default. Your site might be invisible and nobody would tell you.
Customers are asking ChatGPT, Perplexity and Claude before they touch Google. If AI engines cannot read your site, you are not in the answer.
SEO gets you into Google results. GEO (Generative Engine Optimisation) gets you into AI answers. When someone asks ChatGPT "who does IT consulting near me?", the answer comes from sites that AI engines can read, understand and trust. Most small business websites are invisible to everything except Google.
ChatGPT, Claude and Perplexity each send their own bots. Many hosting providers block them by default. Your site might be invisible and nobody would tell you.
Like robots.txt told search engines what to crawl, llms.txt tells AI engines what your site is about, what matters, and how to cite you.
AI does not rank pages, it answers questions. If your site does not contain clear, structured answers, AI has nothing to quote.