Is my website AI-ready?

1. AI crawlers can reach your pages

Open yoursite.com/robots.txt and look for rules that name GPTBot, ClaudeBot, PerplexityBot or Google-Extended, or a blanket Disallow: /. A blocked crawler can’t read your pages, so the engine behind it can’t cite them. The AI visibility checker reads your rules for each engine. To change them, the robots.txt generator writes a valid file with AI crawler options.

2. Your content is in the server HTML

Some AI crawlers don’t run JavaScript. View the page source (not the browser inspector) and check that your headings and main text are there. If the source is mostly empty and the content only appears after scripts run, those crawlers may see a blank page.

3. Structured data describes your organisation

Organization schema tells machines your name, website, logo and official profiles. Paste your page into the schema validator to see what you have and what is missing.

4. Your business is described consistently

Use the same business name and one-line description on your homepage, About page, schema and social profiles. Mixed signals make it harder for AI engines to be sure who you are. The entity SEO guide explains how to tighten this up.

5. Key pages answer questions directly

AI engines quote short, direct answers. Put the answer to a page’s main question in the first paragraph under a clear heading, then add the detail. See answer engine optimisation for examples.

6. You have considered llms.txt

llms.txt is an optional file that lists your most important pages for AI tools. Not every engine reads it, so treat it as a bonus rather than a fix. Read what llms.txt is or build one with the llms.txt generator.

Check these automatically

The Aveena website scanner checks structured data, entity signals, page structure, robots.txt and llms.txt alongside SEO, performance and accessibility, and returns a scored, prioritised list of fixes. For crawler access per AI engine, use the AI visibility checker. If you only want the AI-specific scan results, run a GEO audit. Server-rendered content and answer-first writing are still best checked by eye.

What to fix first

• Unblock AI crawlers you want to be cited by. Nothing else matters while they are blocked.
• Make sure main content renders in the server HTML.
• Add or correct Organization structured data.
• Align your name and description across the site and your profiles.
• Rewrite key pages so the first paragraph answers the main question.
• Add llms.txt once the basics are in place.

AI readiness FAQ