This 28-point checklist covers crawlability, content structure, entity signals, schema, third-party mentions, llms.txt and monitoring. Each check is a concrete action you can verify with the free tools linked in each section. Run the AI Visibility Checker first to confirm bot access.
Section 1: Crawlability and access (checks 1-6)
Crawlability is the gate – if bots cannot reach your pages, nothing else matters. The table below maps each check to the bot it affects and the tool to verify it.
| Check | Bot affected | Verification tool |
|---|---|---|
| 1. robots.txt allows search bots | OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot, Applebot | AI Visibility Checker |
| 2. No WAF/CDN blocks | All AI bots | WAF logs, AI Visibility Checker |
| 3. HTTP 200 from key pages | All search bots | AI Visibility Checker |
| 4. No 429 rate limits | All AI bots | AI Visibility Checker |
| 5. Applebot follows Googlebot rules | Applebot | robots.txt test |
| 6. No CAPTCHA blocks | All AI bots | Manual test |
- Verify robots.txt allows OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot and Applebot. Block only training tokens (GPTBot, ClaudeBot, Google-Extended, Applebot-Extended) if that is your policy. For the complete allow/block matrix, see robots.txt for AI crawlers.
- Confirm no WAF or CDN managed rules block AI bot user agents. Cloudflare defaults to blocking AI crawlers on new domains.
- Test that each search bot returns HTTP 200 from your key pages. Use the AI Visibility Checker to send real GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot and Googlebot user agents.
- Check that your server does not return 429 (rate limit) to AI crawlers. A 429 often means edge rate-limiting, not your robots.txt.
- Verify Applebot follows Googlebot rules if Applebot is not explicitly mentioned in robots.txt (Apple support article).
- Ensure no CAPTCHA or challenge page blocks automated fetches. Anthropic states their crawlers do not circumvent CAPTCHAs.
Section 2: Content structure and clarity (checks 7-13)
- Every important page has a single H1 that matches the primary query intent.
- Key answers appear in the first 100 words (the “direct answer” paragraph) for featured-snippet and AEO eligibility.
- Use H2 sections written as questions people actually type (e.g., “How do I block GPTBot but keep ChatGPT search?”).
- Include at least one comparison table (3-5 rows) and one bulleted or numbered list per major page.
- Define entities explicitly: organization name, founder names, product names, service names, locations.
- Add author bylines with real names and links to author bio pages that include sameAs schema.
- Publish date and last-updated date visible on every article, using ISO 8601 in schema.
Section 3: Entity and brand signals (checks 14-18)
- Organization schema on homepage with name, url, logo, sameAs array (Wikipedia, Crunchbase, LinkedIn, GitHub, Twitter/X).
- Claim and verify your Google Business Profile and Bing Places listing with consistent NAP.
- Wikipedia page or Wikidata entry for your brand (not guaranteed but high-value if notable).
- Consistent brand name, tagline and description across homepage, about page, schema, social profiles and directories.
- Knowledge Panel eligibility: check if your brand triggers a Knowledge Panel in Google search for the brand name.
Section 4: Schema markup (checks 19-22)
- Article or BlogPosting schema on all content pages with headline, description, author, datePublished, dateModified, publisher, mainEntityOfPage.
- FAQPage schema on pages with FAQ sections – questions in h3, answers in p, matching JSON-LD exactly.
- Product or Service schema on commercial pages with price, availability, review aggregateRating if applicable.
- BreadcrumbList schema on all pages deeper than the homepage.
Section 5: Third-party mentions and citations (checks 23-25)
- Earn mentions on sites that AI training data likely includes: Wikipedia, major news outlets, government .gov sites, university .edu sites, industry association sites.
- Build citations in structured directories: Crunchbase, G2, Capterra, Clutch, GoodFirms, industry-specific directories.
- Publish original research, data or methodology that others cite. AI models favor primary sources with unique data.
Section 6: llms.txt and markdown alternatives (checks 26-27)
- Publish a valid llms.txt at your root with H1 (site name), blockquote summary, and H2 file list sections linking to your most important pages. Generate it from your sitemap with the llms.txt Generator. For the honest evidence on whether llms.txt works, see Does llms.txt work?.
- Serve .md versions of key pages (page.html.md or page.md) with rel=”alternate” type=”text/markdown” and rel=”describedby” link headers.
Section 7: Monitoring and iteration (check 28)
- Track AI referral traffic in GA4 (source/medium contains “chatgpt”, “perplexity”, “claude”, “gemini”, “copilot”). Monitor monthly. Run the SEO Audit quarterly for a 100-point technical, on-page, speed, security and GEO score with prioritised fixes.
How to use this checklist
Work through sections in order. Crawlability (Section 1) is the gate – if bots cannot reach your pages, nothing else matters. Content structure (Section 2) and schema (Section 4) are the highest-leverage on-page wins. Entity signals (Section 3) and third-party mentions (Section 5) compound over months. llms.txt (Section 6) is hygiene – do it once, update when site architecture changes. Monitoring (Section 7) closes the loop. Tick each check when verified. Re-run the AI Visibility Checker after fixes to confirm bot access.
Frequently asked questions
How many of these checks do I need to pass to get cited?
There is no threshold. Crawlability checks (1-6) are mandatory – if bots cannot fetch, you will not be cited. Schema and content structure checks (7-13, 19-22) have the strongest correlation with citations in our testing. Entity and third-party signals (14-18, 23-25) build authority over time.
Can I skip llms.txt and still get cited?
Yes. llms.txt has no proven impact on citations. It is hygiene for tooling and developer-facing brands. Prioritize schema, content clarity and crawlability first.
How often should I re-run the AI Visibility Checker?
After any robots.txt change, WAF rule change, CDN config change, or major site migration. Monthly as routine hygiene. The checker tests live HTTP responses from real bot user agents. For tracking which bots have visited, see Check AI Crawler Visits.
What is the difference between this checklist and a standard SEO audit?
Standard SEO audits focus on Googlebot. This checklist adds AI-specific bot access (OAI-SearchBot, PerplexityBot, Claude-SearchBot), training-token control, llms.txt, .md page versions, and AI referral tracking. The SEO Audit includes both traditional and GEO checks.
Do I need all four schema types (Article, FAQPage, Product, BreadcrumbList)?
How do I earn third-party mentions that AI models recognize?
Publish original data, methodologies or frameworks. Pitch journalists with exclusive findings. Speak at conferences. Contribute to open-source projects. Get listed in industry directories. AI training data over-represents authoritative, frequently-cited sources.
What if my site fails the crawlability checks on a CDN or WAF?
Create a custom WAF rule allowing the AI bot user agents combined with verified IP ranges (OpenAI, Perplexity, Anthropic, Google, Apple publish their IP lists). Set the allow rule priority above blocking rules. Perplexity documents this exact pattern for Cloudflare and AWS WAF.
Where do I start if I have limited time?
Run the AI Visibility Checker first. Fix any bot returning non-200. Add Article and FAQPage schema to your top 10 traffic pages. Publish llms.txt from your sitemap. Those three steps cover the highest-impact, lowest-effort items.
