aiseoaudittool.hu / Articles / Guide
Spot-Checking an Audit Tool's Findings by Hand
Tools provide a baseline, but manual verification reveals critical access errors and data contradictions that automated reports miss.
Pick the Findings That Matter Most
Automated tools scan thousands of data points, but most are trivial. Focus your manual effort on the findings that block visibility entirely. If a tool reports a high score while blocking major AI crawlers, the score is meaningless. Prioritize checks that determine whether a bot can actually enter your site.
Look for discrepancies between the tool's summary and its detailed diagnostics. A report might flag a regulated sector risk in the header while stating the sector is unregulated in the body. These internal contradictions signal a broken logic engine. Trust neither the summary nor the detail until you confirm the data source.
Ignore granular metrics like exact word counts or minor heading variations until you solve structural access issues. A perfect word count matters nothing if the crawler receives a 403 error. Start with the foundation of access, then move to content structure, and finally to optimization details.
Use the tool's output as a hypothesis, not a verdict. Treat every critical finding as a question you must answer with a direct server request. This approach saves time by preventing you from fixing issues that do not exist or misinterpreting false positives.
- Check for HTTP 403 errors from GPTBot, ClaudeBot, and Google-Extended.
- Verify that internal link counts match between summary and detail views.
- Confirm that sector classifications align across the entire report.
- Ignore word count variations until access issues are resolved.
Re-Test Bot Access Live
Many tools read the robots.txt file but never send a real request. They assume permission based on text rather than testing the server response. You must simulate a bot request to see the true status code. Use a command-line tool or a browser extension to send a GET request with a specific User-Agent string.
If the server returns a 403 Forbidden code, the bot cannot access your content regardless of what the robots.txt says. This is a common configuration error where firewall rules or security plugins block known AI agents. The audit tool might miss this because it only parses the text file, not the network handshake.
Test multiple agents separately. GPTBot, ClaudeBot, and PerplexityBot often have different rules. One might be allowed while another is blocked. A single failed test invalidates the tool's claim that your site is 'AI-ready.' Record the exact status code and response headers for each agent.
Check for contradictions between the robots.txt file and HTTP response headers. Some servers allow access in the text file but send 'noai' directives in the headers. Sophisticated crawlers follow the headers. If these conflict, the result is unpredictable indexing behavior that tools often overlook.
- Send a GET request with GPTBot as the User-Agent.
- Send a GET request with ClaudeBot as the User-Agent.
- Record the HTTP status code for each request.
- Compare the robots.txt rules against the actual HTTP headers.
Compare Raw and Rendered Content
Search engines and AI models often render JavaScript before reading content. An audit tool might analyze the raw HTML source, missing text that appears only after script execution. You need to verify that the content visible in your browser matches the content the tool analyzed.
Open your page in a browser and view the page source. Search for key phrases or headings. If they are missing from the source but visible on the screen, the tool likely missed them. This discrepancy creates a false negative where the tool thinks content is absent.
Conversely, check for content that exists in the source but is hidden from users. Sometimes developers leave debug text or duplicate content in the HTML that crawlers see but humans do not. This can dilute relevance or trigger spam filters. Manual inspection catches these hidden elements.
Use the 'Inspect Element' feature to trace the DOM tree. Ensure that headings and structured data are present in the final rendered state. If the tool reports missing headings, verify if they are generated dynamically. A manual check confirms whether the issue is technical or a reporting error.
- View the page source and search for target keywords.
- Compare the source text with the visible browser content.
- Check for hidden debug text or duplicate content in the HTML.
- Verify that headings exist in the final rendered DOM.
Check Headings and Signals Yourself
Tools often miscount headings or assign them incorrect levels. A report might claim zero H2 tags while the page clearly displays them. Manually scroll through the page and count the H1, H2, and H3 tags. Verify that the hierarchy makes logical sense.
Check for anchor IDs on headings if the tool mentions them. Some tools expect IDs for navigation or schema purposes. If the report says zero headings have IDs, inspect the code to confirm. This is a simple fix if the tool is correct, but a false alarm if the tool is wrong.
Validate structured data signals like schema markup. Tools might detect missing Person or Organization schemas. Open the source code and search for JSON-LD blocks. Confirm that the data matches the visible content. Incorrect schema can confuse AI models about your entity identity.
Ensure that internal linking signals are consistent. A tool might report zero internal links in one section and eighty-two in another. Count the links manually to resolve the conflict. Inconsistent data suggests a bug in the tool's aggregation logic rather than a problem with your site.
- Manually count H1, H2, and H3 tags on the page.
- Verify that headings have anchor IDs if required.
- Search the source code for JSON-LD schema blocks.
- Count internal links to resolve conflicting reports.
Note Where Tool and Reality Differ
Document every instance where the tool's output contradicts your manual findings. This log becomes your prioritization list. If the tool says you are in the top 6% but you are blocked by major bots, the ranking is false. Address the access issue first.
Look for external factors the tool cannot see. A tool might not know that a competitor owns the top search result for your brand name. It might calculate risk based on generic keywords. You must check search results manually to see who actually ranks for your terms.
Identify where the tool fails to crawl deep enough. It might audit the homepage but miss errors on sub-pages listed in the sitemap. Run a manual check on a random sample of internal URLs. If they return 500 errors, the tool's coverage is incomplete.
Use these differences to refine your strategy. A tool provides a broad overview, but human verification finds the critical gaps. The value lies in spotting the blind spots, not in accepting the automated score. This process turns a generic report into a specific action plan.
- Log every contradiction between the tool and manual checks.
- Search for your brand name to check for competitor dominance.
- Manually test random internal URLs for server errors.
- Prioritize fixes based on verified blind spots.
Key takeaways
- Manual verification of bot access via HTTP requests reveals errors that text-only tools miss.
- Comparing raw HTML to rendered content ensures that dynamic text is visible to crawlers.
- Internal contradictions in audit reports indicate logic errors that require human resolution.
- External factors like brand name competition are invisible to automated scanners and must be checked manually.
- Prioritize fixing access and structural errors before optimizing word counts or minor signals.
Related questions
How do I test if GPTBot can access my site?
You can test GPTBot access by sending a GET request to your homepage using a tool like curl or a browser extension. Set the User-Agent header to 'GPTBot' and observe the HTTP status code. A 200 code means access is granted, while a 403 code indicates a block.
Why do audit tools show different word counts?
Different tools use different definitions for counting words. Some count only visible text, while others include hidden metadata or scripts. This variation leads to inconsistent numbers. Manual verification using a standard text editor or browser inspector provides the most accurate count.
What should I do if my audit report contains contradictions?
Treat contradictions as a sign of a flawed report. Cross-reference the conflicting data points with your own manual checks. If the tool says you have zero internal links but you see many, trust your observation. Use the manual data to guide your fixes and ignore the erroneous metrics.
AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.