We repeated the same AI search questions 26 days later

A single generated answer can look authoritative. Put it into a dashboard, add a score and it can start to look like evidence of market position.

That is a weak foundation for a consequential decision.

The first round gave us an unfavourable result: letsrocc appeared in none of 60 clean, comparable answers. Rather than change the questions or look for a more flattering screenshot, we kept the four buying questions fixed and ran them again.

The test was designed to answer a narrower question:

Does the recommendation recur?

Think of one AI answer as a photograph of moving traffic. It can show what passed the camera at one moment. It cannot give you a reliable map of the road.

letsrocc remained absent.

• October baseline: 0 of 60
• November like-for-like repeat: 0 of 60
• Full November repeat: 0 of 88 valid answers
• Confirmed letsrocc citations: 0

We did not observe directional improvement in this prompt set.

That finding stops us crediting the new website, structured data, content changes or an llms.txt experiment with a result that did not occur.

It also stops us declaring that any one of those things failed. The study was not designed to isolate their individual effects.

A weak measurement becomes expensive when it starts directing work.

An organisation can commission more content, rewrite technically sound pages, buy a monitoring platform or replace a supplier because one generated answer was treated as a stable market signal.

The risk is not limited to the cost of the work. It includes the opportunity cost of solving the wrong problem, the management time spent defending the decision and the false confidence created by a cleaner report.

Before an AI visibility observation influences a budget, it needs enough context to survive three questions:

1. Does the result recur?
2. Can the source and evidence be inspected?
3. Does the observation connect to a commercial event?

Our repeat only begins to answer the first two. It does not establish the third.

A brand mention is not a recommendation.

A recommendation is not a citation.

A citation is not necessarily clickable.

A click is not an enquiry.

An enquiry is not revenue.

Revenue is not automatically contribution.

These events need separate measures because combining them into one score hides the decision a business is trying to make.

If the board wants to know whether AI answer systems are producing qualified demand, a mention rate cannot supply that answer on its own.

The systems varied in how much source evidence could be inspected.

Some ChatGPT Search and Perplexity answers exposed direct provider URLs. Google AI Mode often presented references through its own interface. Gemini frequently answered without preserving directly auditable source URLs in the captured evidence. Claude returned web-supported answers during the valid runs, but the quota ended that cohort early.

A named source does not prove that the supplier claim is correct. It gives the reader a place to begin checking it.

The repeat does not tell us why letsrocc was absent.

Competing explanations include:

• the evidence on the relevant pages;
• third-party corroboration;
• entity recognition;
• links and citations;
• geography or perceived market fit;
• freshness and retrieval timing;
• the wording of the buying question;
• system and model behaviour;
• or several of these acting together.

The data does not choose between them.

It also does not validate the claims made about any recommended supplier. The exercise recorded what the systems produced. It did not independently audit every supplier named in every answer.

This study neither validates nor disproves llms.txt.

We can publish and date a unique machine-readable page summary as an experiment. We cannot claim that the file caused a mention, recommendation or citation unless the test can separate it from content, indexing, links, external sources and changes in the answer systems themselves.

The appropriate language is experiment, not ranking lever.

The missing 12 Claude observations should be completed as a separate dated block if the account permits it. They must not be inserted silently into the 10 November result.

The next full round should retain the same four prompts, systems and reporting slots. It should follow one defined and verified evidence change rather than a bundle of simultaneous site edits.

The next comparison will continue to separate:

• mentions;
• recommendations and shortlist placement;
• citations and clickable citations;
• identifiable visits;
• qualified enquiries;
• revenue and contribution.

If a number moves, that is an observation. The cause still has to be established.

Because a measurement that cannot contradict the preferred story is a sales device, not a useful control.

The first result was unfavourable. The second did not improve it. The recommended supplier field still moved enough to show why a single-screen AI visibility report should not be treated as stable market position.

That gives us something useful: a dated baseline, a repeat protocol and a public record that can be updated without rewriting the past.

The same discipline applies to wider marketing performance. Rankings, traffic, activity and confident explanations can all move while the commercial outcome remains unchanged.