AI search analytics
AI Search Analytics: Measure the Answer, the Buyer Decision, and the Change
A brand can be mentioned but not recommended, recommended without being cited, cited but described inaccurately, strong in ChatGPT and weak in Gemini, visible on educational prompts and absent from purchase questions. That is why AI search analytics needs more than one score.
The five distinctions that prevent bad decisions
Each one is a place where a dashboard can be technically correct and commercially misleading.
Mention≠Recommendation
“Vendor A is the best fit; Vendor B is cheaper but lacks the required workflow” credits both brands equally, and the losing brand reads its own defeat as coverage.
Recommendation≠Commercial importance
Winning “what is [category]?” is scored like winning “which vendor should I choose for a 100-person company?”. Prompt volume then overvalues informational coverage.
Citation≠Actionability
A source appears constantly and gets turned into a task, even though there is no legitimate route for you to participate in it.
Owned evidence≠ProductTruth
A capability missing from a page becomes “the product does not have it”, and a claim written on a page becomes current fact. Both inferences are wrong often enough to damage a client.
Before / after≠Causation
Visibility improves after an edit and the edit takes the credit. AI systems change on their own; an observational delta cannot separate the two.
A practical measurement model, in six layers
Everything above layer 1 is derived — and anything derived from a discarded observation cannot be re-derived later.
Layer 1 · Questions
A frozen, versioned set of buyer questions. Everything above this layer is only as good as this one.
Layer 2 · Observations
One stored record per question, per engine, per run — including the runs that failed.
Layer 3 · States
Named, cited, recommended, absent — detected separately, never collapsed into a single score.
Layer 4 · Evidence
Winning-answer themes, cited sources, owned-page evidence and product facts, kept in separate categories.
Layer 5 · Scores
Commercial gap and action priority, computed as two independent axes.
Layer 6 · Change
Only comparable re-runs. If the protocol moved, the comparison is refused rather than reported.
Layer 1 — Observation
The raw unit is one answer to one question from one provider and model, in one context, at one time. Persist the question, the engine, the run, the response, the citations, the recommendation classification and the provenance.
Layer 2 — Aggregation
Across compatible observations only: recommendation share, mention share, citation coverage, strongest-competitor share, provider breadth, and the count of classifiable versus unclassifiable runs. That last pair is the one most dashboards omit, and it is the one that tells you whether to believe the rest.
Layer 3 — Commercial relevance
Classify the question: purchase and vendor selection, comparison and alternatives, solution evaluation, category research, educational, navigational. Demand data can enrich this where it exists — and where it does not, demand unknown must not become demand zero.
Layer 4 — Commercial gap
Is the brand losing a commercially relevant buyer question, and how strong is the evidence? This is deliberately a separate question from how easy the gap would be to fix.
Layer 5 — Actionability
Inspect the owned assets, the verified product truth, the source addressability, the evidence gaps and the confidence. Only then decide whether the opportunity should be prioritised now.
Layer 6 — Proof and history
After a recorded change, run a compatible follow-up, compare the same question under the same protocol, display the measured delta, and preserve the reason when a comparison is refused as incompatible.
The metrics we treat as first-class
Eight units, each defined so that a client can challenge the number and get an answer.
Recommendation share
The proportion of compatible observations in which the brand was actually recommended. A buyer can see you without the answer favouring you.
Competitor recommendation share
How often the strongest competitor wins the same question. A 0% brand rate means one thing when nobody is recommended and something far worse when one rival wins every run.
Commercial Gap Score
The commercial importance and severity of a measured recommendation gap, dampened by evidence confidence. Deliberately separate from fixability.
Action Priority
Whether the opportunity is worth acting on now, based on addressability evidence. A gap can be commercially critical and still have no action priority.
Positioning state
For approved strategic claims: won, contested or lost — shown per engine, because that is where the disagreement lives.
Evidence coverage
How much of a finding traces back to raw observations, winning-answer themes, owned assets, product truth and sources.
Proof rate
How many completed actions were actually remeasured under a compatible protocol. A better retention metric than dashboard sessions.
Confidence
Compatible runs, provider coverage, consistency, missing engines and evidence quality. It must reduce certainty and can never strengthen a weak finding.
Why one composite score is not enough
Composite scores are useful for scanning a trend and dangerous when they become the decision.
The failure case, concretely
Ten educational prompts improve while two high-intent buyer questions decline. A weighted overall score rises while commercial performance gets worse. Any rollup worth shipping has to let you open the buyer-question outcomes underneath it.
The same applies to prompt count. Tracking a thousand prompts sounds impressive, but the value of a monitoring programme depends on the relevance of the prompts, not the count. Use large prompt databases for discovery, market coverage and research. Use curated buyer-question cohorts for recurring monitoring, decisions and proof. They are different instruments and the price difference between them is not a quality difference.
Partial provider failure is not brand absence
This is the subtlest analytics bug in the category.
Unknown≠Zero
If an engine fails to return a usable response, the correct state is unmeasured / unclassifiable — never brand absent. The same principle covers failed API calls, inaccessible source pages, missing product truth and unverified source routes. A system that quietly treats unknown as zero will report a bad API day as a collapse in your market position.
Source analytics: relevance versus addressability
Two dimensions that must never be allowed to contaminate each other.
| Dimension | What it asks | Signals |
|---|---|---|
| Source relevance | How strongly is this source associated with the observed answers? | Citation frequency, provider breadth, recurrence across buyer questions. |
| Source addressability | Is there evidence the brand can reasonably act on it? | A public listing or profile route, clear eligibility, a known claim workflow, a verified editorial path. |
A source can be highly relevant with completely unknown addressability, and citation frequency must never be allowed to raise the addressability score. That rule is enforced in the engine, not left to the person writing the report.
Owned-site analytics and ProductTruth
Inspect the pages you control, then check the product fact before recommending any claim.
For each buyer question, match the most relevant inspected assets and classify the evidence as present, partial, not found in the inspected set, or unknown — then compare only against the evidence themes actually observed in answers that won. Comparing every page against a universal checklist generates work rather than findings.
| ProductTruth state | What it means | What the system may recommend |
|---|---|---|
| Verified true | The capability exists and has provenance. | A communication action on a verified URL. |
| Verified false | The capability does not exist. | An explicit “do not make this claim”, and a different criterion to compete on. |
| Unknown | Nobody has confirmed it. | A verification task. No page is named until it is settled. |
AI search analytics and first-party analytics
Answer measurement tells you what AI systems say. First-party analytics tells you what happened on your own site. Neither replaces the other.
The useful long-term view is AI answer state → referral behaviour → owned-site conversion, with each layer labelled according to what it actually proves. Do not merge them into an attribution model that implies causation nobody measured.
The dashboard hierarchy we would build
Insight before analytics. Today that hierarchy is the running order of a report file rather than a screen you log into, and it is stated that way on purpose.
- The most important current decision
- Strategic positioning risks
- Biggest commercial gaps
- Best next actions
- What changed since the last compatible run
- Secondary metrics — mentions, citations, share of voice, provider distribution
Common questions
What is AI search analytics?
AI search analytics measures generated-answer outcomes across AI systems: brand mentions, recommendations, citations, competitors, prompt-level performance and historical change. The design question is not which chart to draw but which units you refuse to collapse into one number.
What are the most important AI visibility metrics?
For a commercial programme: recommendation share, competitor recommendation share, buyer-question importance, engine coverage, citation and source evidence, confidence, and measured change after a completed action.
Is share of voice enough?
No. Share of voice is useful context but hides which prompts are commercially important and whether the brand was actually recommended rather than merely named.
How is this different from Google Analytics?
Google Analytics measures behaviour on properties you own. AI search analytics measures the generated answers that happen before or entirely outside that visit. Neither replaces the other, and joining them into a causal chain requires care.
Can AI analytics show revenue?
First-party analytics can show revenue from tracked sessions. AI-answer measurements can be correlated with those outcomes, but observational data should never be presented as causal attribution.
How we measure, and how this page is sourced
What we actually measure
Every result comes from the official OpenAI, Google Gemini
and Perplexity APIs with web search or grounding enabled. In our data those
surfaces are recorded as openai_api_web_search,
gemini_api_google_search and perplexity_api_sonar, and reports name
them explicitly.
That is not the same thing as a logged-in person using the ChatGPT app. These are official interfaces with web search on, but they are measurement surfaces rather than consumer sessions: no personalisation, no memory, no app-only features. Results can differ from what any individual user sees, and we do not claim the two are identical. We would rather put that on the pricing page than in the small print.
How this page is sourced
- Vendor pricing and features come from each vendor's own public pricing page, with the date we checked it. We do not copy figures out of other people's comparison posts.
- Google-specific claims come from Google Search Central, not from SEO commentary about Google.
- Research claims name the paper and separate what was demonstrated in a benchmark from what holds in the open web.
- First-hand numbers come from our own measurement runs. Where the brand measured is founder-owned, the page says so in the same sentence as the finding.
This page is maintained by the person who built the measurement engine. There is no editorial team and no external reviewer behind it, and we are not going to invent one for a byline. How the measurements are collected.