AI search monitoring
AI Search Monitoring That Tells You What Changed—and What to Do Next
One audit is a snapshot. Monitoring is what turns it into evidence. The basic version tracks mentions; the useful version tells you which buyer questions you newly lost, which competitor started leading, and which completed action is ready to remeasure.
What has to stay constant for a trend to be real
Four invariants. Break any one of them and the chart stops being about your brand.
- The question set is frozen and versioned. Change the questions and the trend becomes a property of your list rather than of the market. Cohorts carry a version, and a change is a recorded event: new version, pause the affected recurring measurement, require approval, resume.
- The engines and models are recorded per run. A number that moved because a provider swapped its default model underneath you is not a result about your brand. Every observation stores what was requested and what answered.
- Failures never become absences. If an engine times out, that run leaves the denominator. A provider error must never appear in a report as “the brand was not recommended”.
- Raw answers are kept. Every response is stored before it is scored, so when a detection rule improves the history is re-scored offline instead of re-purchased. Our own last full recompute of 84 stored observations cost nothing at the providers.
Unknown≠Zero
An unmeasured engine is unmeasured. Treating it as an absence is the single most common way a monitoring product manufactures a decline that never happened.
What a monitoring alert should say
The difference between a number moving and a person knowing why to open the report.
| Quality | The message | Why |
|---|---|---|
| Bad | “Your visibility score fell 3 points.” | Nothing to act on, and no way to tell noise from a real change. |
| Better | “Gemini now recommends Competitor A for a buyer question you won last week.” | Names the question and the competitor. |
| Useful | “Your ‘no monthly subscription’ positioning is still recommended by OpenAI, but Gemini and Perplexity now favour Competitor A. The same question moved on two engines.” | A person knows why to open it, and roughly what it will cost to ignore. |
The seven change types worth surfacing
Everything else is movement. The job is separating movement from meaningful movement.
1. Newly won
Recommended for a question you previously lost.
2. Newly lost
A question you won is now led by a competitor.
3. Improved
Recommendation share or engine coverage increased.
4. Declined
Recommendation share or engine coverage decreased.
5. New competitor leader
A different brand now occupies the answer.
6. Source change
The answer starts leaning on a new source, or stops citing one that mattered.
7. Positioning change
A strategic claim moves won → contested → lost, or back.
A system that reports every changed sentence trains you to ignore it. That is the failure mode, not a feature gap.
Why weekly is usually enough
AI answers can change daily; that does not mean every business needs a daily bill.
Week 0
The question set is frozen and the first run establishes the baseline. Nothing is comparable yet, and the report says so.
Weekly
The same questions, the same engines, the same protocol. A weekly cadence is fast enough to catch a real move and slow enough not to sell you noise.
On a change
A recorded action gets its own follow-up run, checked for protocol compatibility before any before/after is shown.
On a protocol break
If the cohort, providers or context moved, the trend line is cut and labelled instead of quietly continued.
For most brands, weekly measurement of 10–20 high-value buyer questions gives a better signal-to-cost ratio, especially when those questions represent category shortlists, competitor alternatives, high-value use cases, pricing criteria and strategic positioning. Daily can be added when the business case justifies it. Frequency is a cost lever, and any vendor using it as a quality signal is selling you provider calls.
Monitor recommendation and positioning, not just the brand name
The easiest dashboard metric to inflate is mention rate. Brands win because buyers associate them with something, so track the claim.
If the brand appears in a negative comparison or a caveat, it is technically visible while losing the decision. For commercial questions the monitored fields should be: mentioned, recommended, strongest competitor, recommendation share, and the state on each engine separately.
Brands also do not win by being generically known. Best for agencies. No subscription. Enterprise security. Lowest total cost. A specific vertical. Treat those claims as first-class monitoring objects and ask the sharper question: which engines understand this claim, and which recommend someone else for it?
Three stated claims contested across engines; one lost on all three.
In our founder-owned OwnQR validation, “lifetime access”, “no monthly subscription” and “one-time payment” came back recommended by OpenAI and lost to the same competitor on Gemini and Perplexity. “Editable destination after printing” — the claim the brand considers its differentiator — was lost on every measured engine.
Connect monitoring to actions, or it is just a bill
The biggest retention failure in visibility software is a dashboard that creates no workflow.
The loop that earns a renewal is gap → action → work completed → remeasure → result. An action should retain the original opportunity, the buyer questions affected, the target asset or source, the evidence, its status, the completion date, the baseline measurement and the follow-up. That is what lets monitoring answer “did the measured state change?” instead of regenerating the same recommendation every week.
Recurring monitoring
Re-runs the approved cohort on a schedule to find market change.
Targeted remeasurement
Re-runs only the questions tied to a completed action, under the original compatible protocol, to close the proof loop. It should not share an entitlement with a full recurring scan.
Before and after, without the causal overclaim
Suppose a brand edits a pricing page. Before: the competitor wins on all three engines. After: the brand wins on two.
Before / after · same question, compatible protocol
Illustrative example“Which [category] platform has no monthly subscription?”
Baseline
Follow-up
Valid
“Recommendation coverage improved from 0 of 3 to 2 of 3 engines after the recorded change.”
Invalid
“The pricing-page change caused two engines to recommend the brand.”
Before and after≠Causality
AI systems change on their own. An observational before/after is directional evidence, not a controlled experiment — and if the cohort, providers or context moved enough to make the two runs incompatible, the honest output is a refusal with a reason, not a delta.
What runs by hand today
We would rather lose a deal over this table than win one and have you discover it in week two.
| Capability | Status |
|---|---|
| Frozen, versioned buyer-question cohorts | Live |
| Three engines, reported separately, never averaged | Live |
| Recommendation separated from mention and consideration | Live |
| Failure-aware denominators; unmeasured never counted as absent | Live |
| Raw answers retained; offline re-scoring at no provider cost | Live |
| Compatible before/after comparison, with incompatible runs refused | Live |
| Competitor occupancy and citation-source separation | Live |
| Strategic positioning tracked per engine | Live |
| Automated recurring schedule | Not yet — re-runs inside a pilot happen because we run them |
| Alerts and change emails | Not yet — changes are reported in the deliverable |
| Self-service dashboard, login, agency portfolio view | Not yet — pilots are founder-reviewed report files |
| Public API, integrations, SSO, white-label | Not yet |
Who needs this
- Agencies — monitor several client brands and know which one needs attention. See AI visibility for agencies.
- In-house SEO teams — track buyer-question outcomes next to rankings and Search Console.
- Product marketing — check whether strategic positioning is understood consistently across engines.
- Founders — catch a competitor taking over a decision criterion before it shows up in pipeline conversations.
AI search monitoring is the broad recurring answer-measurement workflow. AI brand monitoring is the brand-focused application of it, weighted toward representation, factual accuracy, positioning and competitor narratives. Rank tracking asks what position a URL holds in a result page; AI monitoring asks what the generated answer decided.
Start with questions that can change a decision
Build the first set from ten to twenty questions, then run them consistently.
- best [category] for [ICP]
- alternatives to [competitor]
- [you] versus [competitor]
- most affordable [category]
- [category] with [strategic capability]
- “which [category] do you recommend?”
Then turn the results into metrics with AI search analytics, and into a decision with the tool.
Common questions
What is AI search monitoring?
AI search monitoring is the repeated tracking of how AI systems answer a fixed set of relevant questions, including brand recommendations, competitors, citations and what changed since the last compatible run.
How often should AI visibility be monitored?
Weekly is a practical default for most brands. Daily can make sense for larger or faster-moving programmes. The cadence should match the value of the questions and the cost of repeated measurement — frequency is a cost lever, not a quality signal.
Why do AI answers change?
Models, retrieval, source freshness, prompt phrasing, location, context and ordinary stochastic generation can all move an answer. That is why a single run is a sample rather than a rank.
Can I compare AI visibility month over month?
Only if the question cohort and the measurement configuration are compatible. If prompts, providers or context changed materially, the comparison should be refused and the reason shown, rather than a misleading delta being displayed.
What should I monitor besides mentions?
Recommendation rate, citations, the strongest competitor, source changes, strategic positioning by engine, engine disagreement, and whether a completed action has been remeasured under a compatible protocol.
How we measure, and how this page is sourced
What we actually measure
Every result comes from the official OpenAI, Google Gemini
and Perplexity APIs with web search or grounding enabled. In our data those
surfaces are recorded as openai_api_web_search,
gemini_api_google_search and perplexity_api_sonar, and reports name
them explicitly.
That is not the same thing as a logged-in person using the ChatGPT app. These are official interfaces with web search on, but they are measurement surfaces rather than consumer sessions: no personalisation, no memory, no app-only features. Results can differ from what any individual user sees, and we do not claim the two are identical. We would rather put that on the pricing page than in the small print.
How this page is sourced
- Vendor pricing and features come from each vendor's own public pricing page, with the date we checked it. We do not copy figures out of other people's comparison posts.
- Google-specific claims come from Google Search Central, not from SEO commentary about Google.
- Research claims name the paper and separate what was demonstrated in a benchmark from what holds in the open web.
- First-hand numbers come from our own measurement runs. Where the brand measured is founder-owned, the page says so in the same sentence as the finding.
This page is maintained by the person who built the measurement engine. There is no editorial team and no external reviewer behind it, and we are not going to invent one for a byline. How the measurements are collected.