How to Measure Whether AI Search Is Working

Most AI search activity never becomes a visit to your website.

Someone asks ChatGPT which supplier to use. Your brand appears in the answer. They remember the name, ask a colleague, search for you later, or make a decision without ever opening your site.

From a traditional analytics perspective, very little happened. From a marketing perspective, quite a lot did.

That gap is becoming one of the biggest measurement problems in search. AI search can now send measurable referral traffic, but analytics still only sees the part of the journey that produces a click. The better question is: is AI search making our brand more visible, more trusted and more likely to be considered? Here is a practical way to start measuring that.

Why doesn't AI search tell you everything in your traffic reports?

Because an AI answer can influence a decision without producing a session.

Google gave Analytics a dedicated way to see some of this on 13 May 2026, adding a native AI Assistant channel to the Default Channel Group. Sessions referred from ChatGPT, Gemini, Deepseek, Copilot and Grok now get tagged automatically, filed separately from ordinary Referral traffic. That's a genuine improvement, and it's still not the whole picture. Perplexity traffic currently still lands in plain Referral, and a large share of AI-referred sessions, some estimates put it at a third to two-thirds, arrive with no referrer header at all and get counted as Direct.

Google Search Console got something similar. On 3 June 2026, dedicated AI Overviews and AI Mode impression reports launched inside the Performance tab, UK site owners first. Useful, but it's impressions only for now, no clicks, no CTR, no query-level data, and AI Overviews, AI Mode and Discover's generative features are all counted in one combined bucket.

Both of those are real progress. Neither tells you how many times ChatGPT recommended your company to someone who never visited your site, or whether Perplexity cited one of your pages while the buyer stayed inside the answer. That's the gap that's left over even after the platforms catch up.

Traditional analytics was built around an observable event: someone arrives. AI search increasingly creates influence before that event happens. AI referral traffic measures clicks. AI visibility measurement needs to measure influence as well.

What should I measure instead of traffic?

Not instead of traffic. Alongside it.

I'd track three things. AI referral traffic is how many visits came from AI assistants, and what those visitors did once they arrived. Citation presence is whether your brand or your website is being used as a source in an AI answer to a relevant buyer question. Share of answer is how often your brand appears across a fixed set of important questions, compared with the alternatives your buyer is also being shown.

Traffic tells you whether somebody clicked. The other two tell you whether an AI system is treating you as part of the answer, and whether that holds up consistently over time.

There's an important detail buried in that second one: being mentioned and being cited are not necessarily the same thing. A brand can appear in an answer without its website ever being used as the source, and an AI system can cite a page without naming the brand behind it at all. Semrush's ghost citation study, run with Kevin Indig across 3,981 domain appearances, 115 prompts, 14 countries and four engines, found that 61.7% of citations were exactly this: a source link with no brand name attached to it. Perplexity was the worst offender at 52%, Microsoft Copilot the best at 19%.

That is why a useful measurement framework needs to record both.

How do you actually test AI visibility?

You build what I call a Citation Panel: a fixed set of real buyer questions, run through the same AI platforms on a repeatable schedule, with the results recorded over time. Think rank tracking, but for AI answers.

1. Write fifteen to twenty questions your ideal customer could genuinely ask, not questions designed to flatter your company. Real buyer language: "Who are the best compliance training providers in the UK?" rather than "Why is [brand] the best?"
2. Run the same questions across the same platforms every time. ChatGPT, Gemini and Perplexity are sensible starting points for most businesses; add others where your specific audience is known to use them.
3. Use a fresh session each time and strip out as much personalisation as you can, so the answer reflects what a stranger would get rather than what the model thinks you want to hear.
4. For every question, record whether the brand was mentioned, whether a source was cited, which page or domain appeared, which competitors showed up, and how the brand was described.
5. Repeat monthly, on the same date, using the same scoring each time. Consistency in method is what turns a single snapshot into a trend you can actually read.

What's the difference between being mentioned and being cited?

This is where a lot of AI visibility reporting gets fuzzy.

A mention is simple: the AI says your name. A citation is a source or reference used to support information in that answer. Neither should automatically be treated as a win. A mention can be positive, neutral or negative. A citation can point to a page that contains very little useful information about your brand. And, as the ghost citation numbers above show, an AI system can cite your website without naming your business at all.

So the useful question isn't "did AI mention us?" It's "how is AI representing us, and what evidence is it using to support that representation?" That's much closer to the real problem. AI visibility, done properly, is about being associated with the right thing, in the right context, for the right buyer, not simply about showing up somewhere in an answer.

Isn't tracking citations just another vanity metric?

It can be, if you measure it badly. So was traffic.

Traffic was never revenue. Rankings were never revenue. Impressions were never revenue. They were useful because they acted as signals further up the funnel. AI visibility is no different. You are not trying to prove that a citation equals £1,000 of revenue. You are trying to establish whether the market is increasingly encountering your brand when it asks commercially important questions. That's a meaningful signal, especially when clicks are no longer the only outcome you care about.

A professional services firm might be recommended by ChatGPT before the buyer ever searches its name. That influence may eventually show up as branded search, direct traffic, a referral, an enquiry or a sales conversation. The final conversion might be measurable. The original AI influence may not be. That doesn't make the influence imaginary. It means your measurement model needs more than one window.

Does this actually tell you anything useful month to month?

Yes, provided you're measuring consistently.

Imagine a panel of twenty questions. In January your brand appears in four of them; by March, eight. That sounds like progress until you notice only two of those eight appearances include your website as a source, three competitors are turning up on nearly every question that matters, and you're visible for informational questions but disappear the moment the question turns commercial. Those are the findings worth having, because they tell you exactly what needs fixing, and you can compare them against content work, digital PR, technical SEO, reviews and other authority-building activity to see what actually moved the number.

I've trained a dog, badly at first and then less badly, and it taught me something about measurement a dashboard never did. You don't wait for the one dramatic breakthrough to know if the training is working. You watch the small consistent signals, session after session, the same cue in the same order, and you notice when the response starts arriving half a second faster than it did last week. Checking a dashboard once a quarter and hoping it tells you something is the equivalent of waiting for the breakthrough. A Citation Panel is the small consistent signal. It moves before the big number does, if the big number moves at all.

What about traditional SEO?

Keep it. This is not a replacement for SEO.

Google's own guidance is that foundational SEO still applies to AI features such as AI Overviews and AI Mode: pages still need to be crawlable, indexable and useful, with no special technical requirements beyond being eligible for Search in the first place. The fundamentals still matter, which is a point worth its own longer answer. But the job has expanded. You're no longer trying only to rank a page. You're trying to become a source that search systems, AI systems and people all recognise as credible.

That's the shape of it: Traditional Search gets you indexed and discoverable, Discovery Platforms build distributed signals around your brand, and AI Retrieval determines whether ChatGPT, Gemini, Perplexity and Google's generative search experiences can find, understand and reference you. A Citation Panel is effectively how you watch that third layer.

Where does this leave traffic reporting?

Useful. Just no longer sufficient.

Keep measuring AI referral traffic in GA4, organic sessions, conversions and branded search. But stop asking those numbers to answer questions they were never designed to answer. Traffic tells you what happened after someone clicked. AI visibility tells you what may be happening before they do. That distinction matters, because the businesses treating AI visibility as "unmeasurable" are not necessarily wrong that it's hard to see. They're often just looking through the wrong window.

A useful measurement stack now looks more like this: visibility → consideration → click → conversion. AI search increasingly lives in the first two stages.

How do I start measuring this without buying another tool?

Build the panel yourself. Take fifteen to twenty real buyer questions, run them through three relevant AI platforms, and record mentions, citations, competitors and how your brand is described, in a spreadsheet, repeated on the same schedule every time. You'll learn more from doing that every month for six months than you will from checking ChatGPT once and declaring yourself "AI visible." Done consistently, it becomes a signal you can actually trust, not just another number to screenshot.

At Sticky Frog, I treat AI visibility as one layer of the wider Search Visibility Stack rather than a standalone trick, because the underlying goal was never to "optimise for ChatGPT." It was to build the authority, evidence and recognition that make a brand easier to find, understand and trust across search, wherever the question gets asked. That's the bigger picture behind modern search visibility.

Frequently asked questions