How I track brand citations across six AI answer engines

When somebody asks ChatGPT or Perplexity who does rubbish removal in Melbourne, the answer either names my client or it does not. Rank tracking cannot see that. There is no position ten to be in.

So I built something to measure it. It puts 42 questions to every AI answer engine I have switched on, reads what comes back, and records whether the brand appeared, whether a competitor did, and which URLs were cited. One question put to one engine is a cell. The most recent sweep for It’s Done Rubbish Removal (itsdone.com.au) was 42 questions across five engines, so 210 cells, and the results feed the dashboard I use for that client.

Most of the work in it is not the asking. It is making sure the number that comes out the other end is one I would defend in front of the client.

Six engines, and what each one stands in for

The tool supports six: Perplexity, ChatGPT, Gemini, Claude, Microsoft Copilot and Grok. Five of them run for this client today, and the sixth is the subject of the next section.

Three are reached through an API. Three are reached by driving a signed-in browser and reading the answer off the page. That split is the most consequential decision in the whole thing, and it comes down to one uncomfortable fact.

perplexity/sonar is a model I can call. perplexity.ai is a product, with its own retrieval, its own index and its own system prompt sitting in front of a model. The answers correlate. They are not the same answer, and a client who reads “you appear in 30% of ChatGPT answers” is entitled to assume somebody actually asked ChatGPT.

So every engine in the report carries what it stands in for and how good the stand-in is, printed beside the number rather than buried in a methodology note. Perplexity through its API is a strong proxy for perplexity.ai. A GPT-4o search model is a medium proxy for chatgpt.com, and Gemini with grounding is a medium proxy for both gemini.google.com and AI Overviews.

Claude, Copilot and Grok are the three read through a browser, and that is what makes them strong. Signing in and reading the answer is not a proxy for the consumer product. It is the consumer product.

A missing engine is not a competitive loss

Scraping buys accuracy and pays for it in reliability.

Copilot accepts a submission and then never answers on roughly 42% of attempts, pooled over 64 live tries. That was not a selector, not slowness, not an interstitial and not throttling; I ruled out all four with evidence before accepting that the surface is simply flaky. It now gets a longer wait budget and one retry on that specific failure signature.

Grok is worse, and Grok is the engine currently switched off. It accepts the question, creates the conversation, and renders nothing: three separate probes froze at the same character count for two minutes, with the page still showing only my own question. Leaving it enabled cost about 90 seconds per question, 30 to 40 minutes of every sweep, and returned five usable cells out of 43.

Here is where a citation tracker gets to lie to its owner. The easy move is to report on the engines that worked. Drop the broken one, and the score goes up. So the measurement is built to make that impossible:

• The unit is the cell, not the citation. A brand named five times in one answer scores exactly the same as a brand named once. The answer opportunity was used once. Counting occurrences measures verbosity.
• A failed cell leaves the denominator. Only cells that returned an answer count, in the numerator or the denominator. A scrape that broke, or came back so short that it probably read the wrong part of the page, is recorded as no data. Scoring it as “brand not mentioned” manufactures a zero.
• insufficient_data is a fourth status an engine can carry, alongside leading, stable and lagging. It fires when fewer than half the attempted cells came back usable, and it short-circuits the other three, so an engine I could not collect reads as a gap on my side rather than as ground lost to a competitor.
• The engine count stays at six with Grok off. Dropping it to five would quietly flatter the tally of engines the brand appears on, by hiding one it cannot appear on.

What you ask decides what you can measure

The first question set was 20 questions, nearly all of them Melbourne-wide discovery. That is the most contested ground in the market, and a suburban operator loses it to the national franchises.

GA4 said the traffic that actually converts was arriving somewhere else. Of 29 chatgpt.com sessions and 12 key events over seven weeks, only 24% landed on a page that any tracked question covered. Suburb pages were the biggest untracked group, and a manual Perplexity check on one suburb returned the client first, on a question the tracker had never asked.

So the set was rebuilt from that evidence rather than from intuition: 36 unbranded questions and six branded ones. Every question that had surfaced the brand at least once was kept. Three that never had were kept deliberately, so a persistent zero stays visible instead of being quietly dropped. Fifteen suburbs that have produced an actual booking were added.

Then the first sweep on the new set showed that phrasing mattered more than topic. Questions opening “How do I” surfaced the brand in 2 of 35 usable cells, and only 14% of those answers named any business at all. Ask an answer engine how to get rid of something and it tells you about council collections, charity pickups and a trip to the tip, which is a good answer and a useless one to measure a business against. Questions opening “Who” surfaced the brand in 37 of 140, and 48% of them named a business. Seven questions were rewritten from one form to the other, and on the next sweep they went 5 of 26 against their predecessors’ 2 of 30.

That is three extra appearances on one sweep. Directionally right, not proof, and it is recorded that way.

It also means the rate before the rebaseline and the rate after it are not the same quantity. Two dated constants in the reporting code mark where the ruler changed, and the dashboard plots the current series only rather than drawing a trend line across two different question sets.

A brand is two YAML files

Adding one is a directory holding brand.yaml and queries.yaml: the domains, the ways the brand writes its own name, the competitors and theirs, which engines are enabled, and a per-run budget. Each brand gets its own database.

Matching is where that config earns its attention, because AI answers are not written in ASCII. Of 1,207 stored answers, 79 wrote the client’s name with a typographic apostrophe, and every one of them matched nothing. The fix is a folding table that is strictly one character to one character, so the folded text can be searched while the original is sliced for the quoted snippet, and the author’s real punctuation survives into the report.

Competitor names have the same problem in a more expensive form. One competitor writes its name with a hyphen where the config had a space. It matched nothing across 375 cells of a full month, and read on the report as a competitor with no presence at all.

The fix there was the config, not a more forgiving matcher. Stripping suffixes heuristically would have credited that competitor with 38 occurrences that were not it: a generic service phrase, and another company’s page title.

What it costs, and what runs on a Monday

The most recent sweep put 42 questions to five engines, produced 210 cells and cost $1.29 in API spend. Two of those five are browser sessions and carry no per-query fee at all.

Every run has a budget cap, and a run that hits it is marked as such and excluded from reporting rather than used up to the point it stopped. Truncation does not drop a random sample of the question list. It drops the tail, and the tail is not representative of the whole.

A cron entry runs the sweep at 09:00 on Monday and refreshes the dashboard straight afterwards. A watchdog runs at 10:15 and sends one line to Telegram: a run completed, a run failed, or a run is still going. If there was no run at all it says nothing, which is a deliberate trade. A silent Monday means either the sweep never fired or nobody looked, and the message cannot tell me which. The logs can.