You can build a working AI visibility tracker in about 200 lines of Python. It polls Google AI Overviews for a fixed prompt set, records which domains get cited, captures your own organic position on the same query, and writes dated JSON so months compare cleanly. Our 23-prompt run cost $0.092 in API credit. The commercial tools that rank for this term start at $99–$165 a month. Here is the build, and an honest account of what it cannot do.
This guide is written from our own tracker, which we run against atilab.io. The code below is the code we actually execute — not pseudocode. Research and drafting were AI-assisted; every number is from our own run files or a source linked in-line, and a human checked each one.
What does an AI visibility tracker actually have to measure?
Three things, and most dashboards report only the first:
- Does an AI answer appear at all for the query? Trigger rate is the denominator for everything else. If AI Overviews fire on 20% of your category's queries, this whole exercise is a side project. In ours they fired on 21 of 23.
- Are you cited? Split branded from non-branded and never blend them. Branded citations tell you your entity is understood. Non-branded citations tell you whether you are in the consideration set. Only the second is a growth metric.
- Where do you rank in classic organic on that same query? This is the column almost nobody captures, and it is the one that converts "we are invisible" into a specific instruction. More on why below.
Everything else — sentiment, share-of-voice indices, a single blended visibility score — is a presentation layer over those three fields. You can add it later. You cannot add the third field later, because it has to be captured in the same request, on the same day, against the same SERP.
What does it cost to run yourself versus buy?
Our numbers, from the run files. The Google signal uses DataForSEO's live advanced SERP endpoint, which returned an API-reported cost of $0.0040 per query on every call in our run. Twenty-three prompts is $0.092. Run it monthly and the year costs about $1.10.
For comparison, published pricing on the tools currently ranking for "ai visibility tracker" (checked 12 August 2026): Rankscale lists Pro at $99/month for 1,200 credits, Growth at $385/month, Enterprise at $780/month. Semrush bundles AI visibility into its main plans — Starter at $165.17/month billed annually, with 50 prompts tracked daily, rising to 200 daily prompts on Advanced at $455.67/month.
Those are not equivalent products and we are not going to pretend they are. The subscriptions sample answers from ChatGPT, Gemini, Perplexity, Claude and others, schedule the runs, store history, and produce reports somebody else maintains. The DIY build covers two surfaces and produces a JSON file. What the cost comparison establishes is narrower and still useful: the underlying measurement is cheap. Price the subscription against convenience and coverage, not against access to data you could not otherwise get.
Step 1: build the prompt set — this decides everything downstream
The single biggest determinant of whether your tracker is useful is the prompt set, and it is the part no tool can do for you. Ours is 23 prompts: 4 branded, 19 non-branded, tagged by funnel stage and by the page each one should ideally send traffic to. Each record carries two forms of the same intent:
{"id": 6, "type": "non-branded", "funnel": "MOFU", "pri": "P1",
"target": "/ai-agent-cost",
"prompt": "What does it cost to build and run AI agents for a business?",
"query": "how much does it cost to build ai agents for business"}The prompt is the conversational form a person types into an assistant. The query is the search-shaped form you send to Google. They are different strings on purpose, and collapsing them into one field is the most common way these projects produce uninterpretable data.
Two rules that matter more than the prompt wording:
- Keep the ids stable forever. The value of this tracker is the diff between months. Change a prompt and its history is worthless; add prompts at the end with new ids instead.
- Tag each prompt with the page it should serve. When a prompt shows an AI Overview and you are absent, the
targetfield tells you instantly which page has failed, rather than starting a research project.
Step 2: query the AI Overview and capture the references
One request per query. The parameter that matters is load_async_ai_overview — without it the AI Overview block is frequently missing from the response even when it fires on the live SERP, and you will conclude your trigger rate is low when it is not.
res = dfs.call("serp/google/organic/live/advanced", [{
"keyword": p["query"],
"location_code": 2840, # US
"language_code": "en",
"depth": 20,
"load_async_ai_overview": True,
}])Then walk the items once, pulling both signals out of the same response:
for item in res[0].get("items") or []:
itype = item.get("type") or ""
if itype == "ai_overview" and not out["aio_present"]:
out["aio_present"] = True
refs = item.get("references") or []
out["aio_refs"] = [host_of(r.get("url") or "") or (r.get("domain") or "")
for r in refs]
out["aio_cites_brand"] = any(BRAND in d for d in out["aio_refs"])
elif itype == "organic" and out["organic_rank"] is None:
if BRAND in (item.get("domain") or ""):
out["organic_rank"] = item.get("rank_absolute")Note the fallback on the reference: some entries carry a url, some carry only a domain. Read one field and you will silently drop citations.
Why capturing your organic rank in the same request changes the report
Because citation is mostly downstream of ranking. In our study we took six non-branded queries where an AI Overview fired, captured all 31 citations, and checked each cited domain against the classic results for the identical query: 74% also ranked in the organic top 20, 48% in the top 10, and only 25% were cited without ranking on the first two pages. Small sample, and we would not present it as a universal law — but the direction is clear enough to plan against. Google is largely drawing its citations from a pool it has already decided to rank. A tracker that reports citations without reporting your position on the same query has hidden the causal variable, and its output is a number you cannot act on.
With both fields, every row falls into one of four cells, and each cell has a different instruction attached:
That distribution is the reason we are unsentimental about AEO tactics. Nineteen of twenty-three prompts told us the same thing: rank first. We published the full findings in our measurement of 23 AI Overviews in our category, including the queries where Google ignored vendor sites entirely.
Step 3: add a retrieval signal
The second surface worth polling is the embeddings layer that many AI applications query before generating an answer. We use Exa: send the conversational prompt form, and record whether your domain comes back and at what rank.
def check_exa(p, api_key):
out = {"exa_rank": None, "exa_url": None, "exa_top": []}
res = exa_search(p["prompt"], api_key)
for i, r in enumerate(res.get("results") or [], 1):
if BRAND in host_of(r.get("url")):
out["exa_rank"] = i
break
return outThis signal behaves very differently from the Google one and that is the point of having it. We surfaced in retrieval on 5 of 23 prompts, at ranks 1, 1, 1, 1 and 6 — four first places on a set where Google cited us three times, all branded. Retrieval visibility and citation visibility are not the same thing, and a tracker with one surface will tell you a confident, incomplete story.
Step 4: count domains per prompt, not per reference
A small decision with a large effect on the competitor map. One AI Overview citing five pages of the same domain is still one prompt's worth of visibility. Count it once:
freq = {}
for r in rows:
for d in set(r.get("aio_refs") or []): # set() — one prompt, one vote
freq[d] = freq.get(d, 0) + 1Skip the set() and any domain with a habit of multi-page citation looks like it owns your category. With it, our map reads cleanly: YouTube appeared in 11 of the 21 triggered AI Overviews, Reddit in 8, LinkedIn in 4. Counted as raw references those three domains are 37 of 174 citations, 21%. That is a meaningful minority, not the domination the common advice implies — the remaining 79% went to ordinary vendor and agency content, spread across 116 distinct domains. The citation pool is not closed, which is the encouraging half of the result.
Step 5: write dated output, never overwrite
The whole value of this thing is the month-over-month diff, so the output directory is the date and nothing overwrites anything:
outdir = os.path.join(OUTDIR, datetime.date.today().isoformat())
os.makedirs(outdir, exist_ok=True)
json.dump({"date": date, "location_code": loc, "language_code": lang,
"brand": BRAND, "results": rows},
open(os.path.join(outdir, "results.json"), "w"), indent=1)Write a flat summary.csv alongside it. JSON for diffing, CSV for the person who wants to sort it in a spreadsheet.
What broke when we ran it
Four things, all worth knowing before you spend an afternoon debugging them:
- Transient SERP API errors. The provider intermittently returns an internal server error on an otherwise valid task. Wrap every call in a retry with backoff, and return
Nonerather than crashing the run — losing one query is fine, losing the other 22 is not. - Missing AI Overview blocks. Covered above: without the async-load parameter your trigger rate will be understated.
- Reference objects with no URL. Read
urlfirst and fall back todomain. - Location and language drift. Pin them explicitly in the code, not in a config you might edit. A run at a different location code is a different study, and if it lands in the same series you will spend a month explaining a trend that is an artefact.
def call_dfs(path, payload, attempts=3):
for n in range(attempts):
try:
return dfs.call(path, payload)
except Exception as exc:
if n == attempts - 1:
return None
time.sleep(2 * (n + 1))What this build cannot measure
Be straight about this, because the vendors ranking above this page are not always. No public API exposes what ChatGPT, Claude, Gemini or Perplexity actually said to a real user in a real session. Every tool reporting "your ChatGPT visibility" is running its own prompts through an API and treating the output as a sample. That is a legitimate proxy and often a useful one. It is a sample of what a model tends to say, not a record of what your buyers were told.
You can build that proxy yourself too — send your prompt list to an assistant API, check whether your domain appears in the answer, repeat n times per prompt because the outputs vary. Just cost it honestly: a meaningful sample means several runs per prompt per platform, and that is where the DIY approach stops being nine cents and starts approaching a subscription. This is the point at which buying is a reasonable decision.
How often should you run it, and what do you do with the output?
Monthly, on the same day, with an unchanged prompt set. More frequent runs mostly measure SERP volatility. Re-run early only after a specific content or technical change you want to attribute.
Then read the report as a work queue, not a scoreboard. Rows in the bottom-right go to ordinary ranking work. Rows in the top-right — you rank, you are not cited — go to a formatting pass on an existing page, which is the cheapest work on the list. Rows in the top-left get a diary note and a re-check. That is the whole method, and it is the same discipline we apply in AI consulting engagements, where the readiness audit exists to say where AI should not be applied as much as where it should. A measurement baseline first; a build only where the baseline says one is justified. We wrote about the same trap on the delivery side in measuring AI agents on engineering teams — the metric that is easy to collect is rarely the one that changes a decision.
Frequently asked questions
What is an AI visibility tracker?
A tool that measures whether AI answer engines mention and cite your brand when people ask questions in your category. A useful one captures three fields per query: whether an AI answer triggered at all, whether you were cited, and which domains were cited instead. Adding your classic organic position for the same query turns those observations into an instruction.
Do I need to code to track AI visibility?
No — commercial tools start around $99–$165 a month and cover more surfaces than a script does. Building it yourself is worth it when you want the raw data, control of the prompt set, and month-over-month files you own. The build described here is a single stdlib Python file plus one SERP API account.
How much does a DIY AI visibility tracker cost to run?
Our 23-prompt Google run cost $0.092 in API credit, at an API-reported $0.0040 per query, plus retrieval-API usage on its own plan. Run monthly, that is a little over a dollar a year for the Google signal. The real cost is the hour you spend building the prompt set properly.
Can a tracker see what ChatGPT says about my brand?
Not in real user sessions. No public API exposes what an assistant told a specific user. Tools reporting this are sampling their own prompts through an API, which is a reasonable proxy that should be labelled as one.
Is AI visibility a separate discipline from SEO?
Mostly not, on our data: 74% of the AI Overview citations we traced also ranked in the classic organic top 20 for the same query. A real but minority share of citations goes to pages that do not rank — that is where answer-shaped formatting earns its keep. Treat AEO as a formatting and structure layer on top of ranking work, not as a replacement for it.
How many prompts should I track?
Twenty to thirty is enough to be diagnostic and small enough that you will actually keep it current. Weight it towards non-branded commercial questions, tag each prompt with the page that should serve it, and never change a prompt once it has history.
Start with the measurement
The reason we built this rather than bought it was not the $99. It was that we wanted the raw rows, including the ones that made us look bad — and on our first run, nineteen of them did. If you would like a read on where AI visibility sits against the rest of your growth priorities, book a strategy call and we will go through your own numbers rather than ours.