Monitoring Methodology
Last updated: 2026-09-20 · This page fully documents how AIHonest measures. The biggest problem in this industry is black-box numbers — we do the opposite.
Our position: a mention rate you can't explain is worthless. This page describes how we measure, how we remove interference, and how we handle randomness — use it as the benchmark against any competitor's report.
1. Dual engine methodology: knowledge-type vs retrieval-type
Different AI engines "mention" brands through completely different mechanisms. Averaging them into one number is a common industry mistake:
| Type | Engines | Mechanism | Interpretation |
|---|---|---|---|
| Knowledge-type | DeepSeek | No live retrieval; answers reflect brand impressions in training data | The model "remembers" you — slow-moving, reflects long-term accumulation |
| Retrieval-type | Doubao / Kimi / ChatGPT(:online) | Retrieves the live web before answering | Reflects the current web ecosystem around you |
- Every engine on the dashboard carries a methodology badge; mention rates are computed and tracked separately.
- Retrieval-type mentions are more exposed to corpus pollution (section 4); knowledge-type mentions shift over weeks — the optimization strategies are entirely different.
2. Echo-contamination removal
The problem: when a monitoring prompt itself contains the brand name, models sometimes echo the prompt back ("regarding aihotrank.com, you could…"). Naive tools count that as an organic mention.
Our measurement: without removal, DeepSeek's false mention rate reached 33% — one in three "mentions" was an echo.
The rule: if a brand mention in the answer has ≥12 characters of contiguous overlap with the prompt text, it is classified as an echo and excluded from organic mention rates. Applied automatically on every run.
3. Stable mention rate (multi-sample consistency)
- Raw mention rate: single-sample result. Volatile due to model randomness.
- Stable mention rate: the same prompt is sampled N times (Pro: N=2); a brand counts as stably mentioned only if mentioned in
N-1 or moresamples. - On 1-sample plans (Free/Starter), stable = raw.
- How to read them: raw for trends, stable for true standing — competitor reports typically show only the former.
4. Corpus pollution detection
Background: GEO spam campaigns pollute retrieval-based engines' corpora (see our teardown). In polluted categories, mention rates are distorted by ad spend.
v0 signals (based on the last 7 days of citations):
- Low-authority domain share: proportion of citations not from a whitelist of major media/communities
- Cross-engine overlap: how much multiple engines cite the same set of low-authority domains
- When both exceed thresholds, the dashboard shows a "retrieval corpus possibly polluted" warning
v0 rules are deliberately simple (whitelist + overlap) and will keep evolving. Full citation structure is always visible on the Citations page.
5. Cost & scheduling
- Every monitoring run makes real engine API calls (prompt × engine × samples); usage is logged per real call including failures.
- Engine scheduling is transparent: latency and cost references per engine are visible on the dashboard.
- Monthly quotas are hard limits per plan; the dashboard warns as you approach them.
6. What this page doesn't cover
- Matching thresholds and weights may adjust as the engine ecosystem changes — updates are dated on this page and history is auditable via git.
- Questions about methodology? Email support@aihonest.ai, or test it yourself in our verification tool (coming soon).