Can You Trust AI Search Citations? What the Audits Found for ChatGPT, Perplexity, Gemini and Grok
AIHub
AI answers with little numbered links look more trustworthy than answers without them. The links suggest someone checked. Two independent audits looked at whether those links actually hold up, and the short version is that a citation tells you where a model says it got something, not whether it got it from there.
Here is what each audit measured, what it found per tool, and a five-minute routine for checking citations on the answers you actually rely on. The full breakdown of both studies, with every figure footnoted to the original papers, is in AEO Footnotes’ write-up on how often AI search engines cite the wrong source.
Audit 1: can the tool find the source of a quote?
The Tow Center for Digital Journalism at Columbia tested eight AI search tools in February 2025: ChatGPT Search, Perplexity, Perplexity Pro, DeepSeek Search, Microsoft Copilot, Grok-2, Grok-3 and Google’s Gemini. The researchers took ten articles each from 20 news publishers, pulled a direct excerpt from every article, and asked each tool for the article’s headline, publisher, publication date and URL. Every excerpt was one that a plain Google search found within the first three results, so the task was not hard.
| Finding | What the audit reported |
|---|---|
| Overall error rate | More than 60% of queries answered incorrectly across all tools |
| Best performer | Perplexity, 37% incorrect |
| Worst performer | Grok 3, 94% incorrect |
| Broken or fabricated links | More than half of Gemini and Grok 3 responses; 154 of 200 Grok 3 citations |
| Misattribution | DeepSeek named the wrong source 115 times out of 200 |
| Hedging | ChatGPT was wrong on 134 articles and signalled low confidence only 15 times |
| Paid versus free | Perplexity Pro and Grok 3 had higher error rates than their free versions |
The paid-versus-free row is the one most people get wrong. The pro tiers did answer more prompts correctly, but they also stopped saying “I don’t know” and gave confident wrong answers instead, so the share of wrong answers went up. Copilot was the only tool that declined more questions than it answered.
Two other patterns matter if you use these tools for research. They often cited syndicated copies on Yahoo News or AOL instead of the original publisher, even where the publisher had a deal with the AI company. And blocking a crawler did not keep content out: Perplexity’s free version correctly identified all ten excerpts from paywalled National Geographic articles even though National Geographic disallows Perplexity’s crawlers.
Audit 2: does the citation support the sentence next to it?
The second audit, by Liu and colleagues at Stanford, asked a different question. Not “is this the right source” but “does the cited page actually say what the sentence claims”. The team had 34 human annotators check answers from Bing Chat, NeevaAI, Perplexity and YouChat on 1,450 queries, scraped between late February and late March 2023.
On average, only 51.5% of generated sentences were fully supported by their citations, and only 74.5% of citations supported the sentence they were attached to. In other words, about one citation in four pointed at a page that did not back the claim.
The spread between tools was wide. Bing Chat had the highest citation precision at 89.5, YouChat the lowest at 63.6. On recall, how many claims were properly supported, Perplexity scored 68.7 against YouChat’s 11.1.
The uncomfortable result: the answers were still rated highly fluent (4.48 out of 5) and useful (4.50). The authors found that citation precision and perceived usefulness moved in opposite directions, with a correlation of minus 0.96. Answers that stick closely to their sources read as less helpful, and answers that read as most helpful drift furthest from them.
The two audits are not directly comparable
The Stanford data is from early 2023 and the Tow Center data from February 2025, on mostly different tools, and both have been updated many times since. Treat the numbers as a baseline for how wrong citations can be, not as a current leaderboard. Both audits point the same way, though: low accuracy, hidden behind confident and fluent presentation.
Why the same question gets different sources in different tools
Part of the answer is which crawlers a site lets in. Each AI company runs separate bots for training, for search and for fetching a page when a user asks, and websites can allow or block each one in robots.txt. A site that blocks OpenAI’s OAI-SearchBot will not appear in ChatGPT search answers, while the same site may still show up in Perplexity or Claude if their search bots are allowed. AEO Footnotes keeps a reference of every major AI crawler by purpose, with the robots.txt line that controls each.
So when two tools give you different sources for the same question, it is not always that one searched better. They may simply be allowed to read different parts of the web.
A five-minute citation check
The Tow Center method doubles as a routine anyone can run on the answers that matter.
- Click every link. Fabricated and broken URLs were common for some tools. A citation that names a real publisher can still lead to an error page.
- Find the sentence on the page. Open the cited page and search it for the specific claim. If you cannot find it, the citation is decoration.
- Check whether it is the original. If the link goes to an aggregator or a republished copy, look for the original publisher. Dates and corrections often differ.
- Ask a second tool. Send the same question to a different search-grounded model and compare which sources each one cites. Agreement on the same primary source is a good sign. Different sources for the same claim means read both.
- Watch for confidence without hedging. The audits found that the wrong answers were usually stated just as firmly as the right ones. Confident tone is not evidence.
Step 4 is where most people give up, because it means a second tab, a second subscription and pasting the question twice. In AIHub you can pick Perplexity Sonar or Sonar Pro, Perplexity’s search-grounded models, next to GPT, Claude or Gemini, and send one question to both at once with the answers side by side. The method for doing that properly, with a fixed rubric and blind scoring, is in how to compare AI models side by side, and the broader picture of which model is strongest at which job is in ChatGPT vs Claude vs Gemini.
The bottom line
AI search is useful for finding where to look. It is not yet reliable for telling you what a source says. Both audits found that a large share of citations were wrong, broken or did not support the claim, and that the tools rarely signalled when they were unsure. Use the citations as leads, click through before you repeat anything, and compare two tools when the answer matters. More on choosing and comparing models on the AIHub blog.
