With AI chatbots, image generators, and video generators making it increasingly difficult to determine whether what you see online is real, AI detectors are more important than ever. Pangram is arguably the most well-known, courtesy of some audacious headlines and press releases, but do its claims of 99% accuracy really hold up? Based on my testing, the tool can consistently spot AI content, but it’s also somewhat trivial to fool. Below, I walk you through everything you need to know about Pangram.
Using AI to Fight AI
Pangram bills itself as “an AI detector that actually works” and, as mentioned, claims accuracy rates above 99% for text generated by just about every mainstream AI model, from ChatGPT to Qwen, and everything in between. It also has an AI image detection feature. Interestingly, Pangram itself uses proprietary AI models to power its detection tools.
The company says its AI detection capabilities are a cut above the competition because third parties have independently verified its accuracy and reliability. It also cites its ability to detect content from all major AI models, its training on diverse datasets, years of company research, vast language support, and more. Pangram even claims that its tech is better at detecting AI content than humans, including text from tools that aim to humanize AI content to evade detection.
Pangram’s mirror prompts are a clever way to train an AI detection model. (Credit: Pangram)
These are bold claims, and some of Pangram’s underlying technology is indeed innovative. Take, for example, synthetic mirroring. In these training scenarios, Pangram starts with human-generated text and then prompts an AI model to write a counterpart that matches it as closely as possible. This exercise helps Pangram hone in on the style of AI-generated text. This and other aspects of Pangram’s development sound great on paper, but real-world performance and further exploration of the numbers reveal some flaws.
Breaking the Detector: How I Fooled Pangram
If you think Pangram’s stated accuracy rates mean you’ll almost never see it get something wrong, that’s simply not the case. I’m not saying Pangram never works. It works most of the time, in fact, and it’s the most accurate AI detection tool I’ve tried. Unfortunately, that still just means it’s the best among some clearly imperfect options.
Keep in mind that Pangram requires submissions to be at least 50 words long to ensure accurate detection. To test the tool, I submitted a draft of the first section of this article after adding a sentence I generated using Gemini. Pangram failed to detect any AI content. Then, I added another sentence using Gemini, and Pangram still failed to detect it. Only after I added three sentences (going from an original 250 words to 311) did Pangram find anything amiss. However, it labeled only two sentences as AI-generated, one of which I wrote. When I rephrased the one AI-generated sentence it flagged, Pangram couldn’t detect anything.
Countless AI humanization tools are available for free online. (Credit: HumanizeAI/PCMag)
Pangram also says it can catch humanized text, but it’s easy to use humanization tools to fool it. For example, I generated a poem with GPT-5.6. Then, I searched for an “AI humanizer,” picked the first result that didn’t require an account, and had it ‘humanize’ ChatGPT’s poem. When I submitted that to Pangram, it called the poem 100% human-generated.
Picture Imperfect: Simple Edits Fool Pangram’s Image Detector
Pangram primarily focuses on AI-generated text detection, but it can also detect AI images. The tool detected most AI-generated images in testing, but it still regularly makes mistakes. Pangram’s image detection feature is still in a research preview state, so less-than-perfect results are expected.
For example, relatively believable fake images can trip Pangram up. CNET published a fun article in 2023 that aimed to test your ability to spot AI images. One image (shown below) is of avocado toast with a strange-looking lime. Pangram called this image 100% human-generated, but it’s actually AI-generated.
No, that’s not a real image of avocado toast; it’s AI-generated. (Credit: CNET/PCMag)
You can also fool Pangram by cleaning AI images with detection-evading tools. For example, I used Gemini’s Nano Banana model to crop an image. Though the image itself changed very little, editing images with Nano Banana attaches identifiable C2PA and SynthID markers. Pangram cited C2PA credentials when it called my image AI-generated. However, after I ran my edit through a free tool that makes it harder to detect and resubmitted it, Pangram considered it human-generated.
What Pangram’s ‘99% Accurate’ Fine Print Isn’t Telling You
So, if Pangram still makes mistakes despite claiming such high detection accuracy, what gives? Fortunately, Pangram provides a ton of documentation about its testing. I appreciate this level of transparency, even though I still managed to pick out some deceptive quirks.
For example, Pangram advertises 99.8% accuracy in detecting text generated by Anthropic’s Opus 5 model and cites a test it ran. But this claim doesn’t encompass Opus 5 text edited by a human or humanized by a tool, among other things. Instead, it focuses solely on what Opus 5 spits out in response to prompts. Pangram tests some of the other things I mentioned separately, but they’re buried in its documentation.
According to said documentation, Pangram can “detect humanized text as AI-generated 97.67% of the time and as either Mixed or AI-generated 98.83% of the time.” That’s different from the 99.8% accuracy figure, but it’s still close. However, other numbers indicate worse performance. For example, Pangram also measures its ability to detect AI-assisted text, which it correctly identifies only 55.01% of the time. That’s a problem when the company primarily advertises the narrow scenario in which the 99.8% accuracy applies.
Although Pangram’s numbers aren’t perfect, I have no reason to believe the company fabricated them. That’s great to see, but it only highlights the problem I’ve written about before with AI benchmarks: They just don’t translate well to real-world performance. In Pangram’s case, testing often shows truly exceptional (and improving) detection rates, but in practice, you can break the tool fairly easily.
Cherry-Picked Results: The Reality Behind Independent Verification
Beyond its in-house numbers, Pangram says it’s “proven the most reliable and accurate AI detector on the market by third-party researchers, including the University of Maryland and the University of Chicago.” This isn’t a lie, but that sentence makes the conclusions of the underlying research seem stronger than they are.
When Pangram says it outperforms humans, it only means individually, not as a group. (Credit: Pangram/PCMag)
For example, in the University of Maryland study, Pangram mentions comparing detection tools, including Pangram, against humans in spotting humanized AI-generated text. Pangram did the best among the tools and outperformed the humans individually in the test. However, humans still outperformed Pangram when voting as a group. This partially undermines Pangram’s claim that “independent studies have shown that Pangram’s AI detection outperforms trained human readers in identifying AI-generated content.”
In the University of Chicago study, Pangram mentions testing only three other AI detection tools alongside Pangram, which feels like an especially small selection given how many similar tools are available. Researchers at the Vrije Universiteit Brussel also tested only three other tools alongside Pangram, this time on lengthy academic papers. It’s not that this research doesn’t look good for Pangram, but it’s not necessarily conclusive proof of anything.
Recommended by Our Editors
Outside of research institutions, Pangram also cites other evaluations, too, such as David Gewirtz’s article on ZDNET about AI detection tools. Pangram quotes him as saying, “a newcomer to our tests that immediately soared into the winners’ circle.” While Pangram performed well on his tests, Pangram doesn’t mention that Gewirtz’s article also said that “services marketed as AI content detectors are a mixed bag” and went on to conclude “[he] would advocate caution before relying on the results of any—or all—of these tools.”
To be clear, these testimonies demonstrate good results for Pangram. I just don’t appreciate how Pangram uses them to present an unrealistic expectation of its real-world performance. It still gets things wrong regularly, and false results can create real consequences for people.
It’s increasingly difficult to distinguish what’s real online, and AI is actively making the situation worse. Tools that claim to detect AI-generated content but can’t do so perfectly just add to the confusion.
For example, Substack recently announced the ability to scan posts using Pangram’s tech to determine how likely they are to be AI-generated. But what happens if you make your living by writing on Substack and Pangram incorrectly labels your work as partially or wholly AI-generated? Alternatively, what if Pangram provides cover for writers who mix their own content with AI-generated content by labeling it 100% human-generated? Substack allows you to report and remove scans on your work you believe are incorrect, but that’s not an ideal solution.
It’s good to know if the post you’re reading is real, but what happens when the detector gets it wrong? (Credit: Substack)
And this goes well beyond Substack. Imagine a tenured English professor who doesn’t understand modern technology well, sending student essays to Pangram because their college partnered with Pangram. What happens when Pangram occasionally flags a completely authentic paper as AI-generated? Most people don’t know how to assess the reliability of AI detection tools, so it’s possible that such a scenario could result in an unfair failing grade and no recourse for the student.
These dangers are why I’m so critical of AI detection tools like Pangram. I’d much prefer if Pangram were more up-front about what it can and can’t do well.
There’s No Gold Standard: Every AI Detector Should Be Doubted
AI detection tools still aren’t reliable in real-world scenarios, despite what their marketing and research say. Sure, if Pangram or a tool like it flags something as AI-generated, it can be evidence of something that might not be authentic, but these flags aren’t infallible. And that’s essential to keep in mind, because putting too much trust in detection tools can lead to disastrous consequences. The best way to protect yourself is to learn exactly how companies derive their accuracy claims and maintain a healthy dose of skepticism about their results.
About Our Expert
Ruben Circelli
Writer, Software
Experience
I’ve been writing about consumer technology and video games for over a decade at a variety of publications, including Destructoid, GamesRadar+, Lifewire, PCGamesN, Trusted Reviews, and What Hi-Fi?, among many others. At PCMag, I review AI and productivity software—everything from chatbots to to-do list apps. In my free time, I’m likely cooking something, playing a game, or tinkering with my computer.
Read Full Bio
