We believe it’s crucial for AI detectors’ accuracy to be transparent. Users deserve to know which detector is the most accurate for their use case. Learn about why AI detection matters and how to measure it in this guide.
The world needs reliable AI detection tools; however, no AI detection tool is ever going to be 100% perfect.
It’s important to understand their limitations so that you can use them responsibly.
What does this mean for developers of AI detectors? They should be as transparent as possible about the capabilities and limitations of their detectors.
At Originality.ai, we believe that transparency is a top priority.
So, below we’ve included our analysis of Originality.ai’s AI detector efficacy, including accuracy data and false positive rates.
Here’s what you’ll find in this guide:
Originality.ai AI Detector model release history
Originality.ai AI Detector accuracy
Why is it important for AI detectors to release studies?
How do AI detectors check for accuracy? (learn about confusion matrices)
How AI detection (and the Originality.ai AI Detector works)
Our view: AI detectors in academia and false positives in general
GPT-2, GPT-NEO, GPT-J, and GPT-3 accurate detection. But was able to be “tricked” with Paraphrasing
First GPT-3 trained detector
First commercially available AI detector
1.4 – Apr 2023
Improved ChatGPT detection
Accurate on GPT4-generated content
Only tool capable of accurately detecting paraphrased content.
Reduced the number of false positives with increased training on human-generated content
2.0 Standard — Aug 2023
Reduced False Positives
Improved Accuracy on the Hardest to Detect AI Content (GPT4, ChatGPT & Paraphrased)
Release of Open Source Benchmark Dataset.
Release of Open Source AI Detection Efficacy Testing Tool(s).
Between 1.4 and 2.0, our team built many models, which slightly increased AI detection capabilities, but we weren’t going to release a model until it materially reduced false positives.
September 2024 Update: This model has been retired from our platform. Sign up to try our latest model!
3.0 Turbo — Feb 2024
Trained on the newest LLMs (Grok, Mixtral, GPT-4 Turbo, Gemini, Claude 2)
Accuracy increased on our toughest testing dataset from 90.2% to 98.8%
False Positives have been slightly improved from 2.9% to 2.8%
2.0.1 Standard (BETA) — July 2024
Improved version of the flagship 2.0.0 Standard model (BETA).
September 2024 Update: Thank you for your feedback! BETA testing has now concluded, and with Lite being your top choice, this model has been retired.
1.0.0 Lite — July 2024
Highly accurate with 98% accuracy (under 1% false positive rate).
Allows for lightly AI-edited content (like Grammarly’s grammar and spelling suggestions) while still differentiating between light AI editing and fully generated AI content.
This model has retired: September 2026
3.0.1 Turbo — October 2024
99%+ accuracy in detecting AI content (under 3% false positive rate).
Best for use cases where there’s a0-tolerance policy for AI content.
Robust against bypassing methods — extremely challenging to bypass.
Multilingual 2.0.0 — May 2025
The Originality.ai Multi Language 2.0.0 model now supports 30 languages!
Notable improvements with an overall accuracy of 97.8%
Reduced false negatives to 1.99% and lowered false positive rate to 2.4%
1.0.1 Lite — June 2025
Improved 99%+ accuracy on all leading flagship AI models from OpenAI, Gemini, Claude, and Deepseek (exceptionally low false positive rate, now 0.5%).
New, more robust capabilities in identifying content from the latest AI Humanizer tools.
1.0.2 Lite — September 2025
Exceptional 99% accuracy on leading flagship AI models from OpenAI, Anthropic/Claude, Gemini, and DeepSeek.
Maintains low false positive rates of 0.5% to recognize human-written content as human.
This model has retired: September 2026
3.0.2 Turbo — September 2025
Notable 99%+ accuracy on leading flagship AI models from OpenAI, Anthropic/Claude, Gemini, and DeepSeek.
Improved resistance, with up to 97% accuracy on the latest AI humanizers and AI bypassers.
Even lower false positive rate of 1.5%.
This model has retired: September 2026
0.0.5 Academic — September 2025
New, highly accurate model with 99%+ accuracy, built for educators.
Low false positive rates of <1% to support students and teachers.
Originality.ai 5.0 and 5.0 Max are our latest AI detection models:
Originality.ai 5.0 has 99%+ accuracy
Originality.ai 5.0 Max has 99%+ accuracy
For an in-depth look at the accuracy of Originality.ai 5.0 and 5.0 Max, visit this study.
How Do AI Detectors Check for Accuracy?
How do AI detectors check for accuracy? Here’s a bit more information for background context.
Below are the best practices and methods used to evaluate the effectiveness of AI classifiers (i.e., AI content detectors).
One single number related to a detector's effectiveness without additional context is useless!
Don’t trust a SINGLE “accuracy” number without additional context.
Here are the metrics we look at to evaluate a detector's efficacy…
Confusion Matrix
The confusion matrix and the F1 (more on it later) together are the most important measures we look at. In one image, you can quickly see the ability of an AI model to correctly identify both Original and AI-generated content.
True Positive (TP): AI detector correctly identified AI content as AI.
False Negative (FN): AI detector incorrectly identified AI content as human.
False Positive (FP): AI detector incorrectly identified human content as AI.
True Negative (TN): AI detector correctly identified human content as human.
Version 1.4 Confusion Matrix on a GPT-4 & Human Dataset Test
True Positive Rate (Recall) — AI Text Detection Capabilities
Identifies AI content correctly x% of the time. True Positive Rate TPR (also known as sensitivity, hit rate or recall).
Identifies human content correctly x% of the time. True Negative Rate TNR (also known as specificity or selectivity).
True Negative Rate TNR = TN / (TN & FP) = 1- FPR
Accuracy:
What % of your predictions were correct? Accuracy alone can provide a misleading number. This is in part why you should be skeptical of AI detectors' claimed “accuracy” numbers if they do not provide additional details for their accuracy numbers. The following metric is what we use to measure accuracy.
Combines Recall and Precision to create one measure to rank all detectors, often used when ranking multiple models. It calculates the harmonic mean of precision and sensitivity.
F1 = 2 x (PPV x TPR) / (PPV + TPR) where Precision (PPV) = TP / (TP + FP)
Metrics Considered but Not Used:
ROC & AUROC: Not used since we can't adjust the sensitivity of other tools and some tools do not provide a percentage.
Precision: PPV = TP / (TP + FP) - Not used
Why Is It Important for AI Detectors to Release Studies?
FTC Addresses Unsupported AI Detection Accuracy Claims
Claimed accuracy rates with no supporting studies are clearly a problem.
We hope the days of AI detection tools claiming 99%+ accuracy with no data to support it are over. A single number is not good enough in the face of the societal problems AI content can produce, and the important role AI content detectors have to play.
The FTC has come out on multiple occasions to warn against tools claiming AI detection accuracy or unsubstantiated AI efficacy.
In 2025, the FTC addressed misleading accuracy claims from one company offering AI detection without the data to back it up:
“The order settles allegations that Workado [Content at Scale now BrandWell] promoted its AI Content Detector as “98 percent” accurate in detecting whether text was written by AI or human. But independent testing showed the accuracy rate on general-purpose content was just 53 percent, according to the FTC’s administrative complaint. The FTC alleges that Workado violated the FTC Act because the “98 percent” claim was false, misleading, or non-substantiated.” - Source: FTC
So, we created this guide and tools, because we believe…
In the transparent and accountable development and use of AI.
That AI detectors have a role to play in supporting AI transparency across industries
AI detection tools' “accuracy” should be communicated with the same transparency and accountability that we want to see in AI’s development and use. Our hope is that this study will move us all closer to that ideal.
At Originality.ai, we aren’t for or against AI-generated content… but believe in transparency and accountability in its development, use, and detection.
Originality.ai helps ensure there is trust in the originality of the content being produced by writers, students, job applicants, or journalists.
Pro Tip: Scanning high volumes of content for AI? Check out our Bulk Scan feature.
How AI Detection (and the Originality.ai AI Detector Works)
Our AI detector works by combining advanced language models and specialized AI detection models.
We use a large language model (LLM) and then feed this model millions of carefully selected records of known AI, known mixed, and known human content. It has learned to recognize patterns between the two. More details on our AI content detection.
Our aim is to help you understand how much AI may be involved, where it was used, and why an AI detector believes AI was used.
Below is a brief summary of the 3 general approaches that an AI detector (or in Machine Learning speak, a “classifier”) can use to distinguish between AI-generated and human-generated text.
1. Feature-Based Approach:
The feature-based approach uses the fact that there can potentially be consistently identifiable and known differences that exist in all text generated by an LLM like ChatGPT when compared to human text. Some of these features that tools look to use are explained below.
Burstiness
Burstiness in text refers to the tendency of certain words to appear in clusters or "bursts" rather than being evenly distributed throughout a document.
AI-generated text can potentially have more predictability (less burstiness) since AI models tend to reuse certain words or phrases more often than a human writer would.
Some tools attempt to identify AI text using burstiness (more burstiness = human, less burstiness = AI).
Perplexity
Perplexity is a measure of how well a probability model predicts the next word. In the context of text analysis, it quantifies the uncertainty of a language model by calculating the likelihood of the model producing a given text.
Lower perplexity means that the model is less surprised by the text, indicating the text was more likely AI-generated. High perplexity scores can indicate human-generated text.
Frequency Features
Frequency features refer to the count of how often certain words, phrases, or types of words (like nouns, verbs, etc.) appear in a text. For example, AI might overuse certain words, underuse others, or use certain types of words at rates that are inconsistent with human writing. These features might be able to help detect AI-generated text.
Studies have shown that earlier (ie 2019) LLMs would generate text that has similar readability scores.
Punctuation
This pertains to the use and distribution of various punctuation marks in a text. AI-generated text often exhibits correct and potentially predictable use of punctuation.
For instance, it might use certain types of punctuation more often than a human writer would, or it might use punctuation in ways that are grammatically correct but stylistically unusual. By analyzing punctuation patterns, someone might attempt to create a detector that can predict AI-generated content.
Advantages vs. Disadvantages
Advantages: Once patterns are identified, they can be repeatedly identified in a very cost-effective and fast manner.
Disadvantages: Modern LLMs such as ChatGPT4 and Bard can produce varied enough content that these detectors can be bypassed with clever ChatGPT prompts.
2. Zero-Shot Approach:
A zero-shot approach uses a pre-trained language model to identify text generated by a model similar to itself. Basically, asking itself how likely the content the AI is seeing was generated by a similar version of itself (note: don’t try asking ChatGPT… it doesn’t work like that).
Advantages: Easier to build and does not require supervised training
A fine-tuning AI model approach uses a large language model such as BERT or RoBERTa and trains on a set of human and AI-generated text. It learns to identify the differences between the two in order to predict if the content is AI or Original.
Advantages: Can produce accurate detection that generalizes well across different types of content and new LLMs.
Disadvantages: These can be more expensive to train and operate. They can also lag behind in detection capabilities for the newest AI tools until their training is updated.
The test below looks at the performance of multiple detectors using all of the strategies identified above.
Our View: AI Detectors in Academia & False Positives in General
We do not believe that AI detection scores alone should be used for academic honesty purposes and disciplinary action.
The rate of false positives (even if low) is still too high to be relied upon for disciplinary action.
Here is a guide we created to help writers or students reduce false positives in AI content detector usage.
~Plus, we created Originality.ai Extensions (Chrome, Google Docs, and Firefox) to help writers, editors, students, and teachers visualize the creation process and prove originality.
That means helping to maintain AI transparency in content marketing, journalism, education, and more.
Consider a student submitting an assignment. They want to know they can confidently submit that essay or paper and have their hard work recognized.
Both students and teachers benefit from accurate AI detection. Students can be confident that they can prove their work is original, and teachers can have clarity on AI use.
The same is true for writers and editors. A writer submitting a blog post or article wants peace of mind that authentic, human-written work will be recognized, while editors need clarity on the writing process for submitted work.
Here’s how accurate AI detection can benefit you:
Academic Integrity
Students can prove their work is original, such as with our Extension
Teachers get clarity on AI use, like with our Moodle Plugin
Integrity in University, Scholarship, and Job Applications
Whether you’re writing or reviewing an application, confidently show (or find out) the origins of a document and check if AI usage matches your standards.
Support for SEOs and Marketers Publishing People-First Content
AI content also has SEO implications. We studied possible Google AI Penalties and found that publishing AI content can carry risks.
In 2025, Google released updated Search Quality Rater Guidelinesstating: “The Lowest rating applies if all or almost all of the MC on the page (including text, images, audio, videos, etc) is copied, paraphrased, embedded, auto or AI generated, or reposted from other sources with little to no effort, little to no originality, and little to no added value for visitors to the website.” - Google
Across 4 LLMs (ChatGPT, DeepSeek, Gemini, and Grok) and human-written samples, Originality.ai had perfect accuracy (100%), precision, recall, and F1-Score.
Turnitin, ZeroGPT, Detecting-AI.com, GPTZero, QuillBot, Grammarly, Sapling, Copyleaks, and Originality.ai
Among the clearly commercial detectors in the paper’s principal comparison table (Table 10), Originality.ai achieved the highest reported accuracy, precision, recall, and F1 score.
28 detectors noted in Table 10 of the study, including a mix of research tools and commercial AI detectors
The end result?
Across both internal testing and third-party studies, we continue to outperform competitors as the Most Accurate AI Detector.
Why Is Our Model Accurate?
Below are a few of the main reasons we suspect Originality.ai’s AI detection performance and overall AI detector accuracy are significantly better than alternatives…
Larger Model: Our AI detection system combines a number of AI models to produce a highly accurate and understandable result. This mosaic approach is complex to orchestrate for each document but yields a highly accurate and understandable result.
Train on the Latest LLMs: New AI models are continuously being released. We train our AI detector to identify the latest LLMs.
Train on Harder Datasets: The datasets we continue to create and train our AI on focus on increasingly adversarial detection bypassing methods. The better our AI gets, the more clever the prompt engineering or playground settings need to be to bypass us, and then we train on that new, more challenging dataset.
Final Thoughts
The AI/ML team and core product team at Originality.ai have worked relentlessly to build and improve on the most effective AI content detector!
Discover Originality.ai 5.0 and 5.0 Max today!
If you have any questions on whether Originality.ai would be the right solution for your organization, please contact us.
If you are looking to run your own tests, please contact us. We are always happy to support any study (academic, journalist, or curious mind).
We hope this helps establish clarity and transparency around AI detection and AI detection accuracy, and that other AI detectors will also share their accuracy results to further establish transparency in the industry.
Jonathan Gillham is an engineer, inventor, and entrepreneur. He is the founder and CEO of Originality.ai, an AI content integrity platform that launched the first commercial AI detector in November 2022, just three days before ChatGPT launched. Before founding Originality.ai, Jon worked as an engineer, built and exited two companies. His early work with generative AI in 2020 and 2021 gave him a firsthand view of the coming wave of AI-generated content and the need for technology that could bring transparency and trust to written content. Today, he leads Originality.ai’s work in AI detection and content integrity and is a named inventor on two U.S. patents covering AI detection technology. Jon’s expertise and research have been featured in WIRED, Business Insider, The Register, Global News, The Guardian, Entrepreneur, and The Washington Post, among others.
