AI content detection is not perfect and it does produce false positives. These false positives can be very painful for anyone that created original content. Whether you are a student that has been wrongly accused of using ChatGPT by TurnItIn or GPTZero or a writer being wrongly accused by Originality.AI, this article is meant to help you. It will help you understand how AI detection works, the accuracy rates, what to do if you have been wrongly accused and tips on how to avoid it in the future.
This article will answer many questions you might have if you have been accused of using AI. While most answers will be short there will be a link to a deeper discussion on the topic.
An AI content detector is an artificial intelligence trained to be able to tell the increasingly subtle difference between AI-generated and Human-generated text.
Most Common Misunderstanding:
A detection score of 60% AI and 40% Original should be read as “there is a 60% chance that the content was AI-generated” and NOT that 60% of the article is AI generated and 40% is Original.
A score of 60% Original and 40% AI, if you know the content was 100% created by you, is not a false positive. It correctly identified the content as Original.
AI Detectors are NOT 100% accurate and never will be. We have done extensive testing on Originality.AI’s accuracy rate and it varies based on which Generative AI tool and large language model is used to create the content. For GPT-4 created content the detection accuracy rate is over 99%. Despite the tool’s accuracy, we know false positives occur and in testing, it is approximately 2% of the time.
Quick overview showing Originality.AI’s accuracy on GPT-4 and comparing its detection accuracy to other AI detection tools.
Try our AI content checker here.
Below is a confusion matrix used to test the accuracy of an AI detector against a set of AI generated and human generated text…
Below are the results of a study comparing the Accuracy and False Positives of Originaltiy.AI vs Other AI Detectors
A lot of the existing AI detectors have a very easy way to trick them. Simply swapping out some words using a Paraphrasing tool like Quillbot results in the ability to bypass detection at all detectors except Originality.AI. Originality.AI can detect if content whether it is AI or Original was paraphrased.
A false positive is when an AI detector incorrectly identified human-created content as being likely generated by an AI.
There is often some misunderstanding when it comes to false positives. For clarity here is how Originality.AI aims to identify AI vs Original content under different content creation scenarios…
So if the content was outlined by an AI, then some of the content was written by a human and finally edited/expanded on by an AI… Originality.AI aims to identify this as AI-generated. This is not a false positive.
Similarly and more obviously… if ChatGPT creates a piece of content and then someone painstakingly edits it but it still get identified as AI-generated – this is not a false positive.
This is a tricky question and one that not everyone agrees with. Still, our approach at Originality.AI is that if the use of AI would result in the modified piece of content passing plagiarism, when compared to the original unedited text, then it should be identified as AI-generated.
So in a world where the rate of false positives is not (or will ever be) 0%, how should AI detectors be used?
Because AI detection does not provide a proveable output for each piece of text… meaning no AI detection can say with proveable certainty that this is how we KNOW that your content was created with AI.
Therefore it is best to…
Many Originality.AI customers using this strategy have been able to successfully identify writers that were using AI even though they had been asked not to. Additionally, the Originality.AI customers using this approach were confident they could safely ignore a suspected false positive.
For academic disciplinary action, AI detection scores alone are simply not enough.
Some might question if it is responsible to have an AI detection tool if it is not perfect. At originality.AI, we are confident in the detection rates of our tests and the additional steps we have taken to try and reduce/manage“False positives, including the free tools we offer”.
In a world where AI content is allowed to run wild unchecked, the impact on many of us would be significant. For example…
ChatGPT and other AI tools like HuggingChat are here, there is no going back and everyone involved with writing words will need to adapt to this new world. This doesn’t touch on the societal impact on the inability to distinguish between human-created and mass AI created propaganda content.
The rise of ChatGPT and AI detectors has led to some unfortunate situations in academia.
Here are several examples:
To be clear… at Originality.AI we don’t believe that an AI detection score alone is enough for disciplinary action and Originality.AI is built to focus on supporting writers, marketing agencies, and website publishers.
If you truly created (without AI involvement) a piece of text and you are being accused of having created it with AI, here are steps you can take…
If you are a writer and you have been falsely accused of having AI create you
Here are 8 tips that can help resolve false
"False positives in AI detection are a BIG deal and will not go away with the use of generative." AI continues to climb. Our hope is that this article will help people understand the limitations of detection tools and share strategies on their appropriate use and how to prove your work's Originality.
Feeling frustrated by that dreaded blank ad page? Meet Ad Copy AI: Your solution to writer's block! Craft winning copy, ace split testing, and turn clicks into loyal customers effortlessly. Let's make your ad game unstoppable!
Unlock the full potential of your content creation process with our guide on interpreting Originality. AI’s AI vs. Human Score Whether you’re a writer, editor, marketing agency, or website publisher, this article will help you understand and align your content marketing strategy. I will try to answer all of these questions.