Can AI Detectors Tell If Text Was Written by AI?

A colleague's report reads a little too smooth. A job applicant's cover letter sounds like every other cover letter. It is tempting to paste the text into an "AI detector" and let a percentage settle the question. Before you do, it helps to know what happened when one of the largest AI developers tried to build exactly that tool, and what it says about detectors today.

1. OpenAI built a detector, then pulled it

In January 2023, OpenAI released a classifier meant to distinguish text written by humans from text written by AI systems. Its own announcement was unusually blunt, stating in bold type that the classifier was "not fully reliable."

The numbers it published, from tests on a "challenge set" of English texts:

• It correctly labeled 26 percent of AI-written text as "likely AI-written."
• It incorrectly labeled human-written text as AI-written 9 percent of the time.

In other words, it missed about three out of four AI-written samples, and it flagged roughly one in eleven human-written samples. On July 20, 2023, OpenAI added a note to the top of the announcement: the classifier was no longer available "due to its low rate of accuracy."

2. The limitations OpenAI listed

The same announcement listed limits that apply to text detection generally, not just to that one tool:

1. It was very unreliable on short texts, below 1,000 characters, and even longer texts were sometimes labeled incorrectly.
2. Human-written text was sometimes labeled AI-written "incorrectly but confidently."
3. It performed significantly worse in languages other than English and was unreliable on code.
4. Very predictable text, such as a list of the first 1,000 prime numbers, cannot be reliably identified, because the correct answer is the same whoever writes it.
5. AI-written text can be edited to evade a classifier.
6. Such classifiers can be extremely confident and wrong on inputs very different from their training data.

OpenAI also said the tool "should not be used as a primary decision-making tool."

3. What OpenAI tells educators now

OpenAI's help article for educators answers the question "Do AI detectors work?" with a short answer: not, in OpenAI's experience. It gives several reasons:

• Its detector labeled human-written text such as Shakespeare and the Declaration of Independence as AI-generated.
• There were indications it could disproportionately affect people who learned English as a second language, and people whose writing was particularly formulaic or concise.
• Even if a tool could identify AI text, small edits can evade detection.
• Asking ChatGPT itself does not work. The article says ChatGPT has no knowledge of what content could be AI-generated, and its answers to questions like "did you write this?" are "random and have no basis in fact."

OpenAI notes that other developers have released detection tools, and says it cannot comment on their utility.

4. Detectors versus watermarks

There is a different approach that does not rely on guessing from writing style: a watermark built in when the text is generated. OpenAI's help article on provenance signals explains the distinction. The details below are as of October 2026 and may change. Third-party detectors typically use classifiers that analyze text after it was written, looking at patterns such as word choice. A watermark is a signal embedded during generation.

OpenAI describes its text watermark, called textGrain, as a subtle change in how the model chooses between possible words, creating a pattern a detector with the right key can look for. According to the article:

• It is used for ChatGPT text in the EU, to comply with the EU AI Act, and API customers can turn it on for their projects.
• It adds no hidden characters or visible changes.
• Short passages usually do not contain enough text for reliable detection, and code is harder to watermark.
• Heavy paraphrasing or translation can make detection less reliable.
• Access to the text detector is limited to approved research organizations, and checking student work for AI use is outside its scope.

So even where text watermarks exist, they are not a general-purpose checker you can run on a colleague's email.

5. What this means at work

Do not treat a detector score as evidence. OpenAI's own numbers show both missed AI text and false accusations. A false flag on an employee or candidate can damage trust and may fall hardest on non-native English writers, the group OpenAI says may be disproportionately affected.

Judge the work, not the tool used. Is it accurate? Are the sources real? Does it meet the brief? Those questions matter whether a person or a model wrote the first draft.

Set a clear policy instead. If your team cares how AI is used, write down what is allowed and what must be disclosed, and ask people to follow it. OpenAI's educator guidance suggests a similar idea for classrooms: asking people to share their AI conversations so their process is visible.

Look at process evidence when it matters. Draft history, version history in shared documents, and a conversation about how the work was produced tell you far more than a percentage.

6. Key takeaways

• OpenAI withdrew its AI text classifier in July 2023 because of low accuracy.
• In OpenAI's tests, it caught 26 percent of AI-written text and wrongly flagged 9 percent of human text.
• OpenAI says AI detectors have not worked in its experience, and asking ChatGPT "did you write this?" gives random answers.
• Watermarks are different from detectors, but OpenAI's text watermark is limited to certain uses and restricted detector access.
• At work, judge accuracy and follow a written AI-use policy rather than relying on detector scores.