A Review of OpenAI’s AI Detector

It’s designed for the competition, not for ChatGPT

Michael

--

On January 31, 2023, OpenAI launched an updated free AI detector. The company said that they “recognize that identifying AI-written text has been an important point of discussion among educators”. The new tool attempts to help educators check if a student submitted AI-generated papers.

The news comes following the release of many AI detectors. The tool extends on OpenAI’s previous GPT2 Output Detector with data from many language models.

In this post I will give an honest, full review of the new detector. I will show how you can use, how it works, what is the accuracy, and why you shouldn’t trust this tool.

Image by OpenAI

Sample Usage

The detector is available at OpenAI’s website, at: https://platform.openai.com/ai-text-classifier.

You will need to authenticate to your OpenAI account to use the tool.

Once you have accessed the site, you can copy some text and click the Submit button to get a prediction.

The detector currently responds with one of five predictions. These predictions indicate how confident the model is about the text.

  • Very unlikely AI-generated
  • Unlikely AI-generated
  • Unclear if AI-generated
  • Possibly AI-generated
  • Likely AI-generated

Also, the text must be at least 1000 characters long. The detector does not produce a prediction for texts with fewer characters.

The detector is also limited to 4097 tokens. This corresponds to around 3000 words. If you request more tokens, the server will respond with an error message.

How it works

OpenAI used their datasets of AI-generated and human-written text.

The AI-generated text comes from 34 models from 5 different organizations that deploy language models. This means the model is not specific to ChatGPT — it attempts to detect AI text…

--

--

Michael

I am currently a backend engineer. I like to research new topics and blog about them.