OpenAI released an AI text classifier that attempts to detect if input content was generated using artificial intelligence tools like ChatGPT.

“The AI ​​Text Classifier is a fine-tuned GPT model that predicts the likelihood that a piece of text has been AI-generated from various sources, such as ChatGPT,” explains a new OpenAI blog post.

OpenAI released the tool today after many universities and K-12 school districts banned the company’s popular ChatGPT AI chatbot due to its ability to complete student assignments, such as writing reading and essay reports, and even completing programming assignments.

According to Business InternChatGPT is banned in NYC, Seattle, Los Angeles and Baltimore K-12 public school districts, with French and Indian universities also banning the platform from school computers.

BleepingComputer tested the new OpenAI AI Text Classifier and, for the most part, I found it rather inconclusive.

When testing OpenAI’s AI text classifier against most of our own content, it correctly determined that a human wrote our articles.

However, when analyzing the content generated by ChatGPT and You.com’s AI chatbot, it had great difficulty detecting whether the text was AI-generated.

As educators will likely use the new AI text classifiers to check whether students cheated on their assignments, OpenAI warns that it should not be used as the “single piece of evidence” to determine academic dishonesty.

“Our classifier is not completely reliable”, warns OpenAI.

“In our assessments on a ‘challenge set’ of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as ‘probably AI-written’, while incorrectly labeling text written by human as written by AI 9% of the time (false positives).”

“The reliability of our classifier generally improves as the length of the input text increases”

Classifier success will likely improve over time and will be trained with other data. For now, however, it’s not a reliable tool for detecting AI-generated content.