OpenAI has decided to take down its AI text classifier, citing its low rate of accuracy. The classifier, which was released a few months ago, aimed to determine whether a piece of text was generated by a large language model or written by a human.
The AI classifier was free for users to employ by simply copying and pasting text into it. It ranked the likelihood of the content being machine-generated or human-written, providing a useful tool for discerning the origin of an email, blog post, or essay.
However, OpenAI acknowledged that the classifier was not fully reliable and often misclassified human-written text as AI-generated. It struggled with prose that went beyond its training data and demonstrated overconfidence in its predictions.
The motivation behind launching the AI classifier was to address concerns regarding machine-generated content being used unethically by students. OpenAI cautioned educators against blindly relying on the model’s predictions and encouraged using it alongside other methods to ascertain the source of a text.
Unfortunately, accurate classification of AI text remains a challenging task, as similar tools developed by other entities also suffer from reliability issues. The University of Texas A&M-Commerce made headlines when an instructor withheld grades based on ChatGPT’s prediction of AI-generated text. Similarly, Turnitin’s AI software, touted to have a 98 percent confidence in detecting plagiarism, faces doubts about its actual accuracy.
OpenAI continues to work on solving this problem and has pledged to develop digital watermarks to make AI-generated content traceable. The organization aims to ensure the safety of next-generation machine learning technology.
The exact timeframe for releasing an improved version of the AI classifier has not been provided by OpenAI.