OpenAI, one of the leading artificial intelligence companies, has disabled its artificial intelligence (AI) typed text detection tool, AI Classifier, due to its low accuracy rate. The tool, which was announced in January as a way to detect whether content had been created using generative AI tools such as its own ChatGPT chatbox, failed to serve its purpose and was quietly pulled last week.
In the original announcement, OpenAI had touted the AI Classifier as a potentially useful tool for teachers, supposedly allowing them to identify whether a text was AI-generated or human-written. However, after six months of testing and evaluation, the company concluded that the tool was not reliable enough and had several limitations.
Assessments conducted on a set of English texts showed that the AI Classifier was only able to correctly identify 26% of the AI-written texts as “probably AI-written”, while incorrectly labeling 9% of the human-written texts as generated by AI. In addition, the tool had difficulty working with texts under 1,000 characters and performed poorly outside of its neural network-based training data.
The educational sector had a special interest in this tool, since it sought to accurately detect the use of AI in academic work. Since the launch of ChatGPT, educators had raised concerns that students could use the chatbot to write essays and papers, which could affect academic integrity.
OpenAI has recognized the limitations of its AI Classifier and has expressed its commitment to continue to improve and develop more effective tools for detecting AI-generated content. The company is researching more efficient provenance techniques for text and is committed to giving users a clear understanding of whether audio or visual content is AI-generated.
“We are working to incorporate feedback and are currently investigating more effective provenance techniques for text, and have committed to developing and implementing mechanisms that allow users to understand whether audio or visual content is AI-generated,” OpenAI wrote.
Despite this setback, OpenAI remains one of the leading companies in the field of artificial intelligence and continues to work on improving its tools and technologies. Accurate detection of AI-generated content remains a major challenge, but the company is determined to continue expanding its reach and learning from its experiences to offer more efficient and reliable solutions in the future.