Artificial Intelligence | August 1, 2023
One of the main concerns surrounding artificial intelligence is the difficulty in distinguishing between text written or generated by a human and that produced by AI. Companies like OpenAI have attempted to address this issue, but with limited success, as evidenced by the fact that ChatGPT is ending its tool for detecting whether a text is written by AI, which raises some concerns.
On July 20th, OpenAI announced the retirement of AI Classifier, a tool trained to identify which texts were written by a human and which by ChatGPT (or any other artificial intelligence). The reason, according to the company, lies in its low level of accuracy in performing this task, or in other words, its inability to determine if a text is authored by a machine or a person.
AI Classifier was launched in January with the aim of helping identify disinformation campaigns based on artificial intelligence, fraudulent academic work, or instances where a robot is presented as if it were a human. However, the effectiveness of this text classifier has been far from ideal, and its training over the months has not resulted in improved results.
Initially, OpenAI stated that the tool was only capable of detecting 26% of texts generated by artificial intelligence in English. At the same time, AI Classifier also misidentified 9% of texts written by humans as artificial. The hope within the AI company was that these percentages would improve over time to make it a reliable application, but this did not happen.
Looking ahead, OpenAI has assured that it will continue researching to find more effective text recognition techniques. The company behind ChatGPT has also committed to seeking mechanisms for users to clearly identify if audio, images, or videos have been generated by AI, which has been a subject of considerable debate in recent months.
What are the main challenges faced by AI Classifier in detecting if a text is written by AI? OpenAI listed the limitations of this technology on its website, and one is particularly concerning due to its potential impact on social media: it is unreliable with short texts (less than 1,000 characters). This means that in brief posts, such as tweets, it has not been able to accurately determine whether a text is generated by AI or not, leaving room for the spread of hoaxes and misinformation in general.
Furthermore, in addition to the aforementioned errors in labeling human texts as artificial and vice versa, AI Classifier was only available during its short lifespan for analyzing texts in English. This implies that it could not perform these checks in other languages or programming languages.
Finally, ChatGPT texts that had been subsequently edited (involving human intervention to varying degrees) also passed under the AI Classifier’s radar, another drawback that has not been addressed in the past six months and has contributed to its withdrawal to prevent potential confusion among its users.
