OpenAI Launches Tool To Detect AI-written Text

With the news of ChatGPT and AI-generated text managing to clear examinations at B-schools and law schools, ChatGPT-creator OpenAI has released a new tool to detect AI-written text.

OpenAI has launched a “classifier”, which allows one to distinguish between text written by a human and text written by AIs from a variety of providers.

An AI classifier is a language model that can be trained on data to distinguish between human-written and AI-written text.

But OpenAI has led with the information that the tool isn’t completely reliable as it is “impossible to reliably detect all AI-written text”. The company says that in certain evaluations, its classifier correctly identifies 26 per cent of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9 per cent of the time (false positives).

Moreover, the company said that the tool was very unreliable on short texts (below 1,000 characters). In fact, the tool can even label human-written text as AI-written, which brings to question the use of such a tool.

“Our classifier’s reliability typically improves as the length of the input text increases. Compared to our previously released classifier, this new classifier is significantly more reliable on text from more recent AI systems,” OpenAI mentioned on its blog.

The company has made the tool publicly available.

“Our work on the detection of AI-generated text will continue, and we hope to share improved methods in the future,” said OpenAI.

The company expects the classifier to help educators somewhat and has acknowledged the issue of plagiarism. It also expects the classifier to have an impact on journalists, mis/dis-information researchers, and other groups.

“We are engaging with educators in the US to learn what they are seeing in their classrooms and to discuss ChatGPT’s capabilities and limitations, and we will continue to broaden our outreach as we learn. These are important conversations to have as part of our mission is to deploy large language models safely, in direct contact with affected communities,” said OpenAI.