In recent years we have seen a rapid rise of AI-based language models such as GPT (Generative Pre-trained Transformer). These models are capable of generating human-like text that, in many cases, is barely distinguishable from the real thing. Thanks to advances in machine learning and natural language processing (NLP), these models have become ever more sophisticated and powerful.
Although AI language models such as GPT are promising for a wide range of applications, including education, they also bring challenges. In education, AI generators can be used to develop lesson material, grade essays, and create individual learning plans. However, there are also risks associated with using this advanced technology in education.
One concern is the risk of academic dishonesty. Students can misuse AI language models such as GPT to generate essays, assignments and research reports without putting in the effort themselves. This can lead to a loss of authenticity in the current traditional learning process and ultimately to a decline in the quality of education.
At Exante we are continuously engaged with artificial intelligence and the consequences of developments in this field. The advances made by OpenAI are of unprecedented scale. Not long after ChatGPT was released, we saw the demand for AI detection tools for text emerge. This demand was mostly met with detector tools focused on the English language. We saw this as an opportunity to develop a Dutch variant. This is how the idea of GPT-Detective was born.
In this article we outline the path travelled to develop GPT-Detective from an idea into a fully realized solution. We will explore the development of our innovative AI detector tool, from the initial concept phase to the eventual implementation.
From concept to realization
Our starting point was creating our own Dutch dataset with handwritten and generated texts. We used this dataset to iterate over the various ideas we had about possible detection models. The model ideas were partly inspired by existing scientific literature. After several cycles of rapid prototyping, we had the first minimal viable product (MVP) of our detection model ready. This detection model was selected because it had the highest accuracy on our validation dataset.
The first version of GPT-Detective.
Actually implementing the model in a ready-to-use website is a challenge separate from data science and machine learning, but it was at least as important for the success of GPT-Detective. In addition to devising a data-driven solution, Exante specializes in actually implementing it and making it ready for use. That is why we quickly had the first version of the model integrated into a website, live in production.
Against our expectations, there was quick media attention for our tool. After appearing on TV, radio and in the newspaper, we had the unique opportunity to learn how an AI-driven solution with many users (and peaks in usage) can be managed. Challenges ranging from using user feedback to improve the model, to protecting our website against DoS attacks, were tackled head-on.
Indication of usage after first appearing in the media.
Releasing tools like this is exciting because you never know for sure whether the validation dataset translates well to usage in production. Fortunately, the first version of our model was well received (based on the feedback given). This gave us reason to keep building on this success. In addition to further developing the model (i.e. sentence-level detection), there was the need to monetize our service. This again brought new challenges: scaling the model used for the new sentence-level detection, and integrating a payment platform.
Our CI/CD pipeline greatly helped us — even on a small-scale project like GPT-Detective — to iterate quickly and roll out patches to production. The first version of our document scanner therefore followed quickly. This version had the first version of authentication and the sentence-level model woven into it.
First version of the document detective.
After choosing our payment platform and switching our development platform (from self-hosted to Firebase), we further developed the prototype of the document detective into a standalone product. You know this product as the current version of GPT-Detective.
Conclusion
The rise of AI-based language models such as GPT has led to great progress in a wide variety of applications, but it also brings challenges, especially in education. GPT-Detective is an innovative solution that focuses on detecting AI-generated text to counter academic dishonesty and safeguard the quality of education. In this article we explored the development of GPT-Detective, from idea to product, including the creation of a Dutch dataset, the development of the detection model, and the implementation in a user-friendly website.