In a recent Wall Street Journal headline, we're told that OpenAI has a magical tool to catch students cheating with ChatGPT. Are our academic integrity issues solved in a blinding flash of deus ex machina?
Let's get one thing straight: there is no tool that can "detect text written by artificial intelligence" with 99.9% certainty. What OpenAI has is a way to watermark its own output. It's an invisible signature, not a universal AI detector. The difference isn't just semantic—it's fundamental.
Detecting AI-generated text in general is equivalent to solving the Halting problem for Turing machines. It's not just difficult; it's mathematically impossible. Anyone claiming otherwise is either misinformed or selling snake oil.
Yes, OpenAI can detect its own watermark. But at what cost? This watermarking process works by subtly altering a text's statistical patterns. In other words, they're degrading their own product. How much? They're not saying. But make no mistake—there's a mathematically unavoidable trade-off between detectability and quality.
But let's humour this idea for a moment. Say OpenAI starts watermarking ChatGPT’s output. What then? Cheaters aren't known for their loyalty. They'll simply switch to one of the myriad other AI tools available. Meta's Llama 3.1 405B, for instance, matches the original GPT-4's capabilities without any watermarking. The cat-and-mouse game continues, ad infinitum.
The fundamental issue here isn't technological—it's ethical. We're facing a crisis of values, not a shortage of detection tools. No amount of digital fingerprinting or statistical analysis will instill academic integrity in a student determined to cheat.
Instead of chasing technological silver bullets, educators need to confront the harder questions: Why are students cheating? How can we create assessments that value critical thinking over regurgitation? How do we foster a culture of learning rather than one of grade-chasing?
These are the questions that matter. Everything else is just noise—a distraction from the real work of education in the AI age. It's time we stopped looking for easy answers and started tackling the hard problems. Our students deserve nothing less.