Proving You’re Not a Robot is Really Hard
They've gotten better at solving CAPTCHAs than we are.
Josh Dzieza explains “Why CAPTCHAs Have Gotten so Difficult.”
At some point last year, Google’s constant requests to prove I’m human began to feel increasingly aggressive. More and more, the simple, slightly too-cute button saying “I’m not a robot” was followed by demands to prove it — by selecting all the traffic lights, crosswalks, and storefronts in an image grid. Soon the traffic lights were buried in distant foliage, the crosswalks warped and half around a corner, the storefront signage blurry and in Korean. There’s something uniquely dispiriting about being asked to identify a fire hydrant and struggling at it.
These tests are called CAPTCHA, an acronym for Completely Automated Public Turing test to tell Computers and Humans Apart, and they’ve reached this sort of inscrutability plateau before. In the early 2000s, simple images of text were enough to stump most spambots. But a decade later, after Google had bought the program from Carnegie Mellon researchers and was using it to digitize Google Books, texts had to be increasingly warped and obscured to stay ahead of improving optical character recognition programs — programs which, in a roundabout way, all those humans solving CAPTCHAs were helping to improve.
Because CAPTCHA is such an elegant tool for training AI, any given test could only ever be temporary, something its inventors acknowledged at the outset. With all those researchers, scammers, and ordinary humans solving billions of puzzles just at the threshold of what AI can do, at some point the machines were going to pass us by. In 2014, Google pitted one of its machine learning algorithms against humans in solving the most distorted text CAPTCHAs: the computer got the test right 99.8 percent of the time, while the humans got a mere 33 percent.
Google then moved to NoCaptcha ReCaptcha, which observes user data and behavior to let some humans pass through with a click of the “I’m not a robot” button, and presents others with the image labeling we see today. But the machines are once again catching up. All those awnings that may or may not be storefronts? They’re the endgame in humanity’s arms race with the machines.
After several paragraphs detailing failed attempts to solve this problem, we’re left with this:
The problem with many of these tests isn’t necessarily that bots are too clever — it’s that humans suck at them. And it’s not that humans are dumb; it’s that humans are wildly diverse in language, culture, and experience. Once you get rid of all that stuff to make a test that any human can pass, without prior training or much thought, you’re left with brute tasks like image processing, exactly the thing a tailor-made AI is going to be good at.
The direction scientists are moving now is intriguing:
In his book The Most Human Human, Brian Christian enters a Turing Test competition as the human foil and finds that it’s actually quite difficult to prove your humanity in conversation. On the other hand, bot makers have found it easy to pass, not by being the most eloquent or intelligent conversationalist, but by dodging questions with non sequitur jokes, making typos, or in the case of the bot that won a Turing competition in 2014, claiming to be a 13-year-old Ukrainian boy with a poor grasp of English. After all, to err is human. It’s possible a similar future is in store for CAPTCHA, the most widely used Turing test in the world — a new arms race not to create bots that surpass humans in labeling images and parsing text, but ones that make mistakes, miss buttons, get distracted, and switch tabs. “I think folks are realizing that there is an application for simulating the average human user… or dumb humans,” Ghosemajumder says.
CAPTCHA tests may persist in this world, too. Amazon received a patent in 2017 for a scheme involving optical illusions and logic puzzles humans have great difficulty in deciphering. Called Turing Test via failure, the only way to pass is to get the answer wrong.
We’ve had similar experiences in the natural world with antibiotics and pesticides. Eventually, the pests evolve the become ever more resistant over time. In the end, we just create better pests.
It’s all quite fascinating. For whatever reason, I always found the old-style CAPTCHAs, with fuzzy alphanumerics, challenging. The pictographs are easier for me but annoying–although not as much as the giant pop-ups that I have to defeat in order not to subscribe to newsletters I don’t want or to comply with EU regulations on cookies.