Dartmouth Researcher Develops AI Agent That Evades Survey Bot Detection With 99.8% Success
In Brief
Dartmouth researcher Sean Westwood created an AI agent that bypassed survey bot detection 99.8% of the time, showing the threat coming to online research studies.
Sean Westwood, an associate professor of government at Dartmouth and director of the Polarization Research Lab, developed an AI system he describes as “an autonomous synthetic respondent,” capable of answering survey questions while bypassing nearly all advanced detection methods for automated responses.
Published in the Proceedings of the National Academy of Sciences (PNAS), the research highlights that online survey research, a cornerstone of data collection in many scientific studies, may now face a threat from large language models (LLMs).
The study reports that the AI agent successfully evaded detection 99.8% of the time. Traditional survey safeguards, such as attention check questions, behavioral indicators, and response pattern analysis, are designed to identify inattentive humans or automated bots. Westwood argues these tools are becoming ineffective, as his AI agent circumvented standard detection methods described in prominent studies, including those aimed specifically at identifying AI-generated responses. It also passed “reverse shibboleth” questions, which are tasks that AI can handle easily but are challenging for humans.
According to the paper, titled “The potential existential threat of large language models to online survey research,” the AI system operates in layers: once its reasoning engine determines a response, a secondary layer executes actions designed to mimic human behavior. The system simulates realistic reading times based on the persona’s education level, generates human-like mouse movements, and types open-ended answers keystroke by keystroke, including plausible typos and corrections. It is also configured to work with tools that bypass common anti-bot measures such as reCAPTCHA, further enabling undetectable automated participation in online surveys.
Study Warns AI Personas Could Manipulate National Polls With Minimal, Low-Cost Responses
According to the study, the AI system is capable of simulating “a coherent demographic persona,” suggesting that, in theory, it could manipulate online survey results to reflect any desired outcome based on an AI-generated profile. Only a small number of fabricated responses would be needed to shift results. The paper notes that for the seven major national polls leading up to the 2024 election, introducing as few as 10 to 52 AI-generated responses could have altered the predicted results. Producing these responses would be extremely inexpensive, estimated at around five cents per response, compared with the typical $1.50 paid to human survey participants.
Westwood’s AI agent is model-agnostic and implemented in Python, meaning it can function via APIs from major AI providers such as OpenAI, Anthropic, or Google, or be hosted locally using open-weight models like LLama. In the study, testing primarily used OpenAI’s o4-mini, but other models including DeepSeek R1, Mistral Large, Claude 3.7 Sonnet, Grok3, and Gemini 2.5 Preview were also employed to demonstrate the approach’s compatibility across multiple LLMs. The agent operates from a single prompt of approximately 500 words, which instructs it on the type of persona to emulate and how to respond in a human-like manner.
The paper further highlights potential strategies for mitigating the risk of AI-generated survey manipulation, though each approach carries trade-offs. Enhanced identity verification of participants could reduce the threat but raises privacy concerns. Researchers are also encouraged to increase transparency in survey data collection and consider more controlled participant recruitment methods, such as address-based sampling or using voter registration files, to maintain data integrity.
Disclaimer
In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.
About The Author
Alisa, a dedicated journalist at the MPost, specializes in cryptocurrency, zero-knowledge proofs, investments, and the expansive realm of Web3. With a keen eye for emerging trends and technologies, she delivers comprehensive coverage to inform and engage readers in the ever-evolving landscape of digital finance.
More articles
Alisa, a dedicated journalist at the MPost, specializes in cryptocurrency, zero-knowledge proofs, investments, and the expansive realm of Web3. With a keen eye for emerging trends and technologies, she delivers comprehensive coverage to inform and engage readers in the ever-evolving landscape of digital finance.
