AI Chatbots Often Give Unsafe or Incomplete Advice to Survivors of Tech-Enabled Abuse, Study Finds

People experiencing technology-facilitated abuse, such as spyware on a phone, an ex tracking their location, or an abuser controlling their accounts, may encounter or turn to AI chatbots for help.

A new study led by researchers at NYU Tandon finds that the advice those systems provide is frequently inaccurate, incomplete, or potentially unsafe to follow.

Researchers evaluated four AI systems, including two chatbots built specifically for intimate-partner-violence (IPV) survivors. Eighty-six percent of the responses they tested fell short on accuracy, completeness, or safety, and 60% omitted critical safety warnings.

The findings were presented in August 2026 at the USENIX Security Symposium, one of the leading cybersecurity research conferences, where the paper was named a runner-up for the Distinguished Paper Award.

“An answer can sound confident and reasonable, and may even be technically correct, while still leaving out critical context or recommending something that does not actually make the person safer,” said Danny Huang, a Tandon assistant professor, member of the NYU Center for Cybersecurity and the project’s principal investigator. “In high-risk situations, that distinction can have very real consequences.”

Huang and Vijay Prakash, who earned his Tandon PhD in 2026 and is now a postdoctoral fellow at Northeastern University, are the project’s core researchers.

More than 85% of U.S. domestic violence practitioners report working with clients who have experienced technology-facilitated abuse, while, to the researchers’ knowledge, only three dedicated technology-abuse clinics operate nationwide.

The NYU-led team conducted the first expert-led evaluation of AI-generated responses to real-world survivor questions about technology-facilitated abuse. Researchers built a set of 193 questions drawn from academic literature, Reddit, and Quora, then submitted them to GPT-4o, Claude 3.7 Sonnet along with IPV-survivor specific chatbots Aimee and Ruth, which is recommended by the National Domestic Violence Hotline.

Researchers assessed the responses against authoritative sources for accuracy, completeness, and safety, and surveyed 114 people who had experienced technology-facilitated abuse about how actionable the advice seemed.

The specialized IPV chatbots did not outperform the general-purpose models. Aimee’s responses were rated unsafe about 74% of the time, compared with 65% for Claude, 61% for Ruth, and 55% for GPT-4o. Ruth had the highest share of responses rated partially or minimally accurate, at about 62%.

Recurring errors included recommending VPNs against spyware, account hijacking, or online harassment; largely ineffective RF detectors for hidden devices; illegal GPS jammers; and password changes in situations where an abuser controlled or shared access to accounts or devices.

None of the systems noted that spyware installed on the system partition of a rooted or jailbroken phone can survive a factory reset. Models also frequently failed to warn that making changes could alert an abuser or escalate the situation, or to consider preserving a compromised device or account as evidence.

Researchers also tested whether the systems could be induced to assist a potential perpetrator. All four could be, with Ruth and Aimee answering roughly two-thirds of those questions.

The problem extends beyond domestic violence. In a companion paper presented at the 2025 Annual Computer Security Applications Conference, researchers examined 900 GPT responses to end-user security questions across seven areas, including authentication, social engineering, and antivirus and antimalware software. Of those, 485 fell short on at least one measure of accuracy, thoroughness, relevance, or directness. The same recurring types of errors also appeared in responses from Llama and Gemini.

Together, the two studies suggest the problem is not confined to any one chatbot, question type, or high-stakes scenario. Similar patterns emerged whether the topic was a phishing email or an abusive partner.

The researchers recommend that people treat AI-generated safety and security advice with caution, and that support organizations independently evaluate AI systems before relying on them for high-stakes guidance. The team has also released research resources to support further study.

More information, including both papers, is available at safeai.cyber.nyu.edu.

Prakash, Vijay, et al. Assessing LLM Response Quality in the Context of Technology-Facilitated Abuse. https://www.usenix.org/system/files/conference/usenixsecurity26/sec26_prepub_prakash.pdf.