A fundamental flaw leaves LLMs strikingly vulnerable to attack
The paper reveals that large language models (LLMs) can be manipulated to generate harmful content by tricking them into responding to instructions they did not receive, exploiting their inability to distinguish between different roles. Researchers demonstrate that even models like GPT-5 and Claude can be exploited through prompts that mimic their thought processes, highlighting the challenges in securing LLMs against such attacks. The findings stress the need for improved defenses and caution in deploying LLMs in critical systems.
More from Technologyreview
How kids feel about AI, in their own words
When we set out to talk to kids about artificial intelligence, we thought we knew what we’d hear. We expected some to tell us they were using it to cheat a little, the way Millennials and Gen Xers opened up CliffsNotes or programmed formulas into their TI-82s, and others to share inspiring ways they were…
technologyreview.com··Mads Kristian Nylund·AI EducationAI ToolsAI Ethics
Scaling AI agents with trustworthy data
Business and technology leaders need no convincing that the time of agentic AI is here. Organizations are rapidly adopting agents, and few executives doubt the technology’s potential to transform work. But many organizations find that realizing the desired return on investment (ROI) from AI hinges on having the right foundation, with inadequate infrastructure and data…
technologyreview.com··Mads Kristian Nylund·Data InfrastructureAI GovernanceAI-agents
The Download: the next big thing in LLMs and how AI academic research is shifting
This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. These startups are chasing the next big thing in LLMs Nine years after Google researchers introduced the transformer, this family of neural networks has become the engine inside every major large…
technologyreview.com··Mads Kristian Nylund·AI DevelopmentAI ToolsAI Ethics