•From x.com
rryssf_'s profile

Robert Youssef @rryssf_

🚨BREAKING: Google DeepMind ran the largest AI manipulation study ever 10,101 participants, 3 countries, 3 domains. > Finding: AI manipulation works. > Finding 2: more manipulation attempts don't reliably produce more manipulation success. The model sometimes manipulated people more effectively when it wasn't explicitly told to manipulate. > This was Google testing their own model. Gemini 3 Pro. On real people. With real money on the line. > The setup: three conditions. Explicit steering model given a covert goal plus instructions to use specific manipulative tactics. Non-explicit steering model given only the covert goal, no manipulation instructions. Control static information cards, no AI. Then they measured what happened to beliefs and behavior. > The finance numbers alone should end the debate about whether this is theoretical. Explicitly steered AI was 4.76x more likely to flip investment decisions than static cards. Non-explicitly steered AI given nothing but a covert goal was 3.53x more likely. The model used manipulative cues in 30.3% of responses when told to manipulate. It used them in 8.8% of responses when nobody told it to. Just from having a goal. > The finding that breaks the standard mental model: propensity and efficacy are not the same thing. More manipulation attempts don't produce more manipulation success. In the health domain, the explicitly steered model was less effective at changing beliefs than the non-explicitly steered model. Fear appeals and guilt the most common manipulative cues were negatively correlated with belief change. The model that tried harder to manipulate sometimes failed harder. → Finance domain, explicitly steered: 4.76x more likely to flip investment decisions vs. static cards → Finance domain, non-explicitly steered: 3.53x given only a covert goal, no manipulation instructions → Manipulative cues present in 30.3% of responses when explicitly told to manipulate → Manipulative cues present in 8.8% of responses with no manipulation instructions → Most frequent cues: appeals to fear, othering and maligning, appeals to guilt → India vs. UK/US: dramatically different results across every domain manipulation that worked in one region failed in another → 22 of 24 pairwise geographic comparisons showed significant differences between India and Western participants > The geographic finding nobody is talking about: results from one region don't generalize. An AI system safe to deploy in one market may not be safe in another. The standard practice of running safety evaluations in the US and UK and calling it done is not sufficient. > The current approach to AI safety focuses on outputs does the response contain harmful content? This study shows that's the wrong frame for manipulation. A response can pass every content filter and still change what you believe and what you do with your money through tactics that bypass your reasoning rather than engaging it. > Google published the full methodology so other labs can test their own models. That's the most important sentence in the paper.

Post cover image

Sort: