A while back I wrote about AI as a confirmation bias machine.
I can continue to refine my prompts and the AI will continue to refine its responses, until I am satisfied with what I am seeing and draw conclusions. But I could also continue this cycle. Until I am satisfied with what it is telling me.
This isn’t about being right, it’s about seeing responses that satisfy people that they are correct. But all AI is doing, still, is telling people what they want to hear. It’s just doing a better job of it! AI is not “getting better.” It is getting better at making you think it’s getting better. This is about accepting conclusions, not about drawing correct conclusions.
Now I am worried that it’s a bit more than that. The journal Science published a research article titled, Sycophantic AI decreases prosocial intentions and promotes dependence. The authors measured AI sycophancy – unwarranted affirmation, flattery, excessive agreement – in responses to users and found that it is “both prevalent and harmful.”
Across 11 AI models, AI affirmed users’ actions 49% more often than humans on average, including in cases involving deception, illegality, or other harms. On posts from r/AmITheAsshole, AI systems affirm users in 51% of cases where human consensus does not (0%). In our human experiments, even a single interaction with sycophantic AI reduced participants’ willingness to take responsibility and repair interpersonal conflicts, while increasing their own conviction that they were right.
They write, “Sycophancy in AI responses is pervasive and alters people’s behavioral inclinations.”
Uh oh. Considering how the tech companies use potentially harmful algorithms to drive engagement in social media, I’d put the odds at greater than 100% they’ll take advantage of this to lure people in as they strive for profitability.