Nine published ways of measuring AI persuasion rank the same 15 models very differently, a preprint finds ↗
The authors ran nine automated persuasion evaluations in one shared setup on the same 15 language models and found their rankings agreed only weakly (mean Spearman correlation 0.25). Refusals, mostly on manipulation tasks, cut agreement by about a quarter. They conclude that persuasion scores mix ability with willingness, so even the lab measure behind the problem is unsettled.
arXiv · LLM Persuasion Is in the Eye of the Evaluation