On the conversational persuasiveness of GPT-4
Abstract
Early work has found that large language models (LLMs) can generate persuasive content. However, evidence on whether they can also personalize arguments to individual attributes remains limited, despite being crucial for assessing misuse. This preregistered study examines AI-driven persuasion in a controlled setting, where participants engaged in short multiround debates. Participants were randomly assigned to 1 of 12 conditions in a 2 × 2 × 3 design: (1) human or GPT-4 debate opponent; (2) opponent with or without access to sociodemographic participant data; (3) debate topic of low, medium or high opinion strength. In debate pairs where AI and humans were not equally persuasive, GPT-4 with personalization was more persuasive 64.4% of the time (81.2% relative increase in odds of higher post-debate agreement; 95% confidence interval [+26.0%, +160.7%], P < 0.01; N = 900). Our findings highlight the power of LLM-based persuasion and have implications for the governance and design of online platforms.
How strong is the evidence?
Evidence Hub reads this as a randomized trial, done in people. Its evidence level is 2 of 5 (High · randomized trial). Randomized trials of a reasonable size are the strongest single-study design. Reported size: 900 participants or records.
Read the paper
Related studies
- In silico toxicology protocols
- QM7-X, a comprehensive dataset of quantum-mechanical properties spanning the chemical space of small organic molecules
- Quantum Informed Machine-Learning Potentials for Molecular Dynamics Simulations of CO2's Chemisorption and Diffusion in Mg-MOF-74
- In-silico strategies in nano-drug design: Bridging nanomaterials and pharmacological applications
- mRNA codon optimization with quantum computers
- The computational power of the human brain
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.