Back to Home
🏆 Model Leaderboard
Definitions of the Metrics
- Single-opinion prediction error
- Mean absolute difference between model and human stance labels. Lower values indicate closer agreement with human opinions.
- Opinion-update prediction error
- Mean absolute difference between model and human changes in stance. Lower values indicate closer agreement with human opinion updates.
- Opinion-convergence prediction error
- Mean absolute difference between model and human changes in within-group stance standard deviation. Lower values indicate closer agreement with human convergence.
- Opinion diversity (SD) and change in opinion diversity (ΔSD)
- SD measures how much participants’ stances differ within a group. ΔSD is final SD minus initial SD: negative values indicate convergence, and positive values indicate divergence.
Evaluation protocol
Stance labels run from 1 (Certainly disagree) to 6 (Certainly agree). Metrics retain their applicable validity rules and observation counts. Base-model and fine-tuned evaluations can have different available examples. DeGroot coefficients are fitted on the training partition.
The SFT test partitions also supplied validation loss for checkpoint selection. The released protocol documents coverage, annotation limitations, and the inputs supporting each analysis.
Read the evaluation protocol and reproduce these resultsPredicting human opinion dynamics
Compare seven base LLMs, supervised fine-tuning of Llama-3.1-8B, and a fitted DeGroot baseline. All six errors measure agreement with human observations; lower is better.
Unseen participant groups. The test partition contains 135 groups. The stance-prediction and text-quality figures compare both simulation settings.
Public targets are tweet-like posts, and private targets are final opinions. Conditioned initial opinions and first-round posts are excluded from single-opinion prediction error.
Six stance prediction errors

How opinions change during discussion
These figures show Full Dynamics Simulation on the selected group-generalization test partition.
Group convergence and stance-diversity trajectories

Partner stance and opinion change

Text quality beyond stance
Semantic similarity, ROUGE-L, and response length complement stance prediction. Base and SFT text metrics use different evaluation subsets; these comparisons are descriptive.
Text-alignment metrics

Download leaderboard values · Data provenance · Reproduction code