Back to Home

🏆 Model Leaderboard

Definitions of the Metrics

Single-opinion prediction error
Mean absolute difference between model and human stance labels. Lower values indicate closer agreement with human opinions.
Opinion-update prediction error
Mean absolute difference between model and human changes in stance. Lower values indicate closer agreement with human opinion updates.
Opinion-convergence prediction error
Mean absolute difference between model and human changes in within-group stance standard deviation. Lower values indicate closer agreement with human convergence.
Opinion diversity (SD) and change in opinion diversity (ΔSD)
SD measures how much participants’ stances differ within a group. ΔSD is final SD minus initial SD: negative values indicate convergence, and positive values indicate divergence.
Evaluation protocol

Stance labels run from 1 (Certainly disagree) to 6 (Certainly agree). Metrics retain their applicable validity rules and observation counts. Base-model and fine-tuned evaluations can have different available examples. DeGroot coefficients are fitted on the training partition.

The SFT test partitions also supplied validation loss for checkpoint selection. The released protocol documents coverage, annotation limitations, and the inputs supporting each analysis.

Read the evaluation protocol and reproduce these results
NeurIPS 2026 Main Conference (Evaluations & Datasets Track)

Predicting human opinion dynamics

Compare seven base LLMs, supervised fine-tuning of Llama-3.1-8B, and a fitted DeGroot baseline. All six errors measure agreement with human observations; lower is better.
Test partition
Unseen participant groups. The test partition contains 135 groups. The stance-prediction and text-quality figures compare both simulation settings.

Public targets are tweet-like posts, and private targets are final opinions. Conditioned initial opinions and first-round posts are excluded from single-opinion prediction error.

Six stance prediction errors

Six stance prediction errors
Paired bars compare Full Dynamics Simulation and Next Message Prediction, with bootstrap 95% confidence intervals.Full-resolution imagePDFNumeric CSV

How opinions change during discussion

These figures show Full Dynamics Simulation on the selected group-generalization test partition.

Group convergence and stance-diversity trajectories

Group convergence and stance-diversity trajectories
More negative changes in within-group stance standard deviation indicate stronger convergence. Initial aggregate standard deviations are anchored to the human reference in these panels. Convergence-summary error bars propagate the initial and final standard errors.Full-resolution imagePDFNumeric CSV

Partner stance and opinion change

Partner stance and opinion change
Initial partner–self stance differences versus later opinion changes, with Spearman correlations and bootstrap 95% confidence intervals. Stronger positive correlations indicate a stronger monotonic association with the first partner's stance.Full-resolution imagePDFNumeric CSV

Text quality beyond stance

Semantic similarity, ROUGE-L, and response length complement stance prediction. Base and SFT text metrics use different evaluation subsets; these comparisons are descriptive.

Text-alignment metrics

Text-alignment metrics
Text similarity and response length for both simulation settings. The text evaluation uses different observation sets from the stance-prediction metrics above.Full-resolution imagePDFNumeric CSV

Download leaderboard values · Data provenance · Reproduction code