Rethinking the link between cognitive sophistication and politically motivated reasoning

Journal of Experimental Psychology: General 150(6): 1095–1114, doi 10.1037/xge0000974 (online first 2020). All three studies pre-registered on OSF (osf.io/j7hrb, osf.io/2byaq, osf.io/w2g9y); data and code at osf.io/yt3kd (not downloaded). Read by researcher R5b (change with reasons), 5 October 2026. Provenance: papers/change/tappin2021_sophistication_pmr.provenance.json.

What was read

• Read in full: the 46-page PDF from the first author's website: the APA online-first article (pages 1–20: introduction, causal argument, Studies 1–3, general discussion, references) and the appended Supplementary Information (pages 21–46: treatment wordings, Tables S1–S24, captions of Figures S1–S6).
• Conversion note: pdftotext -layout interleaved the two columns, so the read file was made with pdftotext in reading order and folded. Study 3's SI tables come out scrambled by column; their values were reassembled by row order and only the clearly assignable ones are used below.
• Not inspected as images: Figures 1–7 and S1–S6 (the interaction estimates they plot are in the tables).

What it is

• The claim under test ("magnification"): the most cognitively sophisticated partisans disagree most about facts because sophistication magnifies politically motivated reasoning, reasoning driven by wanting conclusions congenial to one's political group.
• The paradigm: people evaluate the same information whose implication is congenial or uncongenial to their side; the outcome is how good they judge the evidence.
• The confound the paper targets: political identity correlates with prior factual beliefs about the issue, and coherence with prior beliefs changes how evidence is judged even with no political motive (e.g. Koehler 1993 with randomly assigned, apolitical priors; belief bias in syllogisms). In the causal diagram:
  • Path 1: identity acts on reasoning directly, which is clear evidence of motivated reasoning;
  • Path 2: the prior belief acts on reasoning, which counts as motivated only if identity caused the prior, and common causes (media, friends, trusted sources) make that unidentifiable.
• Studies (all pre-registered; outcome rescaled to 0–1):

• Identity–prior correlations: r = .42, .28, .47. Party and ideology correlated .80, .67, .67.

Main results (verified)

• Three-way interaction, sophistication × treatment × identity, alone:
  • Study 1: 0.001 (SE .004, p = .74), so Kahan (2013) did not replicate on MTurk;
  • Study 2: 0.010 (p = .004), a conceptual replication, driven entirely by liberals and Democrats;
  • Study 3: significant for all four indicators, about 0.034–0.058.
• The same with prior belief instead: significant in all three studies (Study 1 0.015, p = .025; Study 2 0.022, p = .001; Study 3 about 0.046–0.070).
• Joint models, both entered (the critical test):
  • the identity three-way shrinks to near zero and is non-significant everywhere (Study 1 −0.002; Study 2 0.006, p = .11; Study 3 about −0.008 to 0.018);
  • the prior-belief three-way survives (Study 1 0.016, p = .035; Study 2 0.015, p = .049; Study 3 0.039–0.072, all p ≤ .001).
  • Conclusion: sophistication magnifies the effect of prior factual beliefs on reasoning, not the effect of identity.
• Identity still has a direct effect, unmagnified (two-way models without sophistication, Tables S10–S12; 0–1 outcome):

People rate otherwise identical evidence less favourably when it is uncongenial to their party, even holding their prior factual belief constant (as MacCoun and Paletz 2009 found). In Study 3, where both predictors are standardized, the identity term is about 0.3 of the prior term (0.033 against 0.111 per SD). In Studies 1–2 identity is a sum of two standardized scales, so the coefficients are not per-SD comparable.

• Asymmetries flip between samples: in Study 1 the effect was driven by people who thought sceptics more open-minded; in Studies 2–3 by liberals and gun-control supporters. The authors attribute this to sampling and population differences.
• Robustness: party and ideology analysed separately (Tables S13–S24), a binary prior in Study 3 (the prior was bimodal), the original two-wave Study 3 sample (n = 851, after about 60% attrition), and the pre-registered identity coding all give the same pattern. Multicollinearity was mild (one VIF 10.45 in Study 1).
• Cited context (not read here): CRT test–retest r = .75–.81 (Stagnaro et al. 2018); high-numerate people were persuaded by strong information even when the opposing party sponsored it, while low-numerate people followed only their own party's information regardless of strength (Mérola and Hitt 2016); high-CRT reasoners discriminate valid from invalid syllogisms better overall, but more so when the conclusion contradicts their beliefs (Trippas et al. 2015).

Limits

• Observational on the key variables: identity, prior belief and sophistication are measured, not randomized. Measurement error in the prior could leave a spurious residual identity effect, and the authors stress the estimates are not causal effects.
• One-shot evaluations of evidence quality, not belief change over time; US online samples; three issues framings (open-mindedness and climate, gun control).
• Definitions matter: under a "belief-based" definition of motivated reasoning (defending prior beliefs as possessions), the prior-belief result would itself count as motivated reasoning; the authors argue that definition is not the dominant one and is normatively ambiguous.

What it means for Kurisutina (inference)

• Split the "bias" in the rule of change into two terms:
  • coherence with the prior: evidence that contradicts what one already believes is judged weaker. This is the larger term, it grows with sophistication, and it is compatible with sensible updating (a confident prior makes contrary evidence more likely to be flawed);
  • identity congeniality: evidence that hurts one's group is judged weaker even at the same prior. This term is smaller and does not grow with sophistication.
• Sophistication is not a general "rationality" dial. More able reasoners are more sensitive to argument substance and to coherence with their priors at once. A person slot holding sophistication should scale both quality sensitivity and prior-coherence weighting, not reduce bias.
• For M1: with identity, prior and argument quality all known, the synthetic rule can include both terms with separate person settings, and ground truth stays exact.
• For the battery: a matched human reference for a "same evidence, opposite implication" probe: identity × treatment ≈ 0.03 and prior × treatment ≈ 0.07–0.15 on a 0–1 evaluation scale, with no magnification of the identity term by CRT.