Monday, October 18, 2021

Growth Mindset Theory: Failures to Replicate

Psychologist Carol Dweck's "growth mindset" theory has become a popular solution and intervention technique in (mostly American) schools of all ages. We might say that it's become the new version of the "self-esteem" movement seen in the 80's. While Dweck first developed the theory in the 90's, it's really taken hold of popular consciousness from the 2010's on.

Unfortunately, we should remember that psychology has an ongoing replication crisis in many of its landmark findings. Many of the "easy" ideas for transformative effects have not borne fruit over the years, and been later found to have tainted methods by core researchers. Sure enough, in recent years many or most of the large-scale, high-quality attempts at replicating the claims of growth mindset have failed to so. Here are a few examples:

Li, Y., & Bates, T. C., Ph.D. (2017). Does growth mindset improve children’s IQ, educational attainment or response to setbacks? Active-control interventions and data on children’s own mindsets. https://doi.org/10.31235/osf.io/tsdwy (Study done in China, students aged 9-13 years, N = 624)

No effect of the classic growth mindset manipulation was found for either moderate or more difficult material... children’s mindsets were unrelated to resilience to failure for either outcome measure... Finally, in 2 studies relating mindset to grades across a semester in school, the predicted association of growth mindset with improved grades was not supported. Neither was there any association of children’s mindsets with their grades at the start of the semester. Beliefs about the malleability of basic ability may not be related to resilience to failure or progress in school.

Foliano, F., Rolfe, H., Buzzeo, J., Runge, J., & Wilkinson, D. (2019). Changing mindsets: Effectiveness trial. National Institute of Economic and Social Research. Summary at PsychBrief. (Study in England, Year 6 students, N = 4584.)

The difference between the control group and the intervention group on all 3 primary outcomes [math, reading, GPS] was 0... The difference between the groups for all 4 secondary outcomes was also 0... This RCT was a highly powered test of the efficacy of growth mindset in a real-world environment across a wide range of schools in the England. The fact none of the primary or secondary outcomes were distinguishable from 0 raises serious questions as to the efficacy of growth mindset for Year 6 students... Given the evidence so far, it is unrealistic to expect growth mindset to have large and/or wide-scale impact.

Caitlin Brez, Eric M. Hampton, Linda Behrendt, Liz Brown & Josh Powers (2020) Failure to Replicate: Testing a Growth Mindset Intervention for College Student Success, Basic and Applied Social Psychology, 42:6, 460-468, DOI: 10.1080/01973533.2020.1806845 (U.S. study, university math & psychology students, N = 2607).

The pattern of findings is clear that the intervention had little impact on students’ academic success even among sub-samples of students who are traditionally assumed to benefit from this type of intervention (e.g., minority, low income, and first-generation students)... These findings support some of the emerging literature that demonstrates that growth mindset interventions may not be as effective as once thought... The proposition that a one-time intervention at the postsecondary level will result in long-term measurable student outcomes was not supported in the present study.


Now, a not-uncommon defense in a number of these cases in psychology is that the attempts to replicate didn't properly recreate the conditions or variables for a true test. The counter-argument here would be the observer-expectancy effect -- in some cases a primary researcher has even argued that only they have the necessary knowledge to ever do so. Indeed, Dweck has made the "not anyone can do a replication" argument (BuzzFeed News interview). In response, Nick Brown, who developed the GRIM (Granularity-Related Inconsistency of Means) test and found several errors in Dweck's seminal paper, said this:

The question I have is: If your effect is so fragile that it can only be reproduced [under strictly controlled conditions], then why do you think it can be reproduced by schoolteachers?

Finally, psychologist Russell Warne wrote on his blog:

I discovered the one characteristic that the studies that support mindset theory share and that all the studies that contradict the theory lack: Carol Dweck... So, there you go! Growth mindsets can improve academic performance –if you have Carol Dweck in charge of your intervention.

This is somewhat hyperbolic, but clarifies the issue at stake. Growth mindset theory fits fairly snugly into the basket of psychological "quick fixes" that make up the replication crisis, broadly cuts against long-standing findings from neuroscience on intelligence, and is racking up more failures-to-replicate as it garners more attention. Like other similar principles that came before, it's probably a bad bet that institutional interventions based on the theory will be worth the resources spent on them.

This post was initially written as an answer on Stack Exchange: Mathematics Educators. Thanks to the community there for reading and refining it.

No comments:

Post a Comment