Monday, October 18, 2021

Growth Mindset Theory: Failures to Replicate

Psychologist Carol Dweck's "growth mindset" theory has become a popular solution and intervention technique in (mostly American) schools of all ages. We might say that it's become the new version of the "self-esteem" movement seen in the 80's. While Dweck first developed the theory in the 90's, it's really taken hold of popular consciousness from the 2010's on.

Unfortunately, we should remember that psychology has an ongoing replication crisis in many of its landmark findings. Many of the "easy" ideas for transformative effects have not borne fruit over the years, and been later found to have tainted methods by core researchers. Sure enough, in recent years many or most of the large-scale, high-quality attempts at replicating the claims of growth mindset have failed to so. Here are a few examples:

Li, Y., & Bates, T. C., Ph.D. (2017). Does growth mindset improve children’s IQ, educational attainment or response to setbacks? Active-control interventions and data on children’s own mindsets. (Study done in China, students aged 9-13 years, N = 624)

No effect of the classic growth mindset manipulation was found for either moderate or more difficult material... children’s mindsets were unrelated to resilience to failure for either outcome measure... Finally, in 2 studies relating mindset to grades across a semester in school, the predicted association of growth mindset with improved grades was not supported. Neither was there any association of children’s mindsets with their grades at the start of the semester. Beliefs about the malleability of basic ability may not be related to resilience to failure or progress in school.

Foliano, F., Rolfe, H., Buzzeo, J., Runge, J., & Wilkinson, D. (2019). Changing mindsets: Effectiveness trial. National Institute of Economic and Social Research. Summary at PsychBrief. (Study in England, Year 6 students, N = 4584.)

The difference between the control group and the intervention group on all 3 primary outcomes [math, reading, GPS] was 0... The difference between the groups for all 4 secondary outcomes was also 0... This RCT was a highly powered test of the efficacy of growth mindset in a real-world environment across a wide range of schools in the England. The fact none of the primary or secondary outcomes were distinguishable from 0 raises serious questions as to the efficacy of growth mindset for Year 6 students... Given the evidence so far, it is unrealistic to expect growth mindset to have large and/or wide-scale impact.

Caitlin Brez, Eric M. Hampton, Linda Behrendt, Liz Brown & Josh Powers (2020) Failure to Replicate: Testing a Growth Mindset Intervention for College Student Success, Basic and Applied Social Psychology, 42:6, 460-468, DOI: 10.1080/01973533.2020.1806845 (U.S. study, university math & psychology students, N = 2607).

The pattern of findings is clear that the intervention had little impact on students’ academic success even among sub-samples of students who are traditionally assumed to benefit from this type of intervention (e.g., minority, low income, and first-generation students)... These findings support some of the emerging literature that demonstrates that growth mindset interventions may not be as effective as once thought... The proposition that a one-time intervention at the postsecondary level will result in long-term measurable student outcomes was not supported in the present study.

Now, a not-uncommon defense in a number of these cases in psychology is that the attempts to replicate didn't properly recreate the conditions or variables for a true test. The counter-argument here would be the observer-expectancy effect -- in some cases a primary researcher has even argued that only they have the necessary knowledge to ever do so. Indeed, Dweck has made the "not anyone can do a replication" argument (BuzzFeed News interview). In response, Nick Brown, who developed the GRIM (Granularity-Related Inconsistency of Means) test and found several errors in Dweck's seminal paper, said this:

The question I have is: If your effect is so fragile that it can only be reproduced [under strictly controlled conditions], then why do you think it can be reproduced by schoolteachers?

Finally, psychologist Russell Warne wrote on his blog:

I discovered the one characteristic that the studies that support mindset theory share and that all the studies that contradict the theory lack: Carol Dweck... So, there you go! Growth mindsets can improve academic performance –if you have Carol Dweck in charge of your intervention.

This is somewhat hyperbolic, but clarifies the issue at stake. Growth mindset theory fits fairly snugly into the basket of psychological "quick fixes" that make up the replication crisis, broadly cuts against long-standing findings from neuroscience on intelligence, and is racking up more failures-to-replicate as it garners more attention. Like other similar principles that came before, it's probably a bad bet that institutional interventions based on the theory will be worth the resources spent on them.

This post was initially written as an answer on Stack Exchange: Mathematics Educators. Thanks to the community there for reading and refining it.

Tuesday, August 31, 2021

On Chained Relations

In all of the college math courses I teach -- from basic algebra, precalculus, calculus, discrete mathematics, etc. -- there's a particular piece of syntax that perpetually trips up students, and it's this: chained relations

To be clear, chained relations are compound statements in mathematics with more than one relational symbol (including equalities and inequalities). Crack open any math textbook and you're bound to see almost any piece of symbolic expression written in that format. And yet my students are always tripping all over themselves at the difficulty of either reading or writing them. Have you ever noticed this before? Let's consider the several factors contributing to this difficulty:

  1. Even a single equality is hard for people to truly understand. Numerous academic papers have been written on this. More than one person has pointed out that the use of the equals-sign in grade-school problems and the calculator point people in the incorrect direction of a functional understanding, rather than a relational understanding.

  2. There is no explicit instruction in the form in any curriculum. To my knowledge, I've never seen the status of chained relations directly addressed or tested in any math textbook at any level (again, whether in basic algebra, precalculus, etc., etc.). At some point instructors just start using it and we assume students will understand by osmosis.

  3. The compound form is entirely foreign to a natural language like English. Consider something super simple like \(a = b = c\). Translated literally to English, it says, "a is equal to b is equal to c" -- and that's a run-on sentence, disallowed by the rules of English grammar. But here in the algebraic language we have an entirely novel mode of permitted expression.

Considering that last point,we might observe that there is (surprisingly, for such a basic point?) unresolved confusion about how one should even pronounce out loud a simple chained relation. For example:

(Note that while the question is essentially the same, those two queries have entirely different top-voted answers.)

In my opinion, the status of chained relations is one of those classic blindspot/submarined issues that's buried in math education, and winds up troubling students throughout their career. To instructors: it's "obvious" and never rises to consciousness as an issue. To students: it's a quagmire that's never clearly addressed or exercised.

To this end, I've found that I need to start my discrete mathematics classes foremost with direct instruction on this issue; namely a short document that I ask students to read -- and to which I'll be referring them throughout the semester when mistakes are made. You can download it here:

On Chained Relations (PDF)

And then to practice reading them, a timed quiz at the Automatic Algebra site:

Quiz on Chained Relations

Interestingly with that quiz, I've had different math-trained professionals try it and tell me variously that (a) it was entirely trivial and of unclear value, or (b) it was entirely impossible within the span given on the timer. Isn't that interesting? What do you think?

Tuesday, July 20, 2021

Veritasium on Learning Styles

The Veritasium channel on YouTube recently released a very high-quality video on "The Biggest Myth in Education", to wit: the Learning Styles theory. It's much needed and much appreciated. Includes comments by famed cognitive psychologist Daniel Willingham, with whom I had the chance to speak in person on this (constantly frustrating) subject a few years ago. From the climax:

Review articles of learning styles consistently conclude there is no credible evidence that learning styles exist. In a 2009 review, the researchers note: "The contrast between the enormous popularity of the learning styles approach within education and the lack of credible evidence for its utility is, in our opinion, striking and disturbing... If classification of students' learning styles has practical utility, it remains to be demonstrated."

Big thanks to Veritasium for producing this. Pass it on!

Tuesday, July 6, 2021

Minimal Mental RSA Cryptosystem

As an exercise, I went hunting for smallest possible RSA cryptosystem such that I could perform all of the encryption and decryption calculations in my head. This turns out to be quite silly, of course. But perhaps you could consider using this as a toy-box classroom example, or possibly like me, you may find it helps you remember the process by being able to exercise the whole thing concretely in your brain. 

I'll use the same notation as in the Wikipedia: RSA (cryptosystem) article, which you may want to review (I'll avoid repeating the basic principles of the system here). So our system needs distinct primes p and q, the product n = pq, and decryption/encryption exponent keys d and e. I'll also use, as per the original RSA paper, φ(n) = (p − 1)(q − 1) as the value with which our keys need to avoid common factors.

Obviously, the fundamental calculation for both encryption and decryption is computations of me mod n. Which looks pretty simple, unless you have a large exponent e, and you're trying to do this in your head. Secondarily, having big plaintext values m complicates the job. Being able to reduce modulo n seems nice, but even as an intermediary step I don't want to be raising double-digits m's to double-digit e's, say. 

So the first job is to find a usable encryption exponent e. As noted on Wikipedia, "the smallest (and fastest) possible value for e is 3". Let's just reflect on why that is for a second. You can't use e = 0, aside from collapsing every block to a value of 1, because gcd(0, φ(n)) = φ(n) ≥ 2, instead of the required 1 (since the minimal primes p and q are 2 and 3, the minimal φ(n) is (1)(2) = 2). You can't use e = 1, because that would send every block to itself, and not demonstrate any encryption at all. You can't use e = 2, because at least one of your pair of primes is odd, so either (p − 1) or (q − 1) has a factor of 2, which can't be shared by e; and for the same reason e can't be any other even number.

So our first valid option for exponentiation is e = 3, and the next one is e = 5, etc. I don't know about you, but I really don't think I can do 5th or 7th powers of arbitrary numbers reliably in my head (even modulo n), so I'm aiming to just use e = 3 here.

Now, what can we use for the primes p and q? Even using one as big as 7 causes problems; say if p = 7, then (p − 1) = 6 = 2(3), at which point the factor of 3 blocks you from using our desired encryption key of e = 3. If I used a pair like 5 and 11, then n = 55, and I'm poised to be cubing numbers as big as 54, which is way outside my mental times-tables. So it looks like I'll be compelled just use 3 and 5 as our prime pair.

In this case, if p = 3 and q = 5, so n = 15, then we don't even have sufficient space to encode all the letters of the alphabet (I said this was silly, right?). Also, we'll only be able to encode one of our limited characters at a time, which loses sight of the fact that RSA is a block cipher, but let's choose to forgive that. We'll enumerate the first part of the alphabet as A = 0, B = 1, C = 2, ... O = 14. Also note that
φ(n) = 2(4) = 8 (so the only factor we need to avoid in e is 2, and we can indeed use e = 3).

What about our decryption key exponent d? Well, there's a goofy coincidence here: solving the required congruence de ≡ 1 (mod φ(n)), that is, 3d ≡ 1 (mod 8)), we get d = 3 (because 3(3) = 9 = 8 + 1). Here we have d = e, so our encryption and decryption keys happen to be the exact same thing. On the one hand, this seems like a degenerate example for how the RSA system should work, but for our mental arithmetic, it's pretty darned nice that both keys are the smallest value possible. (As noted, I really didn't want to be raising values to a 7th power or something for decryption purposes!). So we can work with that.

Here's an example. Let's say we want to mentally encrypt the plaintext message "MIND". Using our number-encoding scheme, these four characters are represented by the numbers (12, 8, 13, 3). Computing the necessary cubings-modulo-15 to encrypt, we can feasibly find in our head:

  • 12³ = (144)(12) = (135 + 9)(12) ≡ (9)(12) = 108 = 105 + 3 ≡ 3 (mod 15)
  • 8³ = (64)(8) = (60 + 4)(8) ≡ (4)(8) = 32 = 30 + 2 ≡ 2 (mod 15)
  • 13³ = (169)(13) = (165 + 4)(13) ≡ (4)(13) = 52 = 45 + 7 ≡ 7 (mod 15)
  • 3³ = (9)(3) = 27 = 15 + 12 ≡ 12 (mod 15)

So our encrypted message is (3, 2, 7, 12), or "DCHM" if we send it as alphabetic characters. Decrypting is (fortunately or unfortunately, depending on your perspective) exercising the exact same mental algorithm:

  • ≡ 12 (mod 15) [as above]
  • ≡ 8 (mod 15) [super easy!]
  • 7³ = (49)(7) = (45 + 4)(7) ≡ (4)(7) = 28 = 15 + 13 ≡ 13 (mod 15)
  • 12³ ≡ 3 (mod 15) [as above]

And indeed we recover our original transmission of (12, 8, 13, 3), or "MIND".

Okay, hopefully you can appreciate that example. Because I had to work really, really hard to find one that avoided being totally nonsensical. Here's why: In our toy cryptosystem, most characters get sent to themselves under encryption! Specifically (looking at the numerical representations): 0 → 0, 1 → 1, 2 → 8, 3 → 12, 4 → 4, 5 → 5, 6 → 6, 7 → 13, 8 → 2, 9 → 9, 10 → 10, 11 → 11, 12 → 3, 13 → 7, 14 → 14.

In summary: Only the six plaintext values 2, 3, 7, 8, 12, and 13 get sent to other images; the other nine values all get sent to themselves. That is, the only letters that get transformed are C, D, H, I, M, N -- and you'll notice that I carefully constructed my example of "MIND" to use exclusively, and almost all of, these available characters. Note in particular that "I" is the only available vowel in that set (initially I considered encoding A = 1, B = 2, etc.; but in that case we get no vowels with images different from themselves.)

In case you want to memorize these transformations directly, then it might help to observe the pattern, starting with 14 and wrapping around, that you get 3 values sent to the same as themselves, then 2 with different images, then 3 same, 2 different, 3 same, and 2 different. (Again noting a total of 6 characters that get sent to images different from themselves.) And the decryption function is, obviously, identical. 

Is that of any interest? As I opened with, it's pretty darned silly. We can only handle letters up to O, we lose the block-cipher action, the encryption and decryption keys are identical, and most of our values get sent to themselves under the transformation. But I thought it was an interesting puzzle to see if there as any example that I could compute mentally without any mechanical or even pen-and-paper aid, and it turns out there is.

Leave a comment if you ever use that in a class or paper as a warm-up demonstration!

Monday, June 29, 2020

Community College Students and the Pandemic

Outstanding article by Elke Weesjes, one of my colleagues at CUNY/Kingsborough, on the immense difficulties faced by our community college students pursuing an education in a time of pandemic:
In fact, unstable living conditions are one of many chronic crises low-income community college students face. Others include higher exposure to violence and racism, poor health due to limited access to preventative healthcare, personal, or familial immigration issues, and a myriad of other socioeconomic disadvantages. They have many responsibilities, ranging from working full-time jobs to caring for children, siblings, or parents. Many are first generation students, members of minority groups, and non-native English speakers. While some of these challenges and crises are visible to professors, others remain outside their periphery.
Business Insider.

Tuesday, February 18, 2020

The Course Syllabus is Not a Contract

Here's a really great article that came to my attention on the abusive notion (or at least, sloppy analogy) that "the course syllabus is a legal contract": Rumore, Martha M. "The Course Syllabus: Legal Contract or Operator’s Manual?." American journal of pharmaceutical education 80.10 (2016). Some excerpts:
For several decades the literature has referred to syllabi as legal documents and/or contracts between students and professors. A review of the legal precedents reveals that syllabi are not considered contracts because the courts refuse thus far to recognize educational malpractice or breach of contract as a cause of action...

Based on the literature review and the search of university websites, identification or declaration of syllabi as contracts was evident, including widespread use of the term “learning contract.” For the past several decades, much of the literature has referred to syllabi as contracts and invoked the term “contract” or “learning contract” when referring to syllabi. Both faculty members and students appear to view a syllabus as a contract...

Although there have only been a handful of cases involving syllabi, the courts have consistently ruled that a syllabus is not a contract. In these cases, students brought lawsuits for breach of contract where the professor did not follow the syllabus or applied a different grade assessment... Students have been unsuccessful in asserting these claims, and courts remain reluctant to create a cause of action for either breach of contract or educational malpractice. Courts have generally ruled so because it is difficult to define the duty to educate; causation is difficult to determine; courts are reluctant to insert themselves into public policy issues such as the quality of education; and such interference by the courts would open a floodgate of litigation from academically unsuccessful students...

Some authors and university websites explicitly state a syllabus is a contract between the professor and students. Claiming a syllabus is a contract might produce a different legal outcome. While there may not be harm in thinking a syllabus is a contract, there may be legal risk in proclaiming it so...

Contracts are legally enforceable documents; syllabi are not. Syllabi have persisted in the culture of higher education and are foundational parts of the pharmacy curricula that encourage students to develop an acceptance of a lifelong responsibility for learning. For decades, syllabi have been referred to in the literature as contracts between students and professors. In the handful of cases involving syllabi, the claims have been mostly for breach of contract. To date, courts have not recognized claims of breach of contract for syllabi and do not consider syllabi as contracts.  

Monday, April 29, 2019

Solitaire: Chance for Starting Duplicates

In Klondike Solitaire, what's the chance for starting with two face-up cards of the same rank & color (what we might call "shadows", cards which don't help us very much)?

There are 7 face-up cards in starting Solitaire, so we consider the ways to choose 7 cards randomly without any duplicates. Every time we select such a card, the number available for the next draw (avoiding "shadows") goes down by 2, while the cards in the total deck goes down by 1. Therefore the chance for no-shadows is:
$${52 \over 52} \times {50 \over 51} \times {48 \over 50} \times {46 \over 49} \times {44 \over 48} \times {42 \over 47} \times {40 \over 46} \approx 0.63$$
This in turn means that the complement event, i.e., getting any shadows at all (at least two cards with duplicate rank & color), has a probability of:
$$1 - 0.63 = 0.37 = 37 \%$$
That is: About one chance in three.

More about Solitaire.