David Franklin Quotes

In advance of the class, he asked students to fill in a quick survey in which he asked them to describe their own experiences with one of these three uses of statistics. He tells the students that he has read their answers, as well as their student profiles, and asks for volunteers to talk about what they wrote. Several hands go up, and he picks Juliana, a student from Brazil, interested in education. She starts to talk about a program she helped to run in Brazil which assessed whether improvements in teacher training had a causal effect on test scores. As she is talking, her words appear on the big screen, in big quotation marks, along with her photo. The class laughs, and as she looks up she realises she has become a celebrity.

The Google team at Project Aristotle concluded that there were five main traits of a successful team. Psychological safety was the most important, and their research suggested that it underpinned the other four[xxvi]. The five traits were as follows: (1) Psychological safety: team members feel safe to take risks and be vulnerable in front of each other. (2) Dependability: team members get things done on time, and meet Google’s high bar for excellence. (3) Structure and clarity: team members have clear roles, plans, and goals. (4) Meaning: work is personally important to team members. (5) Impact: team members think their work matters and creates change.

The brain has conflicting goals: to provide you with information, and to reduce your anxiety about the worrying outcome. If you wanted Clinton to win in 2016, your brain achieved both goals by accepting the 30% figure but telling you it would not happen.

Acknowledging that students have competing purposes is the first step to managing them.

Now – those who got it right. On the count of three, I’d like you to sit down if you’d seen the question before. 1, 2, 3!” Everyone who was standing sits down. This gets a huge laugh. The students who got the answer wrong suddenly feel much better about their mistake: in a Harvard class, not a single person got it right unless they had seen it before.

I remember he said something like ‘this should be completely obvious to you’, and to me that was crushing… we had a conversation after the mid-term exam, in which the class had averaged 20 out of 90. It was 25 years ago, and I still remember what he said: ‘Frustration is necessary for learning. This idea that you can enjoy learning is a very American idea’.” Dan pauses, the memory of that conversation etched on his face. “I felt so offended by that claim. My professor felt that to learn, you had to push yourself. One of my dreams was to go back to Venezuela[7] and start a university, and I vowed that I would write in the walls of the university that frustration was not necessary for learning.

I guess… it was just a shocking event. We’re not used to someone like him being President.” Here, the student reveals an availability bias in which we expect future outcomes to look like what has gone before.

He asks the 80 students to respond based on their ‘gut feeling’. Again, students are given five options, ranging from ‘less than 1%’ to ‘above 40%’. About half of them believe the true answer is less than 5%, of which plenty go for the ‘less than 1%’ option. Only 1 in 6 get it right, picking the highest option: it turns out that the true figure is 41%. He invites those people – 13 in total – to stand up.

So Trump was 29% likely to win. What does 29% mean?” he asks. This is an odd question for many: they’re doing a Master’s degree at Harvard, and they’re being asked what a percentage means. The question is aiming at the gut reaction of the brain to the number. Inevitably, students think it might be a trick question. Dan waits a while, and when no-one raises their hands, he breaks away from the example to tell a story: “You know, when I first started teaching, I was terrified of silence. I thought, ‘oh my god, I’ve got to do something, they’re not saying anything’.” The class laughs: he has eased the tension created by the silence. “The more I taught, the more I realised that silences are important in a class – they give time for people to think. These days, I’m not afraid of silence at all.” After a few seconds, a woman puts up her hand. “Well, obviously, I knew that it meant there was some chance that he would win. But it was still a shock that the 29% happened.

But few of us really stop to ask ourselves what new information might change our minds, especially on questions of politics and identity. We fall into the trap of confirmation bias, in which rather than updating our prior beliefs, we mould new information into a form that will confirm them.

The inclusive atmosphere of the class was the key driver of allowing the class to be eager to learn. The focus really shifted away from grades and points, and towards understanding concepts and learning.” The key link is between inclusivity and the shift of focus away from test scores. When students are given the responsibility to support each other, their own score falls down the list of priorities.

But everyone who guessed in this way went low, rather than high. It suggests that there is some form of systemic bias in play.

After pausing, the student continues: “There’s also an element of believing something because you want it to happen.” This betrays a second bias in which we believe what we want to be true. This is a form of confirmation bias, which exacerbates the distorting effect of availability bias. We seek out information that confirms our existing beliefs or desires, and ignore information that refutes them.

With these issues in mind, Dan has a unique way of tying exams into the learning process. After attending a workshop run by the Nobel Prize winner Carl Wieman, he was converted to the idea of two-stage exams. The first stage of the exam is the same as any other. The second stage brings the students together in groups of four, and gives them all a subset of the questions they just answered in the first stage. The students are encouraged to discuss their answers in their groups, and they all submit a second version. Groups who manage to improve upon the students’ individual scores get a small boost in the final outcome.

The first key idea is that when you have a challenge for which measuring progress is hard, the ability to adapt is just as important as the ability to plan.