David Franklin Quotes

I remember he said something like ‘this should be completely obvious to you’, and to me that was crushing… we had a conversation after the mid-term exam, in which the class had averaged 20 out of 90. It was 25 years ago, and I still remember what he said: ‘Frustration is necessary for learning. This idea that you can enjoy learning is a very American idea’.” Dan pauses, the memory of that conversation etched on his face. “I felt so offended by that claim. My professor felt that to learn, you had to push yourself. One of my dreams was to go back to Venezuela[7] and start a university, and I vowed that I would write in the walls of the university that frustration was not necessary for learning.

He asks the 80 students to respond based on their ‘gut feeling’. Again, students are given five options, ranging from ‘less than 1%’ to ‘above 40%’. About half of them believe the true answer is less than 5%, of which plenty go for the ‘less than 1%’ option. Only 1 in 6 get it right, picking the highest option: it turns out that the true figure is 41%. He invites those people – 13 in total – to stand up.

Dan’s goal, taken from his mentor Richard Zeckhauser, is to get students to think probabilistically about the world. This means engaging with the challenge of understanding and accepting what 29% means in the context of a Trump victory, and fighting the brain’s natural inclination to see things in binary terms.

Acknowledging that students have competing purposes is the first step to managing them.

For these ‘future leaders’ in the classroom, it is more important to understand the psychology of certainty. Often, we believe what we want to believe. Dan starts with an example of the 2016 Presidential Election in the United States. He asks the question: “Relative to what you expected to happen, how surprised were you when you learned that Trump had won the election?

With these issues in mind, Dan has a unique way of tying exams into the learning process. After attending a workshop run by the Nobel Prize winner Carl Wieman, he was converted to the idea of two-stage exams. The first stage of the exam is the same as any other. The second stage brings the students together in groups of four, and gives them all a subset of the questions they just answered in the first stage. The students are encouraged to discuss their answers in their groups, and they all submit a second version. Groups who manage to improve upon the students’ individual scores get a small boost in the final outcome.

I’d like anyone who was born on March 22nd to stand up.” Two students stand up. They link eyes from across the room, and smile. The room laughs. “Now, anyone born on June 14th.” Another two stand up. The room laughs again. “Now, anyone born on August 2nd.” Two more: the class is laughing at each pair, and re-examining their original takes on how low the probability was likely to be. They are learning through experience about human fallibility with estimating probabilities, and at the same time learning something about their classmates. “How about December 21st?” This time, three students stand up. This gets a big laugh for two reasons: firstly, it is unexpected, and secondly, it helps to ram home the idea that in a class that size (there are about 3,000 pairs in a class of 80), there are likely to be “coincidences” everywhere.

The phrases ‘certainty is an illusion’ and ‘think probabilistically about the world’, both introduced in the first class, are the first two examples of high-level concepts which Dan calls airport ideas.

Dan’s use of two-stage exams kills two birds with one stone. Firstly, he maximises learning by ensuring that the exam itself is a learning experience. Second, in doing so, he makes clear that the grade is less important than the learning. Two-stage exams have not yet ‘taken off’ around the world, and grades remain the key outcome of most exams for most students. Dan, though, has taken advantage of his position in a graduate university environment to push the idea forward.

What is the probability that at least two people in a group of 20 people will have the same birthday?

The inclusive atmosphere of the class was the key driver of allowing the class to be eager to learn. The focus really shifted away from grades and points, and towards understanding concepts and learning.” The key link is between inclusivity and the shift of focus away from test scores. When students are given the responsibility to support each other, their own score falls down the list of priorities.

The first key idea is that when you have a challenge for which measuring progress is hard, the ability to adapt is just as important as the ability to plan.

The failure here is an example of a common flaw in our approach to adaptive challenges: we try to solve them with technical solutions. Terrified of the unknown, we seek to replace problems we have no idea how to address with those we know how to solve. This is a form of ‘work avoidance’, a term coined by Heifetz[x]. If you have ever procrastinated by spending your whole afternoon making a brightly-coloured work plan, and then congratulated yourself on being productive, then you will be familiar with the concept. By focusing on what they know they can measure – the extent to which a lesson went as planned – some professors may avoid the harder work of trying to gauge the learning of their students.

But everyone who guessed in this way went low, rather than high. It suggests that there is some form of systemic bias in play.

So Trump was 29% likely to win. What does 29% mean?” he asks. This is an odd question for many: they’re doing a Master’s degree at Harvard, and they’re being asked what a percentage means. The question is aiming at the gut reaction of the brain to the number. Inevitably, students think it might be a trick question. Dan waits a while, and when no-one raises their hands, he breaks away from the example to tell a story: “You know, when I first started teaching, I was terrified of silence. I thought, ‘oh my god, I’ve got to do something, they’re not saying anything’.” The class laughs: he has eased the tension created by the silence. “The more I taught, the more I realised that silences are important in a class – they give time for people to think. These days, I’m not afraid of silence at all.” After a few seconds, a woman puts up her hand. “Well, obviously, I knew that it meant there was some chance that he would win. But it was still a shock that the 29% happened.