FLASHPOINTS #6: When should decision-making be inhuman?

An interview with Daisy Christodoulou

Daisy Christodoulou is one of Britain’s foremost experts on education. A former teacher of English, she went on to write acclaimed and important books, starting with Seven Myths About Education, which draws on cognitive science to make the case for teaching ‘knowledge’ over ‘learning skills’. It’s a brilliantly argued and persuasive book that has been influential on my thinking about education and learning (I drew on it for my book, Curious). Daisy’s subsequent books have been concerned with the role of technology in the assessment of student progress, and she now works for a company called No More Marking, focusing on that challenge. Daisy is a true renaissance woman with a wide span of passionate interests, from poetry to Greek philosophy to neuroscience to sport. One of the themes that joins her interests together is how technology can improve human decision-making, hence the topic I chose for our conversation.

In this interview I start by asking her about why exams are necessary at all - many people believe they should be minimised or abolished in favour of teacher-based assessment. Then we talk about how exams can be improved and the role that technology can play; Daisy explains why she’s a “pro-exam romantic”. We touch on why teaching knowledge is a prerequisite of critical thinking. In the last section I ask her about the use of technology in sport and football refereeing in particular. Daisy draws a fascinating parallel with exams.

---

Hi Daisy. Parents, children, and teachers often express discomfort with exams. Teacher assessment sounds fairer and more human; it’s based on a relationship, not a set of checkboxes. Exams may be efficient but shouldn’t we be looking to minimise their role? 

You’re right that a lot of people think exams can be very dehumanising; that they don't give a full picture of a student’s ability, and that teacher assessment gives a more rounded understanding. But when we look at the research, that is simply not the case. There’s an entire literature that shows teacher assessment is biased against poor students, against ethnic minority students, against students with SEN, against students with behavioural difficulties, and so on and so forth.  Students from these groups do better on exams than on teacher assessments.

Why is this? It isn’t to do with teachers being particularly bad people; rather, it’s that teachers are human. All humans are subject to biases and inconsistencies when it comes to making judgements. When we are faced with tricky decisions, we use short-cuts to make the decision easier. One of those short-cuts involves using stereotypes, in this case about student groups.  

Exams are an effective way of eliminating this problem. Many of the things we dislike about exams are actually methods of reducing the biases and inconsistencies of human judgement. Making all students do the same task at the same time in the same conditions and anonymising their responses makes it much harder for stereotypes to influence the assessment.

The research on teacher assessment versus exams is a subset of a wider body of research on human decision-making versus algorithms. An algorithm doesn’t have to be a complex computer programme: it can mean any form of pre-set rules you use to make a decision. Some of the pioneering research on this was done by Paul Meehl in the 1950s. Meehl compared the performance of humans and algorithms on making all kinds of decisions and showed that the algorithms were superior. More recently, the work of Daniel Kahneman and Amos Tversky has explored many of the different kinds of biases that human judgement suffers from.

It’s worth pointing out the alternative view too: Gary Klein has done a lot of work on where human intuition is very good – for example, firefighters who just know when a building is about to collapse. In fact, Klein and Kahneman collaborated on a very good paper where they worked out the areas in which human judgement can be very powerful, and where it struggles. One of the key factors they identify is the importance of feedback. In areas where you get rapid and accurate feedback, it’s possible to really improve the accuracy of your judgements. But in many of the areas we are most interested in, the feedback you get is delayed and ambiguous or even misleading. The most interesting and frightening example given by Kahneman and Klein is that of a doctor in the early 20th century who would have intuitions about which of his patients had typhoid. In their words “he confirmed his intuitions by palpating these patients’ tongues, but because he did not wash his hands the intuitions were disastrously self-fulfilling.”

Another aspect of human judgement, which Kahneman and Klein don’t mention, but which is a major focus of the work I do, is the difference between absolute and comparative judgement.

To read on, subscribe! You’ll get access to the rest of this fascinating conversation and to previous FLASHPOINTS on the gender gap in mental health, the rise of crypto, how we can tell if someone is irrational and whether NATO is to blame for the war in Ukraine. Plus the whole stash of brainfood behind the paywall in each edition of The Ruffian.