Programming Note: Darts and Letters is off and retooling to relaunch in a new format early 2024.

Teaching Resources: Crisis in Forensic Science

We created a mini-series of podcast episodes on crisis in forensic science, which you can find in full here. If you are an educator, you might find these resources useful.

Ro Nadler of UBC, the research assistant on the project, compiled a set of listening notes, discussion questions, and additional readings to supplement the podcasts. Those are included here, in chronological order of the series’ release.

Episode #1 of series: Derailed, the Crisis in Forensic Expertise

This episode introduces listeners to critical perspectives on forensic science via the wrongful arrest of Brandon Mayfield in 2004, and continues this introduction through practitioner, academic, and popular-media perspectives. It may prove useful in courses on topics ranging from criminal procedure to social epistemology to media criticism.

Listening Notes

  • ~4:45, host Gordon Katic (GK) speaks about the ideal of experts also being intellectuals. A thematic throughline of this series is how certain large-scale institutions and processes, such as the criminal legal system, create a structural hostility to intellectualism (especially as defined here, “a willingness to be puzzled”) in expert practitioners.
  • ~6:25, “forensic experts, the experts of crime scenes” — note that the word “forensic” often has a narrower connotation than its dictionary definition. Some forensic evidence has nothing to do with criminal investigation or proof at criminal trials. For instance, someone testifying as a handwriting analysis expert in a civil lawsuit about a handwritten will is purporting to offer forensic expertise (despite handwriting analysis almost always being unscientific and unreliable).
  • War on Terror context — students who were not yet news-conscious at the time may need a primer on how intense the atmosphere of anti-Muslim suspicion and hostility was in the early-mid 2000s. The Mayfield case usefully illustrates how such a cultural moment can seep into the judgment of experts.
  • ~40:00, Mayfield recounts how the Spanish police felt that the FBI’s approach to the fingerprint evidence was agenda-driven. Meanwhile, the concern that laypeople are prone to bias has often been featured in arguments against democratic participation in the criminal process (especially via the jury system). Contrasts like this one help illustrate how these podcasts are about “the politics of expertise.”
  • ~59:35, Dr. Edmond notes that fingerprints and ballistics have turned out to be “really quite good, but they’re not as good as they claimed.” The label “junk science” is often used to set up a binary distinction with what is imagined as proper, reliable forensic science like DNA. In reality, these forensic practices fall along a more nuanced spectrum, and their status is not static over time.
  • ~1:13:00, students who are avid listeners / viewers of true crime media may be prompted by the “pendulum” metaphor in the Flynn interview to critically reflect on how the genre has shaped their attitudes toward forensic science in the criminal legal system.

Discussion Questions

  • 1:00 – 5:00, on experts versus intellectuals: do you agree with GK’s distinction? Upon completing the episode (or the series), return to this segment and see if your view has changed.
  • The Mayfield case is striking because of the way investigators fixated on forensic evidence to the exclusion of what we might call “common sense.” Common sense sometimes keeps us in touch with basic plausibility, but common sense can also be a vessel for harmful myths, stereotypes, and reactionary attitudes. Indeed, it was common sense to investigators at the time that Mayfield’s religion made him a plausible suspect. How might the legal system better regulate “common sense” when it encompasses such upsides and downsides?
  • ~50:30, GK describes how forensic experts tend to be attached to police and prosecutors; Judge Gertner agrees and characterizes this as a “serious structural problem.” In civil cases, where there are generally no police or prosecutors, can you think of a similar structural problem?

  • Judge Gertner notes that the courts of appeal are not effective in reviewing and reversing trial courts’ acceptance of flawed forensic science. If appellate courts were authorized to review these issues more aggressively, what do you suppose would get better, or get worse? On balance, would this be an effective way of addressing the problems of forensic science?

Further Reading

From the show notes:

Additional resources:

Episode #2 of Series: Mesmerizing Convolutions: The Rise of Fingerprint Identification

This episode offers a social history not just of a specific forensic technique, but of the criminological understanding and societal attitudes that scaffolded its development over the centuries. It should prove especially useful for prompting students to think beyond questions of accuracy and reliability when evaluating the use of expert techniques in policing, courts, administration, and other public and private institutional contexts. Courses that would benefit from integrating perspectives from Science and Technology Studies / History and Philosophy of Science can accessibly accomplish this by assigning episodes 16 and 16.1 in tandem.

Listening Notes

  • ~16:30, Prof. Cole discusses how Bertillon used the tools of Lombrosian criminology because those were the available tools that made sense at the time. Criticism of Cesare Lombroso’s ideas did arise among his contemporaries, especially in France. Perceived imperatives of crime control and state administration have spurred the adoption of problematic or controversial approaches in the past, and may continue to do so contemporarily.
  • ~20:45, Prof. Cole notes that “… fingerprinting’s [early] victories come in the colonies, not in Europe, and principally colonial India.” Later, ~24:45, he touches on the stigma of being fingerprinted, i.e., being subjected to a process that is “for criminals” and incompatible with the dignity of citizens in a free society. A theme of this episode and this series is that even when the State deploys forensic techniques with rigorous concern for accuracy, these uses may still serve as a powerful conduit for the injustice or inequality of an existing political order. Note also the observation ~26:45 that “immigration is one of the main uses of all kinds of biometric technologies today.”
  • ~37:30, Prof. Cole points out the rhetorical force carried by “the word fingerprint [which] sort of became a metaphor for infallibility.” Even if the conduct of forensic science can become better-regulated, culture and rhetoric will continue to influence how legal decision-makers (not only jurors but institutional actors) react to it.
  • ~45:30, Prof. Cole suggests that there is a problem in supposing that a defendant “can always hire [their] own examiner, and if [the prosecution] made a mistake, [their expert] will catch it.” Further, ~49:30, Prof. Cole notes that forensic errors are more likely to go undetected for low-level crime, because accused persons in those situations tend to lack the resources and incentives to litigate such issues. Both of these points illustrate that a legal system’s reliance on adversarial structures (most pronounced in the United States) can not only fail to produce accurate results, but can exacerbate the disadvantages faced by the worst-off defendants.
    Discussion Questions
  • “The mesmerizing convolutions of these tiny ridges are sometimes called upon to determine people’s fates,” writes Simon Cole. This sentence may evoke an image of fingerprinting as palmistry, a practice encountered by the uninitiated as akin to oracular magic. Is this suggestion fair if, in fact, fingerprint identification proves empirically reliable to the extent that human error is minimized? Revisit this question at the end of the episode, thinking especially about how Brandon Mayfield might have regarded the thinking that went into the accusations laid against him.
  • ~35:15, GK remarks how stunning he finds it that practitioners and judges spent many decades failing to see the need for a rigorous, controlled evaluation of fingerprint identification’s reliability. Prof. Cole responds, “well, because you were raised in late 20th-century science.” Consider the possibility that our 21st-century scientific frame of reference may itself undergo transformative growth as new methodological possibilities emerge. Should this possibility change how we currently approach the task of evaluating and managing forensic techniques? What is the healthiest way for society to handle the risk that the knowledge we bring to bear on highly consequential decisions may be incomplete in ways we yet lack the perspective to appreciate?
  • ~47:15, Prof. Cole laments that “forensic science has been … regulating itself by scandal.” Legal scholars have mobilized this observation to argue that governments should establish forensic science regulatory bodies in order to set standards and regulate methods more proactively. What are some advantages of this proposal? What challenges might it encounter? What sorts of problems would it not address?

Further Reading

From the show notes:

Additional resources:

Episode #3 of series: Pathological: The Work of Dr. Charles Smith

This episode supplements the usual academic-interview style of the podcast series with a slower, especially vivid narrative interview. It is both the longest episode in this series and contains the most difficult material; while instructors should exercise discretion in assigning it, the emotional heft of the initial segment may lend special urgency to the overall lessons for many listeners. The combination of Tammy Wynne’s wrongful conviction story with Itiel Dror’s research on bias will help illustrate many of the subtler elements of problematic forensic-expertise culture, including how dynamics of marginalization influence the experience, and how more information can paradoxically lead to worse outcomes.

Listening Notes

  • Around 25:00, Tammy Wynne recalls being encouraged to take a plea deal rather than maintain her innocence; at ~26:30, she continues, “not one doctor would go against Smith … nobody wanted to help somebody on legal aid … [it seems like] if you’ve got money, then you can just pay your way through the courts.” Forensic expertise here can be seen as part and parcel of a legal structure that the poor and the marginalized have little hope of resisting.
  • The character traits of Charles Smith feature in the narrative frequently despite his personal absence from both the episode and many of the scenes recounted. Both Tammy Wynne and James Lockyer note how Smith “never showed up,” including to his own disciplinary hearing. He was not only “loved” (~45:30) but rhetorically deified by police, prosecutors, peers and clinicians; he “wanted to believe” in his own rightness and “engaged in a form of thinking” (~49:30) that led to desired conclusions and, perhaps most importantly, to adulation from respectable actors. The role of character and personal virtue in Smith’s case is, in a word, complicated.
  • “I want the hell that I went through to actually be worth the hell I went through,” concludes Tammy Wynne at 39:00. Her sharing of this story in such intimate detail was an act of considerable vulnerability and generosity. This, too, is part of the lesson of the episode and the series; part of becoming educated on a subject is learning to feel the moral and political stakes attached to it.

Discussion Questions

  • Near the beginning of the episode at ~5:30, Tammy Wynne recounts being labeled as a presumptive future child abuser, thanks to the statistical knowledge that an individual having experienced child abuse is a strong predictor of later committing it themselves. She flatly objects, “I don’t care about statistics.” How did you react to hearing her say this? How would you respond to someone who dismissed her response to empirical data as anti-science or anti-intellectual? Revisit these questions after finishing the interview with Itiel Dror, especially his comments at 1:02:30 & 1:08:30.
  • Charles Smith, who lacked even proper training in the specialty he held himself out as an expert in, represents an especially egregious case of forensic expertise gone wrong. As James Lockyer mentions, Ontario now has a mandatory forensic pathology training course, a peer review system, and other procedural safeguards. By contrast, at ~1:10:00, GK suggests that part of the problem of expertise is that highly siloed hyper-specialization inhibits practitioners from thinking holistically about their role and their work. Do we need better-trained technicians with better procedural guardrails, or a culture of expertise that better emphasizes humility, empathy, and interdisciplinarity? If the answer is “both,” is it possible to decrease the apparent tension between those two approaches?

Further Reading

From the show notes:

  • For more on the story of Tammy Wynne, read coverage from the Toronto Star, and a detailed rundown on the Charles Smith Blog by journalist and lawyer Harold Levy.
  • For more on Dr. Charles Smith, read the full report from the Inquiry Into Paediatric Forensic Pathology in Ontario, i.e. the Goudge Inquiry.
  • Finally, three papers from Dr. Itiel Dror helped us understand why this kind of thing might happen, including: his editorial in Science about bias in forensic experts; his detailed discussion of the bias blind spot and other problems in expert decision-making; and his ground-breaking research on forensic pathologists.

Additional Resources

  • Gary Edmond, Alice Towler, Bethany Gowns et al., “Thinking Forensics: Cognitive Science for Forensic Practitioners” (2017) — for those interested in a broader overview of cognitive-scientific lessons for forensics apart from what is highlighted in Itiel Dror’s work.
  • Alice Towler, David White, Kaye Ballantyne et al., “Are Forensic Scientists Experts?” (2018) — despite ‘yes’ being a useful shorthand framing device for this series, the real answer to the question posed in the paper’s title is closer to ‘sometimes, in some ways, but we ought not assume so.’
  • Charles Smith upended the lives of many other people besides Tammy Wynne. Anyone interested in engaging further with the stories of additional victims can find a good starting point for additional research in this short roundup from Global News.

Episode #4 of series: This Is Your Brain on Trial

This episode will hold special appeal for students whose interests include psychology, artificial intelligence, intellectual history, futurism, and abolitionist perspectives on incarceration. Equally important, the forensic technologies discussed here help shift the focus away from the investigative ‘whodunit’ nature of pattern-matching techniques and toward forensic expertise that is more diagnostic or predictive in nature. Despite notable differences in tone and topic, it pairs well with episode 17 as a thematic follow-up.

Listening Notes

  • ~12:45, Prof. Neal notes that judges have frequently reacted to challenged expert evidence by deciding that any shortcomings should be a matter of the evidence’s “weight, not [its] admissibility.” Students without a background in evidence law should understand that this phrasing means the evidence is admitted, and the court’s fact-finding body (either a jury or the selfsame judge) has free rein to make whatever they please of the challenged material. Any decisions they make about the weight of the evidence will typically be very difficult to overturn on appeal.
  • ~13:15, Prof. Neal speaks to the role of precedent in admitting expert evidence. Students without a background in civil procedure should be aware of the difference between persuasive authority and binding authority. A trial court in one state or province may look to how a neighbouring trial court has decided an evidentiary issue and take that decision as guidance — but since the one court has no direct power over the other, the example is merely persuasive and the subsequent court may choose to reach a different conclusion. By contrast, if a court is faced with an issue and learns that the court of appeals in its jurisdiction (or even a court of final appeal, like the Supreme Court of Canada) has previously decided that issue, the precedent decision has binding authority that the lower court must adhere to. Because of the wide variety of forensic techniques, it is much likelier that a given court will only have persuasive authority to guide its decision on the admissibility of expert evidence, meaning that evidence admitted in one case need not necessarily be admitted in another that happens elsewhere.
  • ~26:15, the colourful characterization of algorithmic decisions as embodying “vibes-based judgment” segues into a more serious point about how automation and high-tech sophistication can be obfuscating, which makes evidence (and therefore law) more resistant to democratic oversight and accountability. An overarching theme of the Darts & Letters and Cited podcasts is critique of elite technocracy — examination of how it goes wrong, and where it sits in uncomfortable tension with the ideals of democratic society. The early episodes of Cited are especially focused on these topics.
  • From ~54:30 and on, Prof. Scull offers an account of how explanations of the mentally ill as treatment-resistant due to being “biologically defective” combined with prevailing sociopolitical conditions in various 20th-century societies to spawn programs of eugenics, overzealous medical incarceration, and outright genocide. Even assuming (generously) that the conditions which might give rise to these nightmarish outcomes no longer exist in the present day, the notion of biological defects leading to madness or criminality has a renewed presence that worries some scholars. On the other hand, these arguments are now most visibly used in law for humane ends, especially to mitigate criminal responsibility. It remains a matter of debate whether there will be a “double-edged sword” quality to these brain-based defenses.

Discussion Questions

  • ~14:15, producer Jay Cockburn and Prof. Neal discuss “jurors who have more interest in science” being potentially better at evaluating expert evidence, and ~14:45 they muse on “whether [a jury full of especially educated or knowledgeable people] is a jury of your peers … should the jury be representative of the entire public?” Perhaps surprisingly to non-lawyers, courts have sometimes held that it is misconduct for a juror with special training or experience to bring their expert knowledge into jury deliberations, because all the evidence in a case is supposed to go through open court with a fair opportunity to object to it. How do you feel about, say, a nurse in the jury room drawing on their medical knowledge to change other jurors’ minds about the cause of death in a case? Would your feelings change if you were the one on trial? See Further Reading for more.
  • ~53:30, Prof. Scull laments the National Institutes for Mental Health moving away from understanding mental illness in terms of social causation and towards somatic causes; note also the observation ~32:15 that “the very idea of accuracy ends up being this filter that helps the gaze of the system hive and funnel away social context.” Isn’t the lesson of contemporary neuroscience that mental illness necessarily arises in the brain? If so, what’s wrong with a reductionistic understanding of these issues? Do you think that it makes sense to take a “both / and” approach when it comes to brain science and social context, or does that end up being incoherent?
  • Consider the idea ~37:00 that algorithmic brain-based risk assessment offers a “small benefit … cutting down on those false positive [designations of high-risk status], set against a much larger systemic reinforcement of the overall practice” of seeking public safety through incarceration. Each false positive can have a huge impact on the life of an individual accused. How do you respond to the idea that it is morally imperative to improve the accuracy of this system even if it doing so makes it harder in the long run to imagine and advocate for alternatives?

Further Reading

From the show notes:

Additional Resources

Episode #5 of series: The DNA of a Wrongful Imprisonment

This episode revisits many of the themes from Episodes 16 and 16.1 while heightening the stakes by focusing on a technology that, unlike many other methods in forensics, is understood to be scientifically sound. This may be pedagogically helpful for instructors who wish to prevent students from forming the misapprehension that the problems of forensic science can be entirely solved by strengthening the reliability of forensic methods. Students of criminology, criminal procedure, and sociology will find it especially relevant. The discussion in the final third of the episode on an algorithmic genetics-based image-production technology creates some thematic ties to Episode 51.

Listening Notes

  • ~13:40, Kimani Boden discusses the lack of corroboration for the DNA evidence in Farah Jama’s case. In many other cases, a DNA match is itself the corroborating evidence to the testimony of a complainant. Over time, corroboration requirements have advanced and retreated in various areas of evidence law, and their presence as well as their absence can promote the overall interests of justice. Canada’s criminal code was amended in the 1980s to specify that corroboration is not required where an accused is charged with sexual assault or similar listed offence, a change made to redress observed injustices encountered by survivors. Meanwhile, many US states require corroboration for the notoriously unreliable testimony of “jailhouse” informants.
  • ~22:40, Stephen Cordner mentions that the DNA evidence “sort of drove common sense out of people’s minds.” Evidence law scholars regard the notion of common sense with complicated ambivalence. On the one hand, “common sense” can often be a refuge for invidious myths and stereotypes in the reasoning of both jurors and judges. On the other hand, slowing down one’s thinking and assessing a story for plausibility is another way of exercising common sense — and, following the old saying, it is sadly not all that common, as illustrated by Jama’s conviction.
  • ~39:20, Prof. M’charek describes race as a “co-production” between genetics and forensics. The histories of race as a concept and criminology as a discipline are extensively intertwined, notably through the work of Italian surgeon and phrenologist Cesare Lombroso, often dubbed the “father of criminology.” Note also that “co-production” here is being used in the sense that STS (Science and Technology Studies) scholars usually mean it, rather than in the policymaker’s sense of a multi-stakeholder collaboration.

Discussion Questions

  • Think back to the important detail of Farah Jama’s case first recounted ~4:18 — that Jama had had a prior interaction with law enforcement regarding a later-withdrawn complaint of sexual misconduct. Not only did that interaction lead to the DNA contamination underlying the evidence against Jama, but the very fact of the police interaction itself may have stymied Jama’s defence team for fear of how the jury would react to hearing it (as surmised by Kimani Boden ~10:50). Imagine instead that the DNA contamination had been totally random, and no other police investigation or interaction involving Jama had happened. Do you think that prosecutors would have persisted with their case against Jama purely on the DNA evidence in such a situation? What lessons can be drawn from the fact that a history of police interactions may cause a fundamentally implausible suspect to appear plausible?
  • ~16:15, Kimani Boden laments the role of DNA in contributing to a policing culture (and a political culture) fixated on easy, authoritative solutions. Reflect on the role of DNA in exonerating wrongfully accused and wrongfully convicted criminal defendants — a context where its authoritativeness serves an important good purpose. Must we simply accept the good with the bad when it comes to DNA evidence in the criminal legal system? Are there deeper roots to the problems here that might be meaningfully addressed?
  • Did you find Parabon Nanolabs’ Ellen Greytak to be convincing in her defense of the company’s composite-generation technology ~41:00? Why or why not? What do you make of Greytak’s mention of “objective knowledge,” and how does it square with Prof. M’charek’s comments earlier in the episode?

Further Reading

From the show notes:

Additional Resources