But you know that experiment doesn’t replicate, right?
Too often the label “does not replicate” is used to imply bad scientific practice on behalf of the original experimenters, be that suspect statistical manipulations, ‘Hypothesising after the results are known’ or downright fabrication. “But it doesn’t replicate” is a comment guaranteed to embarass a speaker during a conference Q&A, or to trip up philosophers like myself who rely on empirical work to support their theories. Worse, it is used to undermine the opinions of expert researchers, leading to increased public distrust in science.
This project takes a fresh perspective on the “crisis” discourse currently dominating scientific debates on replication by highlighting and developing the myriad goods that come from replication work. Philosophers of science have long acknowledged that failed replications are a critical part of the scientific process, allowing researchers to understand the scope and boundaries of their theories, to recognise variation within participants, and to identify and isolate essential parts of the experimental methodology. We will examine questions such as:
- How can replication attempts be used to refine hypotheses?
- What can failed replications tell us about the scope of a hypothesis?
- At what point do we decide that an experimental result was a false positive? What features do we look for in failed replications to help us make this judgement
The second set of questions examined in the project concern what constitutes a replication experiment (failed or successful). How does a scientific community come to decide that a set of experiments are sufficiently similar in method to the original so as to constitute replication attempts of it? How do scientists decide how close the results of a ‘replication’ have to be to the original to be considered a successful replication of it? Deciding whether a set of experiments constitute replications of an original is a highly theory-laden enterprise, and many disputes about the replicability of some experiment come down to disagreement about whether the purported replication attempts match the original in the right ways. Questions here include
- (Descriptive): what features do scientists use to ascertain whether a set of experiments constitute attempted replications of an original? By what means do they decide whether the outcomes of the two experiments match, or are dissimilar?
- (Normative): what features ought scientists use to resolve the above disputes?
The project will consist in a small workshop with philosophers and psychologists focussing on the above questions. In April 2023 there will be an open conference to discuss the findings of the workshop. Throughout the project period there will be opportunities for participants to engage in outreach activities in the community, as well as to present work at international conferences.
PI – Dr. Suilin Lavelle
I am a lecturer in philosophy of mind and cognition at the University of Edinburgh. My interest in replication began with frustration at the ‘name and shame’ culture surrounding researchers whose work reportedly did not replicate, compounded by increasing annoyance when psychologists told me the work I cite in my research on social cognition was ‘not replicable therefore unreliable’. Through joint supervisory work with graduate students in linguistics and psychology, I have learned more about the conflicting pressures on junior researchers surrounding open science. These factors motivated me to look more closely at replication practices, what sort of epistemic tool researchers perceive replication to be, and how replication disputes are resolved. I have several papers in preparation on this topic as well as a publication in the British Journal for the Philosophy of Science.
I am married to a qualitative researcher which gives me access to a completely alternative (competing?) set of views on the value of replication.
- (forthcoming )When a crisis becomes an opportunity: the role of replications in making better theories. British Journal for the Philosophy of Science.
- (2019) Do babies replicate? The impact of the replication crisis on philosophy: two case studies. Peer-reviewed symposium, Annual meeting of the British Society for Philosophy of Science, Durham, UK