“A natural experiment using Scottish clinical data to estimate the real-world effectiveness of adjuvant chemotherapy in breast cancer patients”

See the project page for more and project outputs.

The need for efficient production of real-world evidence

Rigorous assessments of the causal impact of healthcare interventions require unbiased research designs. Randomised controlled trials (RCT) are the gold standard for reliably measuring treatment effects and constitute the primary source of evidence to inform decision-making in health care. However, RCTs are costly, may not be always feasible for practical or ethical reasons, and may have poor external validity. In an attempt to improve the efficiency and real-world relevance of healthcare research, better exploitation of pre-existing or routinely collected healthcare data has been advocated. Observational studies, in which assignment to treatment is non-random, may be a less expensive and more pragmatic source of evidence. However, these approaches rely on strong and sometimes unverifiable assumptions and suffer from various sources of bias. The nonrandomized or “quasi-experimental” designs available to researchers differ in their ability to mirror random assignment under reasonable assumptions; their historical limitation has been that their applicability largely depends on data availability and quality.  The new revolution in digital healthcare information coupled with novel analysis methods means that such methods may now offer real solutions to evidence generation.

Regression Discontinuity – the method

The quasi-experimental regression discontinuity (RD) design has been widely used in social sciences1-4 and has been heralded as a simple to implement and transparent method for providing “real world” effects of treatments, but is underused in healthcare5-9. RD applies when participants are assigned to an intervention using a cut-off value (or threshold) of a continuous assignment variable, e.g. a risk score or test result. The treatment effect is estimated by comparing outcomes in individuals who lie just below the cut-off with those just above it; under several assumptions, any discontinuity in the outcome at the cut-off can be attributed to treatment.

The clinical need for improved evidence

The use of adjuvant chemotherapy after surgical treatment of early breast cancer is a major contributor to the reduction in mortality from breast cancer over the last three decades. A global collaboration of trialists published a definitive individual patient meta-analysis of 100,000 women with breast cancer, concluding that chemotherapy reduces the risk of dying from breast cancer by about a third10. However, the historical clinical trials upon which this evidence relies were performed in highly selected patient populations including few patients older than 70. In the real world we treat patients who would never have been included in those trials due to advanced age, comorbidity, frailty, socioeconomic status or even ethnicity, but such patients are increasingly being treated worldwide with toxic chemotherapy without direct supporting evidence. Recent attempts to conduct further randomised controlled trials in such patients have failed due to poor recruitment, presumably due to a perceived lack of equipoise11. It is therefore unknown whether these patients benefit or are harmed by chemotherapy, and alternative methods for measuring their outcomes from treatment are urgently needed. As the decision to proceed with chemotherapy is partly based on specific values of a continuous score, this clinical question appears well suited to the RD method.

A unique opportunity in Scotland

The very high quality and pre-existing linkage of Scottish healthcare and cancer registration datasets makes Scotland an ideal testbed to evaluate a new method to estimate effect sizes from routine data to resolve the above question, which has been unanswerable for many years internationally. We also need to support projects of this nature in Scotland to develop researchers with the necessary data analytic skills, a high priority for the Farr Institute.  The results of this project will have direct clinical impact on Scots with breast cancer, who currently face treatment decisions made in the face of uncertain evidence. This project will also provide feasibility data to support a larger UK-wide study of similar design, and will have immediate international impact on the ongoing fierce debate about using chemotherapy in this situation.

Aims

1. Measure the benefit from chemotherapy in a real-world early breast cancer population
2. Validate commonly used web-based benefit prediction decision tools in Scotland
3. Determine the feasibility of using the regression discontinuity design in this context

Outputs

1. Lee, D.S. and Lemieux, T. Journal of Economic Literature, 2010. 48(2): 281-355. 2. Cook, T.D. J of Econometrics, 2008. 142(2): 636-654. 3. Imbens, G.W. and Lemieux, T. Journal of econometrics, 2008. 142(2): 615-635. 4. Hahn, J., P., et al. Econometrica, 2001. 69(1): 201-209. 5. Venkataramani, A.S., et al. BMJ 2016; 352 :i1216. 6. O’Keeffe, A.G., et al. BMJ, 2014. 349: g5293. 7. Bor, J., et al. Epidemiology, 2014. 25(5): 729. 8. Geneletti, S., et al. Statistics in medicine, 2015. 9. Moscoe, E., J., et al. Journal of clinical epidemiology, 2015. 68(2): 122-133. 10. EBCTC. The Lancet, 2012. 379(9814): 432-444.