Any views expressed within media held on this service are those of the contributors, should not be taken as approved or endorsed by the University, and do not necessarily reflect the views of the University in respect of any particular issue.

Outcomes measurement in cancer

Outcomes measurement in cancer

Methods for the measurement of outcomes in cancer patients

The SATURNE project

“A natural experiment using Scottish clinical data to estimate the real-world effectiveness of adjuvant chemotherapy in breast cancer patients”

See the project page for more and project outputs.

The need for efficient production of real-world evidence

Rigorous assessments of the causal impact of healthcare interventions require unbiased research designs. Randomised controlled trials (RCT) are the gold standard for reliably measuring treatment effects and constitute the primary source of evidence to inform decision-making in health care. However, RCTs are costly, may not be always feasible for practical or ethical reasons, and may have poor external validity. In an attempt to improve the efficiency and real-world relevance of healthcare research, better exploitation of pre-existing or routinely collected healthcare data has been advocated. Observational studies, in which assignment to treatment is non-random, may be a less expensive and more pragmatic source of evidence. However, these approaches rely on strong and sometimes unverifiable assumptions and suffer from various sources of bias. The nonrandomized or “quasi-experimental” designs available to researchers differ in their ability to mirror random assignment under reasonable assumptions; their historical limitation has been that their applicability largely depends on data availability and quality.  The new revolution in digital healthcare information coupled with novel analysis methods means that such methods may now offer real solutions to evidence generation.

Regression Discontinuity – the method

The quasi-experimental regression discontinuity (RD) design has been widely used in social sciences1-4 and has been heralded as a simple to implement and transparent method for providing “real world” effects of treatments, but is underused in healthcare5-9. RD applies when participants are assigned to an intervention using a cut-off value (or threshold) of a continuous assignment variable, e.g. a risk score or test result. The treatment effect is estimated by comparing outcomes in individuals who lie just below the cut-off with those just above it; under several assumptions, any discontinuity in the outcome at the cut-off can be attributed to treatment.

The clinical need for improved evidence

The use of adjuvant chemotherapy after surgical treatment of early breast cancer is a major contributor to the reduction in mortality from breast cancer over the last three decades. A global collaboration of trialists published a definitive individual patient meta-analysis of 100,000 women with breast cancer, concluding that chemotherapy reduces the risk of dying from breast cancer by about a third10. However, the historical clinical trials upon which this evidence relies were performed in highly selected patient populations including few patients older than 70. In the real world we treat patients who would never have been included in those trials due to advanced age, comorbidity, frailty, socioeconomic status or even ethnicity, but such patients are increasingly being treated worldwide with toxic chemotherapy without direct supporting evidence. Recent attempts to conduct further randomised controlled trials in such patients have failed due to poor recruitment, presumably due to a perceived lack of equipoise11. It is therefore unknown whether these patients benefit or are harmed by chemotherapy, and alternative methods for measuring their outcomes from treatment are urgently needed. As the decision to proceed with chemotherapy is partly based on specific values of a continuous score, this clinical question appears well suited to the RD method.

A unique opportunity in Scotland

The very high quality and pre-existing linkage of Scottish healthcare and cancer registration datasets makes Scotland an ideal testbed to evaluate a new method to estimate effect sizes from routine data to resolve the above question, which has been unanswerable for many years internationally. We also need to support projects of this nature in Scotland to develop researchers with the necessary data analytic skills, a high priority for the Farr Institute.  The results of this project will have direct clinical impact on Scots with breast cancer, who currently face treatment decisions made in the face of uncertain evidence. This project will also provide feasibility data to support a larger UK-wide study of similar design, and will have immediate international impact on the ongoing fierce debate about using chemotherapy in this situation.


  1. Measure the benefit from chemotherapy in a real-world early breast cancer population
  2. Validate commonly used web-based benefit prediction decision tools in Scotland
  3. Determine the feasibility of using the regression discontinuity design in this context

Complete Project Summary



Independent validation of the NHS PREDICT breast cancer prognostication and treatment benefit prediction tool using the Scottish cancer registry.

E Gray, J Marti, D Brewster, J Wyatt, D Cameron, PS Hall.

1st UK International Breast Cancer Symposium.

Breast Cancer Res Treat (2018)

Full paper:

Gray, E., Marti, J., Brewster, D.H., Wyatt, J.C. and Hall, P.S., 2018. Independent validation of the PREDICT breast cancer prognosis prediction tool in 45,789 patients using Scottish Cancer Registry data. British journal of cancer, 119, 808–814 


Real-world evidence of the effectiveness of adjuvant chemotherapy for early stage breast cancer from Scottish routine data

Poster presented at the 2018 Conference of the National Cancer Research Institute

Full paper in forthcoming


1. Lee, D.S. and Lemieux, T. Journal of Economic Literature, 2010. 48(2): 281-355. 2. Cook, T.D. J of Econometrics, 2008. 142(2): 636-654. 3. Imbens, G.W. and Lemieux, T. Journal of econometrics, 2008. 142(2): 615-635. 4. Hahn, J., P., et al. Econometrica, 2001. 69(1): 201-209. 5. Venkataramani, A.S., et al. BMJ 2016; 352 :i1216. 6. O’Keeffe, A.G., et al. BMJ, 2014. 349: g5293. 7. Bor, J., et al. Epidemiology, 2014. 25(5): 729. 8. Geneletti, S., et al. Statistics in medicine, 2015. 9. Moscoe, E., J., et al. Journal of clinical epidemiology, 2015. 68(2): 122-133. 10. EBCTC. The Lancet, 2012. 379(9814): 432-444.

Leave a reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>


Report this page

To report inappropriate content on this page, please use the form below. Upon receiving your report, we will be in touch as per the Take Down Policy of the service.

Please note that personal data collected through this form is used and stored for the purposes of processing this report and communication with you.

If you are unable to report a concern about content via this form please contact the Service Owner.

Please enter an email address you wish to be contacted on. Please describe the unacceptable content in sufficient detail to allow us to locate it, and why you consider it to be unacceptable.
By submitting this report, you accept that it is accurate and that fraudulent or nuisance complaints may result in action by the University.