In this two-part Spotlight on LOUISA post, Data and Equality Officer, Katie Grieve, explores how learning technology decisions can impact attainment gaps. This post builds on Katie’s Post 1: Exploring the impact of dropbox practice on attainment gaps, which introduces the intricate relationship between learning technology choices, anonymity, and attainment gaps, laying the groundwork for more nuanced understandings of educational equity. In this second post, Katie investigates the relationship between assessment design and student outcomes, and shares the findings on the relationship between dropbox number and attainment gaps.
Methodology
The dataset was drawn from administrative data held by the institution on attainment gaps by School, in the Annual Monitoring Report provided by Student Analytics, Insights and Modelling in Registry Services. Snapshot data was gathered as part of a University’s LOUISA project, showing dropbox use across Schools and Deaneries at the University of Edinburgh in the academic year 2023-24, with these decisions being implemented at a School-level and impacting every course.
The study notes the potential for course-level exceptions in dropbox number, but recognises this was likely a very small proportion, with the majority of courses likely aligning with the School-level position. Table 1 depicts the results of this analysis. Schools with “Other” numbers of dropboxes involved either more than two dropboxes, or dropbox use varied significantly in the School. The former is theoretically similar to the two dropbox workflow, with the additional dropboxes often hidden from students. However, they were excluded from the study to avoid misinterpretations. The latter would be too difficult to consistently predict without course-level analysis, and so was also excluded.
To explore as wide a range of data as possible, all three attainment gap metrics available in the institutional data were explored. These were:
- Course Pass Rate (%)
- Average Course Mark (%)
- High Classification Award (%)
The latter is the most common metric included in attainment gap discussions, so correlations and boxplots in the following section are for High Classification Award. This study explored these metrics for gender and ethnicity only, with disability and age excluded for various reasons, including variation in sample size across characteristic.
Boxplots were created comparing the three metrics across characteristic (for example, male and female students) and School (grouped into one and two dropboxes). Point-biserial correlation tests were run to explore if the number of dropboxes had a relationship with the attainment gap, across characteristic. For example, if the correlation was positive, but smaller for white students than BAME, then this indicates two dropbox Schools (with anonymity) had a greater, more positive impact on BAME students than white.
Findings
1. Gender
Both male and female students had higher degree award outcomes in the two-dropbox Schools. Figure 1 shows female students had less overlap between one and two-dropbox Schools than male, with the mean award outcome in one-dropbox Schools 90% for female students and 87% for male. In two dropbox Schools, this gap widens to 93% for female and 87% for male.
Correlations were 0.24 for female students and 0.06 for male, showing a larger increase in award outcomes in two dropbox Schools for female students. This was the largest correlation gap, at 0.19, however female student performance increased more significantly with two dropboxes across all three metrics.
Note, the gender attainment gap shows male students perform worse across the three metrics, so an increase in female performance would actually widen the gap. However, early National Union for Students research (2008) found anonymous assessment was more beneficial for female students, as upheld by this preliminary research, and not a contributing factor to male student under-performance.
2. Ethnicity
Both BAME and white students had higher award outcomes in two-dropbox Schools. Figure 2 shows a greater spread of High Classification Award proportions for BAME students in one-dropbox Schools, with the mean award outcomes at 83%. This was 91% for white students in one-dropbox Schools. In two-dropbox Schools, mean outcomes were 90% for BAME students and 94% for white.
Correlations were 0.42 for BAME students and 0.38 for white. However, the largest correlation difference was for Course Pass Rate, at 0.10. Again, these results could indicate BAME student award outcomes improved in two-dropbox Schools at a larger rate compared to white students.
Summary and limitations
The overall correlations between dropbox number and the three metrics were quite small. This was expected, with there being little indication that one or two-dropbox Schools would have differing award outcomes. However, there were some significant differences in correlation between characteristics, for example, 0.24 for female students and 0.06 for male. This indicates there was a much stronger impact of two dropboxes on award outcomes for female students than male.
The statistical significance of correlation differences such as these largely depends on sample size. The sample size for this introductory research was relatively small, as it contained only one year of data. Future research could investigate past dropbox use in Schools, and create a historic dataset to examine trends overtime, to make conclusions stronger.
These preliminary findings show that there is a relationship between dropbox number and attainment gaps, indicating the inclusion of learning technology in conversations about anonymity and attainment gaps may provide further understanding to many of the debates. It provides evidence of the importance of local learning technology decisions on impacting sector-wide equality and diversity challenges, and thus strategic planning at an executive level in the institution. Although not a comprehensive study, this preliminary research solidifies the need for broad perspectives when combating historic equality and diversity issues, and creativity in exploring speciously small decisions, when they can in fact have potentially long-lasting ramifications on student outcomes and attainment gaps.
If you would like to get involved with the LOUISA project, contact LearnFoundations@ed.ac.uk
You can keep up to date with news from LOUISA here: Project News
References
Borkin, H. (2020). Locating bias in higher education marking practices, Locating bias in higher education marking practices. Advance HE. Available at: https://www.advance-he.ac.uk/news-and-views/locating-bias-higher-education-marking-practices [Accessed: 26 August 2025].
Connor, H. La Ville, I., Tackey, N. & Perryman, S. (1996). Ethnic minority graduates: Differences by degrees. Available from: https://www.employment-studies.co.uk/system/files/resources/files/309.pdf [Accessed 26 August 2025]
National Union of Students. (2008). Mark My Words, Not My Name: The Campaign for Anonymous Marking. Available at: http://samairaanjum.weebly.com/uploads/1/0/5/2/10526755/markmywordsbrief1-1.pdf [Accessed: 26 August 2025].
Pitt, E. and Winstone, N. (2018). The impact of anonymous marking on students’ perceptions of fairness, feedback and relationships with lecturers, Assessment & Evaluation in Higher Education, 43(7), pp. 1183–1193. doi:10.1080/02602938.2018.1437594.
Stonewall, J. et al. (2018). A review of bias in peer assessment, 2018 CoNECD – The Collaborative Network for Engineering and Computing Diversity Conference Proceedings [Preprint]. doi:10.18260/1-2–29510.
Whitelegg, D. (2015). Breaking the feedback loop: problems with anonymous assessment, Planet, 5(1), pp. 7-8. https://doi.org/10.11120/plan.2002.00050007.
Katie Grieve
Katie is the Data and Equality Officer for the Information Services Group. A Mathematics graduate from the University of Edinburgh, she used data science techniques to interpret demographic and service user data, to better understand the experiences of diverse groups of staff and students. In October, she began a PhD in Data Visualisation.
