Analytics in learning
Big data and data analytics are an ever growing part of society, now that so much of our everyday lives take place online. They are also becoming more commonplace in education, which has become more ‘digitised and datafied’ (Williamson 2017). This week, the articles we have been reading are around learning analytics, which is described as ‘ the collection and (automated) analysis of data concerning learners’ background, behaviours and progress’ (Wilson et al. 2017). Learning analytics is a very interesting term which led me to wondering; why should we analyse the data around student learning? And how do we analyse what our students learn?
In a face to face setting, it isn’t possible to collect data about our students in a similar way, yet we seem satisfied with assessments to make sure the students have reached the appropriate level of learning and knowledge. Why do we then feel the need to collect data and analyse it when learning takes place online? I wonder if this is rooted in the idea that digital learning is ‘second best’ to the traditional, face to face model of teaching students (Bayne et al. 2020) and once the teaching takes place in a digital setting, we need additional proof that the learners are in fact engaging the way we want them to. As is made clear from the sample study in Wilson et al. (2017), even if we do analyse the students’ behaviour, it doesn’t necessarily give us information to explain their success or failure in their studies. What we do learn from gathering data in this sample, is how very different the behaviours of the students can be during the same course with the same objective; the diversity of learning is possibly the most interesting thing this study shows. The diversity also makes the learning analytics almost useless in the case study used (Wilson et al. 2017)
Another avenue of thought here is the idea that analytics are available because the learning takes place digital, so we should use it. Which seems related to the thought on digital enhancement in Bayne’s article (2015) on technology-enhanced learning where the question comes up; if we can change (enhance) something, does that mean we should? The technology to collect and analyse date is available and this has become common practice in business analytics, but does it then follow we should use the same methods in education? Gasevic et al. (2019) certainly think so in their article around the adoption of learning analytics in education, where they promote the use of a business analytic style model in education. The rational for adopting learning analytics takes up three lines in the article and comes down to the teacher being less able to read feedback cues when the teaching takes place online (Gavesic et al. 2019). As we have seen earlier in the course though, teaching does not necessarily take place online; students have their own physical learning space as well as an online presence. So collecting data purely from their online presence, does not give us a clear picture of how their learning takes place.
Data from students is mostly gathered through Learning Management Systems (LMSs), which track the students as they click their way through messages from tutors, discussions on forums or resources within the software (Wilson et al. 2017). This is nicely illustrated in the case study within the article, where we can see how many students accessed the forums, used the resources etc. in different weeks of the course. The term ‘activity analytics’ (Wilson et al. 2017) is a good way to describe what data is actually being presented here. This roadmap of clicks shows which page the students visited and how many times, but how does this actually equate to learning rather than activity?
I don’t think there is a direct correlation between visiting the different websites and learning taking place. In my experience as a student on a digital course, I have found a good portion of my learning takes place offline. There is a part takes place online of course; the course page that I read, the forums which I read and contribute to, looking up the resources and downloading them, not being a fan of reading on a screen and posting the blogs as well as reading the replies. But this, right now, is a big part of my learning; having read the articles this week and having spent some time reflecting on them, also offline, I am now writing this blog offline. The writing means I have to think about the articles, put them in perspective and make sense of them in the bigger picture of the course, none of these activities can be tracked by data in an LMS but they are a big and very important part of my learning.
Learning analytics can be useful for keeping track of the activities of students, but this does not necessarily equate to engagement in learning. Furthermore, everyone learns in their own way and it will therefore be a very difficult undertaking to use the data from students as an indication or role model for future students.
Bayne, S. (2015) What’s the matter with ‘technology-enhanced learning’? Learning, Media and Technology, 40:1, 5-20, DOI: 10.1080/17439884.2014.915851
Bayne, S., et al. The Manifesto for Teaching Online, MIT Press, 2020. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ed/detail.action?docID=6317960.
Created from ed on 2021-01-16 00:37:00.
Gasevic, D., Tsai, Y-S., Dawson, S. & Pardo, A. 2019. How do we start? An Approach to Learning Analytics Adoption in Higher Education, International Journal of Information and Learning Technology.
Williamson, B. 2017. Introduction: Learning machines, digital data and the future of education (chapter 1). In Big Data and Education: the digital future of learning, policy, and practice.
Wilson, A., Watson, C., Thompson, T.L., Drew, V., & Doyle, S. 2017. Learning analytics: challenges and limitations. Teaching in Higher Education. 22(8). pp. 991-1007
An interesting idea linking learning analytics with a deficit model of digital education. One of the drivers, as you note later, for learning analytics may well be ‘because we can’ or that VLEs collect this data/ material so what should we do with it. Data from in-person/ on-campus teaching and behaviours is also increasingly being collected: of student attendance, entering & leaving libraries, study space usage and so on. See https://new.siemens.com/global/en/markets/smart-campus/smart-university-campus.html for a near future imaginary of the campus. And, as with online data, we can’t really measure what learning is happening. Your point on the diversity of a course cohort making data almost worthless is an important one as course data involves very small numbers of students making it difficult to generate useful insights about course design and pedagogy. As you describe, a lot of learning activity is hidden as it is either offline or takes place away from the VLE (think of how the Tweetorial activity would show up on the University’s learning analytics let alone informal study groups conducted via WhatsApp or Discord or SnapChat, etc.. I’m not even sure that keeping track of student activities on the LMS is all that useful (except where there is a concern with a student not engaging at all and so there may be a pastoral issue) – although analytics data may be useful in larger courses where keeping track of students may be an issue (but again, the generation of useful information on the design of the course, etc. may be very hard to identify). Good post again. Do remember to consider the multimodal possibilities of the blog for including audio visual material as well as text.