Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6) pp.683–695

I was particularly interested to read of the example of faculty at Texas A&M University being measured against their net contribution (or not) to the University’s financial position. I am aware of similar concerns amongst academic colleagues at the University of Edinburgh. When the University recently announced that they were investing millions of pounds in a new lecture capture service, academic staff raised concerns that their performance metrics would be used to inform their annual review. It perhaps doesn’t help that the current lecture capture service employed by the University is called Panopto.

It’s always nice to see a diagram in an academic paper. Clow’s Learning Analytics Cycle draws on Campbell and Oblinger’s (2007) five steps in the learning analytics process: capture, report, predict, act and refine.

Predictive modelling

Clow outlines the practical differences between predictive modelling and the ‘human’ equivalent of a teacher giving extra help to students they notice may be struggling as thus:

  1. “the output of predictive modelling is a set of estimate probabilities and … many people struggle to correctly understand probabilities” (p.687)
  2. the student data is made available to others (not just the teacher)
  3. the data can “trigger actions and interventions without involving a teacher at all”.

This last point feels significant as it corresponds with many teachers fears that the locus of power is shifting away from the teacher and towards the faceless administrator. It was therefore interesting to read of the Course Signals project at Purdue University. Perhaps integral to the success of the project is the fact that the “the teacher is central to the process and uses their judgement to direct students to appropriate existing resources within the university” (p.688).

This discussion also prompted some other questions:

  1. as predictive modelling ensures efforts are aimed at ‘marginal students’, is this at the expense of other students? (the experience of Signals at Purdue suggests this doesn’t have to be the case)
  2. could an unintended consequence of predictive modelling be a trend towards more conservative choices regarding courses? In other words, would institutions end up prioritising existing courses (because we have data for these) against new courses?

 

Social Network Analysis

It’s hard to think of Social Networks without thinking of The Social Network. Nevertheless, it was interesting to read of the following SNA projects:

What wasn’t discussed here, but is of interest, is if/how students behave differently on ‘professional’ social networks (eg forums where they are being assessed, forums which their tutor can access) and ‘personal’ social networks (eg Facebook). Does ‘editing’ oneself in the former encourage similar behaviour in the latter?

The possibility of a richer (computational) analysis of textual data is an interesting field of study and Clow refers to the Point of Originality tool which uses the WordNet database to identify originality in key concepts. Clow notes “a strong correlation between originality scores in the Point of Originality tool and the grades achieved for the final assessment and also between the originality of their writing  and the quantity of their contributions online” (p.690). However, it is important to remember that correlation does not equal causation.

When Clow suggests that “perhaps the greatest potential benefit [of recommendation engines] lies in more open-ended and less formal learning contexts” (p.691) it’s hard to disagree. However, warnings about the dangers of Filter Bubbles should be heeded here too.

Finally, I was most struck by the following point made by Clow in the Discussion section:

“the opportunity to learn by making mistakes in a safe context can be a powerful learning experience, and not many learners are happy to have their mistakes kept on record for all time” (p.692).

How can we ensure the student data we track and measure, and present to administrators, teachers, and (at times) the students themselves benefits their learning, if by the very nature of the task we are performing, we are creating a relationship of mistrust which compromises the learning at the outset? In other words, the Panopticon (below) doesn’t look to me like the optimal space for learning.

The Panopticon

The Panopticon