Adaptive learning pilots – where everyone learns to adapt

Having returned from the EDUlearn conference in Barcelona in early July 2015, I’ve now managed to re-acclimatize to the Scottish weather and to clear out most of my hundreds of emails. The moment is right therefore to try and capture a few reflections of my own, both on the conference, and on the Adaptive Learning pilot here at the University of Edinburgh.

Together with a colleague in GeoSciences (Eduardo Serafin), I submitted two papers accepted for the oral presentation strand focused on ‘flipped classroom’ teaching. The papers were:

Reflections on a student experience
Two differing pedagogies – a comparative case study.

The conference organizers then invited me to chair the session, which comprised five presentations, each to some degree relating to flip classroom teaching.

For anyone interested in EDUlearn, it is a conference of about 600 delegates, with quite a sizable number coming from southern and eastern Europe, Africa and Asia. There were fewer delegates from Russel Group institutions, and the fifteen minute presentation time felt rather too short for some of the more interesting analysis and project developments.

To provide some background for our own papers, it is worth explaining that The University of Edinburgh has been pursuing a number digital pilots that we hope will help improve our understanding, capacity and quality in delivering assessment and feedback to improve the student experience. For this one, we chose to partner with a locally based company, CogBooks, who offer an Adaptive Learning tool for online course delivery. Our GeoSciences School were willing collaborators, choosing to launch a second semester of lectures through CogBooks from the early part of 2015.

Over 140 students participated via two courses, Global Tectonics and the Rock Cycle (GTRC) and Introduction to Geophysics (IG) where the former course used the material to support lectures, and the second to implement a flipped classroom model.

The adaptive learning tool continuously captures click track data relating to students’ use of text, graphical and animated content. A dashboard of usage and progress through online material is available to both staff and students in real time, so that students can monitor their own activity, pick up where they last left off, and staff can reinforce or adapt teaching face to face, as necessary.

Throughout the pilot, the system allows students to communicate with tutors online, and to receive customized feedback. In addition, regular automated tests within the CogBooks system provide the students with an opportunity to master the subject content, gain confidence, and receive automated feedback and direction toward recommended underpinning content and reading material where required.

CogBooks screen shot

The two courses were delivered on time, following approximately 5 months of content development, publishing and testing. The supplier proved very helpful and collegial, particularly through a tough period where, for a couple of days, technical problems made the platform unreliable for the Geophysics course users.

We chose to assess the pilot through an online survey (21% response rate), a focus group session, and some face-to-face interviews with students.

The result gave us some pretty clear messages.

Context for teaching, including the tutor and discipline, can have a strong bearing on student perceptions and satisfaction, regardless of the delivery platform.
Preparation of content and the quality of quiz feedback in particular is critical for students who are learning independently online.
The online delivery tool must be available without interruption. Students (who are often paying) will accept nothing less.
Students value the structured content presentation, the ability to repeat and review material, the self-paced learning that was enabled, and the options of preparation for lectures and dialogue within lectures.
Students were clearly disinterested in suggesting content or uploading content themselves, but were enthused (70% positive) with the idea of being able to share reflective notes amongst their peers.
In the GTRC course over 80% of respondents felt that the adaptive tool helped with their deeper understanding of the subject, yet this fell to only 40% within the GP flipped model.
Overall, despite some early technical glitches, 73% of respondents wanted to see Adaptive Learning being used by all of their courses. This figure rose to 93% for the GTRC course.

It became apparent from the student feedback, that a flipped classroom teaching model is one that requires considerable preparation, and acclimatization for the students. An abrupt change from traditional lectures to flipped teaching can be difficult for students to adjust to. Furthermore, some students are more pre-disposed than others to such independent learning, with some (perhaps earlier year learners) requiring the assurance of knowing through the lecture, that they have heard and understood the nuances of emphasis and importance of specific elements of the subject materials, particularly when considering their later assessments.

As a result we have been discussing the merits of partial or ‘blended flipping’, where lectures continue to deliver a more traditional core set of content, but with time to address difficult topics and discussion. In this approach students could chose to look at materials either before or after lectures, or both.

For the University of Edinburgh, the potential of adaptive learning is seductive. It has the possibility of improving student satisfaction when delivered well, and the probability of improving student outcomes. Early indications from May 2015 exam results are positive, but require more analysis over a longer period.

The potential for an adaptive system to augment human feedback with machine-coded feedback is also one that requires further exploration. The enhanced tutor presence through online dialogue, might lead us to expect an increase in student satisfaction, yet other factors have clouded this finding within our results.

Areas of our delivery seem ripe for review second time around, particularly the preparation of students’ expectations, the completeness of quiz feedback, the ability to share students’ questions and reflective comments, and the insistence on reliable and usable platform delivery.

Our findings indicate that the tutors themselves have also to learn and adapt to this new teaching mode. Service suppliers may also have to adapt to the requirement to have responsive and robust processes to deliver the 24 x 7 service that students expect.

Hence we feel we have only started on the route to understanding the value and refinements of delivering adaptive learning to improve our student outcomes and ultimate satisfaction. For this reason we are extending our GeoSciences pilot to deliver to four courses during the 2015/16 academic year.

One of the tutors commented on the experience of the pilot encapsulated its value:

“Yes Max is a bright guy and very engaged. He writes questions faster than I can answer. I have had a few of them come and tell me today how much they enjoyed CogBooks and the greater in-class engagement. I am enjoying the classes more too. Definitely the way forward.” (Prof. Wyn Williams)

As for myself, the experience of delivering the reflective paper to an international audience was one that I relished. Barcelona was fantastic, and the conference itself was very well organized. Keeping researchers to their 15 minute speaking allocation however was one that was rather trickier!

For those interested in the full results and paper, these are published via Academia.edu
https://www.academia.edu/13912408/Adaptive_Learning_Pilot_-_Reflections_on_Student_Experience

For those interested in CogBooks, see further information on their company website:
http://pub.cogbooks.com/~cogbooks/aboutus/