Effective Digital Content training – listening to our users
We’ve spent the last year listening to our community of practice to help us to make much needed changes to the Effective Digital Content training.
The Effective Digital Content training course is a series of videos and quizzes that sits on the virtual learning environment Learn. It is a mandatory course for any staff who wish to become editors on the University’s central Web Publishing Platform or who wish to enhance the content design skills in their team.
Since the course was last updated, it has become slightly out of date:
- The sources cited in the course have since updated their research
- The structure and capacity of the UX team has also changed, so references to training on offer is also now out of date.
- With the move to the new web publishing platform, references to EdWeb are out of date.
We want to redesign this course so that users find it easier to learn about best content practice at any time, making sure the course is accessible and inclusive in line with our Digital Strategy and Strategy2030.
EdWeb form analysis
One of the ways we gathered feedback was by analysing data from the EdWeb form from the time the course was set up. Data from this form runs from 2017-2022. About 439 people had completed the feedback form at the end of the course by the time the analysis was completed by October 2022.
How users watched videos
When users were asked how they watched the video:
- 41 said they watched the videos without subtitles
- 196 said they watched the videos with subtitles
- 43 said they just read the transcripts
How users rated the quality of the videos
Users were asked to rate the quality of the videos.
When asked about the information covered:
- 268 said it was very good
- 144 said it was good
- 15 said it was OK
- 1 said it was very poor
How users rated presentation
When asked about the presentation, including presenter:
- 236 said it was very good
- 135 said it was good
- 36 said it was OK
- 6 said it was poor
- 1 said it was very poor
The comments
When asked to expand on their ratings, 214 users commented. 84 of the comments were positive. Of the positive comments, these were mainly about the quality of the course content.
Other comments explained more about aspects of the course they had found difficult or unsatisfying.
Overall these issues fell into the following rough categories:
- Quiz – 42
- Accessing and navigating Lynda – 38
- Audio – 21
- Course content – 20
- Subtitles – 19
- Visual aspects – 14
- Transcripts – 12
- Accessibility and inclusion – 11
- Confusion about next steps at the end of the course – 5
- Resources and good practice to take away after the course – 5
- GDPR/data protection – 4
- Requests for a progress bar – 3
- Engagement/focus on course – 3
- Jargon – 2
Since the training was last undated in 2019, some of these issues were resolved. As such, the following comments were made up to or around 2019, and were discounted from the final analysis:
- Accessing and navigating Lynda – 38
- Subtitles – 19
- Transcripts – 12
In the final analysis, the quiz has been the most difficult element, with audio quality, course content, and visual aspects the next largest categories.
The issues
Quiz
42 people found that the difficulty level was problematic, for example that it is unreasonable to expect 100% on the final test. Some experienced technical difficulties with the quiz not saving results. A lot of time was wasted repeating the quiz to achieve the required marks, and it was then unclear whether the user had ‘graduated’ and if they would be granted access to EdWeb. There were issues with the way questions were worded from an inclusion and usability perspective. Users also asked for clearer feedback on incorrect questions. Related to accessibility comments, we need to think about how to be kinder to people with dyslexia and dyscalculia in the way we write quiz content.
Accessing and navigating Lynda
32 older complaints surfaced from the early days of the course. Navigating between Lynda and the course materials on Learn was understandably a difficult experience for many. Users had to pause videos in Lynda at the right time, then return to the Learn course. Often people forgot to pause and irrelevant material played on. This was very frustrating and time consuming. Accessing and logging in to Lynda and Learn also caused problems. As mentioned earlier, this issue was resolved in one of the iterations up to 2019.
Audio
21 said the audio provided an inconsistent, sometimes disabling experience, which caused some users to rely more on the subtitles and transcripts to understand the content.
Course content
20 users questioned some of the course content itself, for example the advice given on PDFs. Some users also commented on content that was not inclusive, for example uncritical use of the phrase ‘dumbing down.’
Subtitles
19 commented that the subtitles were sometimes out of synch with the audio, or a user had to divide their attention between reading the subtitles and looking at the visuals, missing content. Some commented that they thought the subtitles were automatically generated.
Visual
14 users found the visuals added little to the experience or were not of sufficient quality to enhance the content.
Transcript
12 users commented that a transcript was incorrectly linked to the wrong video.
Accessibility and inclusion
11 found the course was not accessible, for example dyslexia, ADHD, visual impairment, and harmful language to the LGBT and disabled community. There’s an interesting comment about how we might include BSL in the course.
I’m finished, and confused
5 users experienced confusion following the quiz around whether they had passed, and if they needed to do anything in order to progress to EdWeb editing.
Resources and good practice
5 said they would like to see improvements to the resources that can be taken away from the course, including visuals and examples of good practice.
GDPR/data protection
4 few questioned how we ask for consent and the level of personal detail we ask for.
Requests for a progress bar
3 commented that they had trouble navigating their progress in the course – this small group asked for a progress bar.
Engagement/focus on course
3 comments including one saying ‘I did not want to come back’ – these comments about how engaging the course content is shows we can pay more attention to pacing, tone, supporting content such as visuals, and chunking.
Jargon
2 comments is a quite small data set but this is a quite important contradiction to our own advice on our use of jargon, with people not understanding what UX and SEO abbreviations mean.
Survey
Our student employees in 2022 were instrumental in designing a survey to help us delve deeper into our users’ needs. Analysis of the EdWeb form had helped us to see a broad picture of the bigger issues. We were curious to see what our communities of practice needed from the course when they had finished, and how we could make the information in the course more visible.
Useful takeaways
Although the group of 13 respondents was small, some key elements surfaced, such as repeated mentions of accessibility in expectations and parts of the course that had remained in memory and helped their everyday work.
When asked the question of what might improve the course, respondents asked for more in-person training and materials that could be referred to when the course was finished. They also asked for consideration to be given to the way staff work post-pandemic for course delivery.
The survey, although a small data set, is a useful addition to help us address aspects of the course such as plain language, platform accessibility, chunking, and how we help users to plan the course around their work. We can also do some thinking around post-course materials and how we use our channels more effective for new and emerging discussions around content design practice.
Like the form feedback, this small group helped confirm that the course content is solid in essentials, with some expressing pleasant surprise at the in-depth nature of the content, and more experienced content people finding what they would expect.
Interviews
4 respondents from the survey agreed to chat with us further, giving us the opportunity to ask questions about what kind of challenges they faced in their everyday work, what references they used, how we could make the course more accessible, and what materials, channels or formats they might find useful when the course was complete.
We found the following common themes:
Website management was a small part of their job
All except 1 participant had website management as a small part of their job. This tended to load the small amount of editorial work on often inherited sites with a lack of confidence or ownership.
Alt text and plain language were dominant takeaways from the training
All the participants we spoke to remembered accessible content practices and applied it to their work including an understanding of the F-shaped reading pattern, chunking, alt text, and plain language. Their work sometimes included providing rationale to colleagues for accessible practices.
Participants relied on what they knew to reference guidance
All users bookmarked guidance, ranging from the wiki to the Editorial Style Guide, asked the colleague next to them, or simply emailed website.support@ed.ac.uk if they needed to know the answer to a question. Everyone used Google to look stuff up, as you do.
There was an undermined sense of trust in the findability of reliable guidance after the training
Participants said that the University training was too spread out in too many locations, and that what guidance there was tended to be undermined by poor search engine and broken links. Those who used the Editorial Style Guide said it was fine if you knew about it.
Participants wanted different formats
When asked about what kind of formats might help, one participant said they would like in-person training, more than one expressed an appreciation for short, well chunked online training that would fit around their job, and one asked for a version of the Editorial Style Guide that was possible to annotate for personal use, such as a simple Word document.
Participants were keen to explore different channels to communicate about guidance
Participants liked the idea of a weekly hint or tip, or a link to an external resource or emerging discussion. A couple mentioned a place to keep materials when the training was complete, such as a Sharepoint site, or a simple joining pack document with resources. If using a Teams channel it was emphasised by one participant that a sustained effort would be necessary to commit to engaging our community of practice – essentially, if we’re going to develop our channels, someone has to do the work.
Recommendations
After all that work listening to our users, we have arrived at some thoughts as to how to proceed if we’re going to develop a course that is findable, high quality, can be referenced afterwards, and connects with our community of practice.
Revise the online course
The first recommendation, and this is the most obvious since we began this analysis with this aim, is to revise the content of the online course:
- Structure, chunking, quiz
- References
- WCAG 2.1 accessibility guidance
- Language
Run in-person training again
We run a compact version of the Effective Digital Training course when the University Digital Skills Festival comes around, and it is always one of the most popular talks. Talking to our users has prompted some thinking on how often we should run our original in-person training. As one person put it:
The course that I did, it happened to be delivered by both yourself and Lizzie from home. I think it must have been very deep lockdown times but you still managed to work together to deliver it. You had some small people interrupting now and again, but that was very engaging for me, the way that you were able to deliver it online. And in a kinda personable way that wasn’t just kind of like reading through learn articles. I learn much better that way when I’m being instructed.
Create readability guidelines for the Editorial Style Guide
The Editorial Style Guide is a natural home for some of the guidelines that are buried in the Effective Digital Content course in Learn. Creating readability guidelines brings that good stuff up where everyone can search for it, and see it, in a place that is reliably curated by the Editorial Style Guide working group.
Use existing Teams channels to talk about good content practice
Sort of a no-brainer this one really; we already use Teams channels to chat about all sorts of stuff, so why not use one of our existing channels to communicate a weekly ‘nudge’ or thoughtful link to an emerging discussion? One candidate could be the existing Human-centered Network.
Make the online course easier to find
Our problems with visibility are manifold; we have a big estate with some far-flung training and resources that are great, if you know about them. One participant suggested being able to find all training in the MyEd megamenu; that’s a good thought. We should also be asking ourselves if Learn is still the best place for this training.
Come and talk to us
If you’ve been interested in any of the topics raised and have some things to say about your experience with the Effective Digital Training course, or you have some ideas for how you could support its development, we’d love to talk to you.
Email website.support@ed.ac.uk or get in touch with Emma Horrell directly by emailing Emma.Horrell@ed.ac.uk
EDC further reading
Low literacy users
Lower-literacy users: Writing for a Broad Consumer Audience – NNGroup
Low literacy users – Readability guidelines
How people read
F shaped reading original study – NNGroup
F shaped reading misunderstood but still relevant – NNGroup
Images
Images – Readability guidelines
Links
Links – Readability guidelines
Writing hyperlinks: salient, descriptive, start with keyword – NNGroup
Information architecture
What are you going to remove today? – Gerry McGovern
Checklist
