There are a wide-range of assessment options built into or integrated with Learn. Each assessment has its own workflow and a plethora of settings to choose from.

In this blog post we look to highlight some of the common issues, quirks, and key settings to be aware of.

This is part of a series of blog posts related to Tests and Assignments:

• Understanding Test Options
• Learn Assessment “Gotchas”

Learn Assignments

Due Dates

Assignments should have a date/time when submissions should be made by. This date will send notifications to students as the due-date approaches.

We recommend a due-date in working hours so that support is available from ITO or ILTS for any issues that might arise around the submission. So a deadline that is on a Friday at 5pm is not ideal.

Upload Progress

There is no progress bar displayed to students to indicate that the file submission is being processed. The browser will also not show a “spinny-wheel” in the tab to indicate that the page is processing. Students should be instructed to only click the submit button once and be patient.

When the file is submitted the student will be able to see their submission and will receive an email confirmation.

Submission Receipts

When a Learn Assignment is successful the student will receive a submission receipt via email. The students can also see a list of their receipts from their Grades panel which they can access from the top-right of any page.

The general rule is worth following: “if a student does not have a copy of their submission receipt, the submission has not been made.”

Anonymity

Marking should be done anonymously in the majority of assessment scenarios. In Learn Assignment the anonymity settings are very strict – it is not possible to see which student have submitted, and it is not possible to review any attempts related to a specific student until anonymity has been removed.

We recommend setting the anonymity-release date in the future and then changing this when you are ready for anonymity to be removed.

Gradebook

Visible to students

Any activity within Learn that can have a mark associated with it (formative or summative) will have an entry in the Gradebook, which is located in one of the tabs across the top of the course Learn page. The default option for the column is to have the grade visible to students.

By opening up the entry for an assessment, you can choose whether to hide or show it to students.

Calculated Items

The calculated and weighted items are a great way of aggregating grades across activities. However sometimes a calculated item can sometime release a grade from a hidden entry which it is related to.

We recommend taking time to review and manage your Gradebook. You can find out more about navigating the Gradebook here.

You can get in touch with lt-support@inf.ed.ac.uk if you need further support.

Hybrid Teaching put additional emphasis on the content and communication that comes from your Learn course. We have already looked at the way the Informatics courses follow a standard structure to give some consistency to students to help them know where to look for content… but what additional signposting could we consider?

As course materials will sit on Drupal now and course admin on Learn, it is especially important that students know where to go to find what they need. Find out more about the communication tools available on our blog post: Communicating with students

Some Assumptions

Below are a series of contexts and assumptions that we might expect your students to follow during the semester. We have tried to outline some of the things that a student might expect from the course and have offered some tips on ensuring this information is clearly signposted to them.

Student Behaviour

Below are some loose assumptions about the primary audience for your course, the students:

Some students will not read everything

• Use headings, summaries, and bullet points to present important information in a concise manner
• Don’t bury important information in long prose
• If there is a “call to action” for the student make this clear and include a description of what is expected of them

Some students don’t know what they are looking for

• Provide guidance and pointers to help a student get started
• Announcements are a good way of providing a timely narrative to support your course content – these are generally displayed chronologically so can provide an overview as the course evolves through a semester

Students are not yet subject matter experts

• Avoid using language or concepts in the titles that the student may not yet be familiar with
• Provide overview of what the student is expected to learn in each week / topic unit
• Reference materials that might be pre-requisite or used for revision purposes

Students are juggling multiple courses with varying schedule and deadlines

• Make sure the course schedule is clear
• What synchronous events might be taking place

Students are likely having to plan their own self-study workload

• Provide an overview of how much time a student is expected to spend in a week
• Give insight into which materials are essential reading/viewing, and which might be additional information only, or just for fun/consolidation
• Some students will want to make sure they read everything that is posted to the course, so if you include information that is a reference only you might need to indicate that this is not core-reading

Students might not be able to follow the standard course structure/timeline – this could change at any point during the semester

• Is it clear where recordings or alternative materials are located?
• Is there a summary of what a student is required to do each week?

Students might not realise that an element in Learn needs to be clicked for more information

• Try only using a short description/summary in clickable elements in Learn as these can often be overlooked by students
• Provide the full information from within the folder/section
• Use a different colour for the link

Why/When are students accessing the course?

Based on previous work students tend to access courses for the following reasons:

Start of term – just browsing all content

• Check the course readiness article (2022 update) for what is expected for Week 0
• If content is not visible and is due to be released, give an indication to students of when materials will be available and where to find them.
• Scheduled items in Drupal and hidden items in Learn do not show anything at all, not even a placeholder. If all content is hidden a student will see an empty page.

Week to week, looking for specific materials to support their studies for a specific week

• Be sure to identify the week number in titles and links, you can include the date too if relevant
• Try to also include a title or summary of the topic for that week, this will make it easier to find materials when reviewing.
• Announcements are a great way of setting the scene for the week’s teaching.
• In Drupal, using either the schedule table or having a page for each week of the course makes it easy for students to navigate to what they need

Take part in set learning activities

• If there are set actions, dates, and deadlines required for a student to participate be sure that the instructions and link to activity is displayed clearly and consistently
• It is useful to add an announcement at the start of each week identifying what is expected of a student. Please see this example of an announcement, with the option ticked to also send it as an email, from Week 2 of AML (with thanks to Oisin for letting us share this): Applied Machine Learning (2022-2023)[SEM1]- AML Week 2 Instructions.pdf

Revision of topics before some form of assessment

• If materials are organised by week is it still clear which topics and areas where covered in those sections
• Give content meaningful titles and summaries so resources are not just referenced by Week 1, Lecture 1 – this shows when the lecture took place but not what it covered.
• You can add links to your content in Drupal. This allows you to link to content in a different section without having to duplicate content.

Looking back at courses studied in previous years

• Students have access to their courses studied in previous years and may wish to review content so meaningful titles can aid them in finding the materials they are looking for

Following up on an announcement / notification in Learn

• Many students will use the email or Learn app notifications to trigger them to accessing the course
• Announcements and notifications are pooled into separate lists on the app to make it easy for a student to see all activity across their Learn courses from a single feed without having to navigate each course individually looking for updates.

Signposting and Structure

The above information is looking at how information could be presented to students to make sure it is easy to find and presented consistently. We have touched a little on what content you might want to include in your course setup – but we have listed some outlines of content that a student might be expecting to find in your course.

Take a look at the list below and think about whether this information is available within your course, and if it isn’t, how would you expect the student to find it?

Expected Content

Below is a summary of the type of information a student is likely to expect within your Learn course:

• Course Information
  • What is the course all about?
  • What is expected of me in this course?
  • Is there a timetable/schedule of teaching and deadlines?
  • Who is running the course, who can I contact if I need help?
• Course Content
  • What information do I need to prepare for teaching activities?
  • What is happening this week?
  • What should I have learned about this topic?
  • Where can I find the materials used during the teaching?
  • Is there a live teaching session or is it only via recording?
  • How much time should I set aside for this?
  • Is there additional reading or revision materials I should be looking at?
  • Does the teaching refer back to any of my previous studies?
• Assessment
  • What am I being assessed on?
  • When do I need to submit my assessments / when are assessments taking place?
  • How will I be assessed?
  • How do I submit my work?
  • When will I get my feedback?

Test it out

Sometimes as a course organiser you can be a bit too close to the material to view the materials as a student would. You can use Student Preview in Learn or paste a Drupal URL into a private web browsing window to see what a student can see in your course, but you will need to try and think about the questions above to see if you can find the information that a student would expect to see.

Alternatively why not get a friend or colleague to try the course out as a student would to find  specific information. Sometimes a person with no background knowledge of your subject area can be a great person to test the structure of your course; ideally the course should be usable without prior knowledge of the subject.

Your course will likely be made up of a number of patterns that will be replicated across the duration of your course. For example the majority of courses will likely break down their course materials by Week, creating folders within the Course Materials area in Learn for Week 1, Week 2, etc.

You can read the University’s detailed overview of Macro and Micro Patterns here.

Macro Patterns

We have already touched on courses being broken down by Weeks. Each week might consistently have 2 lectures, 1 tutorial, and every other week have a group assignment. Your course should reflect this within the course materials and provide the outline to this from within the structure.

Micro Patterns

If using a flipped classroom approach, the way that you choose to deliver your lectures will likely also follow some “micro-pattern”. In your lectures you might consistently structure them as:

If you can think about how your weekly content might be broken down in this way and represented in your course website, you can quickly repeat these structures for the materials giving your students a consistent and manageable experience.

If you are unsure how to identify these patterns and present your content and activities to students then please contact LT-support@inf.ed.ac.uk or attend one of the Consultancy drop-in sessions for further discussion.

This is a reminder of the tools and services available to you in the event of disruption to campus based activities, such as the current COVID-19 epidemic.

If your *students* can’t access campus

• If students can’t come to lectures, they can access the lecture recordings via the Lecture Recordings link in Learn. Please note: this is only for those lectures delivered in a centrally supported room.
• Any room which supports lecture recording, also supports Live Streaming. Please get in touch if you would like to enable live streaming of your lectures.
• For those courses requiring to use submit, students can download and install Virtual DICE or remote access to normal DICE machines via XRDP or SSH. Please log a call with computing help for further information.
• For those courses which don’t require to use submit, remember that Learn has an assignment tool which will more than likely meet your needs. The Informatics Learning Technology Service can help with this – please get in touch.

If *you* can’t access campus

• The same product used for lecture recording at scale across campus (Echo360) has an application users can download from the website. Please note: the application is only available for Mac and PC. If you require a loan device, please get in touch.
• The Echo360 application – called Universal Capture – allows you to capture audio, screen + video. You can then publish direct to your course via the recording interface. This means students will access your recording in the same place as recordings of campus based lectures. See the bottom of the page for links to video and written guidance.
• The Echo360 player (the interface students use to watch lecture recordings) also has a nice feature where they can ask questions at specific points in the presentation. The lecturer can then review these and answer questions in the appropriate context. See Media Hopper Replay: Q&A discussions, flagging confusing content, and bookmarking for further guidance.
• You may want to deliver smaller, tutorial sized classes via Blackboard Collaborate. Collaborate sessions can be scheduled via MyEd or Learn. All sessions run in the browser (Chrome is recommended) and so there’s no need to worry about user devices.

Further Help

In addition to local help via the Informatics Learning Technology service, Blackboard are running sessions on Tuesday 10 March called “Preparing to scale online teaching and learning during Coronavirus”. This webinar is for anyone involved in administering or delivering teaching and learning, including but not limited to system administrators, eLearning technologists, IT managers, Heads of Teaching and Learning, faculty and academic staff. Register here: http://bit.ly/COVID-19EURUG

Media Hopper Replay’s universal capture tool – video instructions

Media Hopper Replay’s Universal Capture tool – Mac

http://www.docs.is.ed.ac.uk/skills/documents/Lecture%20Recording/Guides/3873_v2.pdf

Media Hopper Replay’s Universal Capture tool – Windows

http://www.docs.is.ed.ac.uk/skills/documents/Lecture%20Recording/Guides/3872_v2.pdf

Media Hopper Replay: Q&A Discussions, flagging confusing content and bookmarking

http://www.docs.is.ed.ac.uk/skills/documents/Lecture%20Recording/Guides/3887.pdf

An Instructor’s guide to Media Hopper Replay: Viewing course and student analytics

https://media.ed.ac.uk/media/An+Instructor%27s+guide+to+Media+Hopper+ReplayA+Viewing+course+and+student+analytics/1_rs96etgi

Please read through the following guidance well in advance of any submission deadline you have.

Check file type

The Learn assignment tool will accept any file types but check the instructions for your course to see which file types are required for your individual assignment.

Check file size

If you are submitting very large files, this will affect the time it takes to upload…

Internet Connection

… We therefore strongly advise that you submit your assignment using a reliable and fast internet connection.

Check Browser

Wherever possible, use a computer and browser you are familiar with, or a computer in the public access labs, when submitting an assignment. You can check your browser compatibility from the Browser information box on the My Institution page inside Learn.

Submission Deadlines

Do not submit the assignment minutes before the deadline, because then you will have very limited time to change computers or report a problem if there is one.

Troubleshooting

If you do have a problem submitting your assignment try these troubleshooting steps:

• If it will not upload, try logging out of Learn / MyEd completely and closing your
  browser. If possible try using a different browser.
• If you do not receive the expected confirmation of submission, try submitting again.
• If you cannot resubmit, contact your tutor by email attaching your assignment, and if
  possible a screenshot of any error message which you may have. (see below).
• If you have a technical problem, contact the IS helpline (is.helpline@ed.ac.uk). Note the course name, type of computer, browser and connection you are using, and where possible take a screenshot of any error message you have.

Always allow yourself time to contact helpline / your tutors if you have a problem submitting your assignment.

Further information can be found on the IS support pages here: https://www.ed.ac.uk/information-services/learning-technology/virtual-environments/learn/assignments-marks-feedback/assignments

On 1 March, the IS User Experience (UX) Service, in partnership with the School of Informatics, ran a Learn usability testing showcase event. Participants from across the University watched screencasts of students using an Informatics Learn course, before prioritising the usability issues identified.

Five students in total took part in the testing – four from Informatics, including those enrolled on single programmes, joint programmes with other Schools, and those from our Undergraduate Apprenticeship Scheme – and one from the School of Economics. Each was presented with the following scenario and four ‘typical’ tasks to perform.

A copy of the Learn course for Computer Security was used for testing purposes. This was chosen as it aligned closely with the Learn template developed for the School.

Scenario

You’re a third year student on the joint programme BSc Artificial Intelligence and Computer Science. This semester you are studying a course called Computer Security. It’s week 3 of the course, and you’re preparing for your first piece of coursework.

Tasks

Task 1: You want to check the deadline for the first piece of coursework and see if it clashes with any other coursework deadlines. Using the Learn course site, find out the deadline for the first piece of coursework, and then see if it clashes with coursework deadlines for any other courses on which you are enrolled.

Task 2: You missed the third lecture of week 1 because of sickness. You’d like to watch the recording so you can catch-up. Using the Learn course site, find and play the third lecture of week 1.

Task 3: You’re going away for the weekend and you’d like to do some reading while you’re away. You’re not sure you’ll have access to the internet, so you’d like offline access to your reading. Using the Learn course site, find the required textbook for the course and see if you can download or print a section of the textbook.

Task 4: You’d like to familiarise yourself with the content of the last lecture you attended, called Cryptography – asymmetric encryption. Can you open the lecture notes from this lecture?

Results

Task 1: Most users found the coursework deadline relatively quickly and with ease. One student checked both the Course Information and Course Content pages prior to selecting the Coursework and feedback page.

However, no users were able to easily find the link to the personalised coursework planner. This was expected, and one of the reasons why I included it in the task.

Task 2: Most users found the link to the lecture recording overview page with relative ease. Some users were expecting to find a direct link in the table on the Course content page. This was not surprising as the Semester 1 course Informatics 1: Introduction to Computation includes this.

There was, however, a significant usability issue identified for all users when it came to identifying a particular recording from the Media Hopper Replay course overview page. This was caused by the unhelpful automatic naming convention of recordings (see below). Users performed a lot of cross-checking between different pages on Learn, various online calendars and and the Media Hopper Replay course overview page to identify the recording from the “third lecture of week 1”.

Task 3: The course organiser had used Leganto, the centrally supported Resource List tool, for the course.  Users could access the text on Leganto via both an in-text link, or an icon associated with the service link. Most Informatics users found the link to the required textbook with relative ease. There was one instance of users navigating to the table on the Course content page where references to specific chapters are included.

This particular textbook was behind an EASE login. As the students were using a dummy account, they were prompted to enter their EASE credentials which would not be the case when logged in as themselves.

Interestingly, the student from Economics searched for the textbook by navigating to the Handbook. This highlights the different approach to content curation and the various roles course and programme handbooks perform across the University.

Task 4: Four users found the link to the lecture notes with ease. One (visiting) student initially checked the Coursework and feedback page. It was noted that the terms lecture notes and lecture slides are sometimes used interchangeably.

Action Points

• Feature request for Media Hopper Replay team: can we automate naming of recordings by date? Venue information would also be helpful here.
• Request for Media Hopper Replay team: can we facilitate production of individual URLs for each recording which will work for enrolled users – *even when they haven’t selected initial LTI link*. Only when this can be a achieved, should we encourage course instructors to include links to Media Hopper Replay recordings in the table on the Course Content page.
• Promote coursework planner across the school. Posters / monitor displays etc.
• Include a thumbnail of a ‘typical’ coursework planner in the next iteration of the template.
• Can the coursework planner display full course name rather than acronym?
• Enquire into possibility of responsive design for coursework planner.
• Can we have the coursework planner work for tutors (eg marking loads)? Do we need this?
• Work with the web and communication team to research how Informatics students use the yearly handbook.

Reflection

I thoroughly enjoyed working with Duncan Stephen on this mini project. The feedback was informative, encouraging, and a call to action. I’m looking forward to embedding similar practice across the School for alternative platforms for content delivery.

The results of the ‘prioritisation of issues’ aspect of the workshop can be found below. If you would like to know more about this particular round of testing, or would like to use your course for further testing, please don’t hesitate to get in touch.

Further links

• User Experience Service: http://www.ed.ac.uk/is/ux
• Join the UX community: http://bit.ly/UX-meetup-blogs
• UX mailing list: http://bit.ly/uoe-ux-mail
• Steve Krug’s Rocket Surgery resources: http://bit.ly/1I1muXo
• David Travis’s prioritisation flowchart: http://bit.ly/1I1mCWW

Background

Edinburgh College of Art (ECA) frequently assesses to multiple learning outcomes (LOs). This means that for each assignment, a student will receive multiple grades and sections of feedback. In the past, ECA used a custom VLE (the Portal) to support this kind of assessment. Tutors would select grades for each learning outcome which was being assessed (often three) and feedback for each learning outcome. Students would then need to leave a piece of self-evaluation and ‘submit’ in order to access their tutor’s feedback.  As we migrated from this legacy VLE (the Portal) to the centrally supported VLE (Blackboard Learn) I went about trying to identify how best we could leverage the tools of the new system for a workflow which was familiar, and favoured, by teaching staff and students at ECA.

The challenge was as follows:

– Allow markers to leave multiple grades and feedback for each assignment

– Allow support staff to download these grades for upload to the central assessment and progression software with minimum intervention (and risk of mistakes)

– Allow students to receive their multiple grades and feedback with as few clicks as possible, and represented in a clear and consistent way.

I identified the rubric tool within Blackboard as the most useful tool we could use to replicate this workflow. While this addressed point 1, it did not address points 2 and 3:

– as it stood, administrators could only download the aggregated grade from the Blackboard Learn Grade Centre.

– students would have several clicks (many of which would not be intuitive) in order to access their feedback. Because of the complexity of this task, and the multitude of ways a student could give up half way through, communicating grades and feedback (essential to the learning process) would be severely compromised.

I proposed that ECA use some if its Information Services (IS) Apps development budget for investigating ways of addressing these issues. I met with a project manager from IS and we blocked out the best part of a day for exploring what the current challenges were, what the risks were for continuing with the current system, and the opportunities for developing something which could improve the experience.

I then took these initial findings to the development team in IS at a subsequent meeting. After ruling out some options, we proposed building a new building block for Blackboard which would provide a different view into the Grade Centre. This would have to be accessed via a new tool, rather than a different stylesheet for the existing tool (My Grades). This would introduce a potential cause for confusion, but I balanced this against the potential confusion from the existing workflow and argued the benefits would outweigh the challenges.

The project then moved into the development phase. I was tasked with testing the Beta tool. The new tool went live for ECA staff and students in September 2016. After a year of successful deployment, the tool was made available to the rest of the University in September 2017.

Project documentation.