The topic of the Teaching Lunch held on 8th of April focused on CodeGrade and was led by Volker Seeker and Devin Hillenius from CodeGrade. CodeGrade provides automatic and immediate feedback, integrated rubrics, efficient manual grading and a plagiarism checker designed for code. CodeGrade was used for the INF1B course and the Teaching Lunch shared Volker’s experience of what he did and how he used CodeGrade to achieve this.
Within the INF1B course, there are around 420 students and two java programming assignments graded by CodeGrade. Volker demonstrated the student view by showing Assignment 1. Clicking on the Assignment 1 submissions box will link you to the CodeGrade interface that will submit the students’ code and run auto grader tests. This is done through Learn.
For the students, clicking on submission opens the CodeGrade interface where they can upload files, view their latest submission, and see which tests they have passed and how well their code works. Steps must be passed to move onto next test based on student’s code. The students can upload, test code, then override their submission up until the deadline so that they can see the outcome to all apart from the final test – this can be edited by the course organiser.
Devin Hillenius commented to say that the rubric can be set up with a combination of manual and automatic marking, not either or, and that you can have a mix, parts automatic, parts manual.
Volker then presented the view of CodeGrade through Learn from the staff point of view. By looking at Assignment 3 and clicking on submission, teaching staff can view what students have submitted so far. Volker presented the steps that the students’ code must pass and how they can work through the steps and see the errors apart from the advanced tests run when marking takes place.
Volker also showed how Assignments can be set up with rubric, hand in instructions, how to set a deadline, that work can be handed in in different ways, and that staff can divide submissions between markers. Volker demonstrated the plagiarism checker, showing how one can compare code to see similarities.
The following questions were raised:
- Dividing up assignments between markers: can you divide different parts of the assignment to grade? For example, can markers be given part A of a students’ assignment to mark rather than the whole assignment? Devin confirmed that this is not currently possible – it is the whole submissions you are dividing.
- How much effort to set things up? Volker confirmed that he had one workshop with Devin that ran for around an hour, plus 2 days playing around with system. He then figured out how he wanted to do and took about a day to put it all in and set it up. Paul asked a follow up question concerning whether you would need to do this yearly. Devin confirms that you can copy over/import from previous assignment into a new year, so it is straightforward to make more tests/change tests.
- How do you check checkpoints? Can you set percentages, how much of previous tests you have to pass to move onto the next step? Volker confirms that if there is an error, you fail the check point.
- Does it scale? Could it be used for e.g. Compiling Techniques? Are there non-functional metrics, e.g. performance or code style? Devin confirms that CodeGrade works with any number of students/assignments. It can execute whatever you want with the tests, it can be more complex or less complex. Additionally, you can write your own tests to measure performance and have markers look over submissions as well as have rubric about readability and structure to give a number out of 5 regarding how clear it is.
- Can you run tests within Jupyter Notebook? Devin will look into whether this is possible in CodeGrade.
Volker then shared how to give feedback through CodeGrade. He demonstrated that you can manually comment on the code with inline comments to provide students with feedback as well as providing general feedback. Students can see both. Devin then shared that, due to the current situation and to aid in distance learning, CodeGrade have added new feature to be realised next week so students can reply to inline comment entries and have conversations about feedback and ask questions on feedback. This can be turned on and off. It allows for personal communication during distance learning.
Devin from CodeGrade welcomes staff to get in contact with further questions, Informatics Learning Technologists can also help within the school:
