Assessment literacy: what it does mean in practice

In November 2016, I was awarded some funding as part of the IS Innovation Fund 2016-2017 in order to launch a website on assessment literacy (or assessment literacies, as mentioned by some authors to emphasize the wide range of skills encompassed by the term).

This award is important for me for two reasons: first, because it will contribute to build up my project management skills and, second, because I would like to produce information around assessment literacy which is relevant not only to tutors, but to students and policy makers too.

Assessment literacy is a relatively new coined term which has become fashionable in the last years. I would defined assessment literacy as the ability of constructing reliable assessment by using the appropriate means and then marking, managing and recording these assessments to facilitate valid instructional decisions. Assessment literacy is intrinsically related to feedback and I would say that the knowledge, or ability to provide a feedback which is relevant, timely and detailed to students could be seen as an assessment literacy skill.

It is a common misconception that assessment literacy is only relevant to tutors. However, the concept and the set of skills that the concept comprises are also relevant to students because students should play an active role in assessment and feedback .

There is a third role which has often been ignored as far as assessment literacy is concerned. I call this role “policy maker” to define the staff members who can bring innovation to assessment policy and regulations, as well as to courses and programmes. I thought that “policy maker” was the right term because the remit of those experts is certainly “to make better policies” assessment-wise. However, after a discussion with some academic colleagues regarding what the term means in this context, I realized that “policy makers” may be a misleading term as it suggests staff in job positions with executive power within the University (e.g., Vice Principals, Deans, or Head of School). By “policy maker” I mean all staff members who have the leadership skills and the capacity of bringing innovation and/or improvement to the assessment strategy of programmes (both undergraduate and postgraduate studies). Programme directors and course organisers are “policy makers” too.

Although tutors also have a certain level of freedom and flexibility to introduce improvement in the assessment methods of their courses, this freedom usually applies to formative assessment, as tutors may decide to introduce additional formative assignments or new methods of assessment, such as video or peer assessment. Nevertheless, the final summative assignment in courses is usually fixed by the assessment strategy of the undergraduate or postgraduate programme the course belongs to. Tutors may be able to influence this strategy, but they cannot define it as programme directors do.

As an understanding or ability, assessment literacy has different meanings depending on whether you are a student, tutor or policy maker. Let’s explore briefly what assessment literacy means to each of the roles:

What assessment literacy means for students:

For students, becoming assessment literate is like learning a new language: you learn a new grammar and vocabulary, how they relate to your mother tongue and the context in which the language is used. Students learn the new language of assessment when they have a clear knowledge of how their coursework will be assessed against a certain set of academic or professional standards, when they will receive feedback from tutors and how they can engage in a feedback dialogue so that the feedback can be used to improve the next assignment.

An assessment- literate student develops a good understanding of the following aspects of the assessment strategy of their courses:

The type of assignment that the tutor will use. Assignments can comprise essays, examinations, presentations, research projects, group work, participation in tutorials, etc.
The marking criteria for the assignment and how the marking criteria will address how well the student have understood something or how student’s work fulfills certain academic standards.
Opportunities for formative feedback before the summative feedback. The University of Edinburgh encourages tutors to develop opportunities for formative feedback as part of the assessment strategy of their modules.
When the feedback will be released: For undergraduate courses, tutors should provide feedback (both formative and summative) within the next 15 working days after the assignment, or in time to be of use in subsequent assessments of the course, whichever is sooner. At the start of the academic year, Schools will publish their timetable for returning feedback.
How the feedback will be released: Feedback on coursework is usually written, but can also be podcasted, emailed or streamed as video feedback. Feedback on exams may well take the form of ‘whole-class comments’ which can be presented verbally in a class, online or emailed. This general feedback after the exam should also include the opportunity to get face-to-face individualized comments from your tutor.

What assessment literacy mean for tutors

Tutors develop assessment literacy skills when they acquire a clear understanding of how assessment and feedback can act as performance enhancer and how intrinsically related both processes are. Tutors should be familiar with the assessment strategy of their course/s, and how this strategy can support opportunities for innovation in the use of new technologies, learning platforms and mechanisms of assessing and providing feedback to students.

Tutors are assessment-literate when they are confident addressing the following questions:

What are you assessing?: what kind of skill, competency or knowledge you want to appraise from their students.
Why are you assessing it?: Assessment should be purposeful. It is not about “punishing” or “exposing” your students, but establishing an interactive process in which you and your students will discover together if they meet a set of criteria or standards to complete a certain task.
Why are we assessing it in this way? Technology is now mature enough to support different assignment types and ways of providing feedback. Assessment literacy means that you should be able to pick the type of assignment that best supports the appraisal of a particular skill or knowledge and the technology can support that particular assignment. The Educational Design and Engagement (EDE) team of the University of Edinburgh can help you to choose the technology that can best implement your assignment.

Traditionally assessment has put strong emphasis on memory recall and stamina using time-controlled examination to assess students’ ability, often forcing them to take multiple exams in quick succession. This approach to assessment is particularly inaccessible for disabled students who may have reduced stamina, finding difficult to concentrate for long periods of time or memorise large pieces of information. Assessment should be accessible which means that tutors should check that the types of assignments that they have chosen for their course do not disadvantage any group of students.

It is important that tutors have a clear insight into the guidelines provided in the “Equality Act 2010 Technical Guidance on Further and Higher Education” because assessing a particular skill or competence also involves to be aware of the human diversity in which such skill has been developed.

Tutors should consider methods of assessment which test knowledge, understanding, abilities or skills without testing students’ stamina, memory or ability to stay calm under pressure. Designing accessible assessment methods is part of the assessment literacy skills expected from tutors.

What assessment literacy means for policy makers

Policy makers play a key role in the design of the assessment strategy for undergraduate and postgraduate programmes. Assessment literacy means that policy makers know how to embed the concepts of inclusive learning, internationalization and accessibility in the curriculum design, teaching and assessment strategy of the academic discipline that they lead.

Promoting an assessment design which is inclusive and accessible is an assessment literacy skill. An assessment strategy is inclusive when it acknowledges that your students come from many different backgrounds and have multiple identities; any knowledge, competency or skill should be assessed in a way that protects and respects the diversity of your students.

Assessment design should be compliant with the Equality Act 2010 and the Public Sector Duty. This implies that any assessment policy or specific guidelines should guarantee that students of protected characteristics are assessed in a way which is fair and that does not cause additional harassment, stress or pressure because of the protected characteristics. The Institute of Academic Development (IAD) website offers more information about this.

Finally, policy makers should be familiar with the University assessment regulations and how those regulations can foster good assessment practices:

At the University of Edinburgh, assessment is an integral part of course design and the main purpose of assessment is to foster learning, both for formative and summative assessment.
The assessment process should operate fairly and no individual or group should enjoy privileged status or suffer undue disadvantage in terms of the academic judgements that are made about their performance.
Appropriate moderation measures should be put in place to ensure consistency and reliability of the assessment process.
The purposes, procedures and criteria of the assessment process need to be open, clearly stated and understood by all involved: assessors, teachers and students. All schemes of assessment should sample a substantial proportion of the work covered in a course in order to be representative, fair, valid, reliable and effective.
To promote fairness and in order to record as full a profile of student strengths and weaknesses as possible, intellectual achievement should be measured by a varied and diverse range of methods.
Assessment processes must ensure the security of their operation in terms of the safe recording, transfer, storage and retrieval of information on student achievement.