In today’s post, the authors reflect on their research project in which they assessed which chat systems best suited students for socialising during the COVID-19 pandemic. Muminah Koleoso is a recent graduate with a BSc in Computer Science and Mathematics, and Cristina Adriana Alexandru is a Lecturer in the School of Informatics. This post belongs to the July-August Learning & Teaching Enhancement theme: Learning and Teaching Conference 2023↗️.
Higher Education (HE) students’ experiences are shaped by the quality of both their studies and social interactions. Positive social interactions can improve student well-being and academic performance↗️in Higher Education. The COVID-19 pandemic resulted in the closure of many universities, enforcing students to adapt to socialising solely via Internet Communication Technologies (ICTs) such as chat systems like WhatsApp↗️ and Telegram↗️.
This project aimed to help students decide which chat system is best for facilitating social interaction and recommend design practices for developers to improve said systems. In particular, it sought answers to the following research questions:
RQ1: What are the characteristics of a beneficial online social interaction?
RQ2: In what ways did students make use of chat systems for social interaction during the COVID-19 pandemic ?
RQ3: Which features of chat systems did students use, and what made them more or less beneficial for social interaction?
RQ4: Which chat systems did students find most effective for facilitating social interaction?
RQ5: What guidelines can be derived for software developers to better facilitate beneficial social interaction between students via chat systems?
A user study involving an online questionnaire was first distributed to Edinburgh University students, and attracted 110 answers to questions addressing RQ1-4. Next, a systematic literature review on students’ experience with using chat systems for social interaction during the pandemic was conducted, as led by the PRISMA statement↗️, to enrich results on RQ2-RQ4. Only academically published literature from during the pandemic, addressing HE students’ experience with chat systems, was included. The freely available and functional systems identified through the above steps (15 in total- see Table 1) were further evaluated against the features that were found to be beneficial, using an adapted version of a design walkthrough, to add even more to the answers to RQ4. This involved giving the systems scores for their inclusion of each beneficial feature and classifying them. Finally, guidelines for designers/developers were derived from the results, to address RQ5.
Results:
RQ1
The user study identified the following as the most important aspects of beneficial online social interaction for students:
- Eliciting positive feelings and emotions – mainly a sense of belonging or community.
- Avoiding negative feelings students – mainly loneliness and isolation.
- Facilitating positive relationships with fellow students, family and friends
RQ2
Most students used chat systems often – at least a few times a week and mostly via mobile phones. Students mainly used chat systems to maintain pre-existing relationships or form new ones and particularly valued being able to provide social support both for academic purposes e.g. questions about homework, exchanging academic advice, collaborative learning/group work, informal language practice; and non-academic purposes such as casual conversation and arranging meet-ups (online or in-person).
RQ3
Features students found beneficial include: graphics, voice notes, message reactions, sharing media files, calls, group chats, ease-of-use, accessibility and security.
Less beneficial features identified include: location-sharing, the primarily text-based nature of messaging resulting in message ambiguity, frequency and volume of messages which was exacerbated by build-up of notifications. Read receipts and online status also created a sense of urgency/ pressure to read and respond to messages when others know they are online or have seen a message and expect a fast reply.
RQ4
The user study (Figure 1) and systematic review identified, in order, Whatsapp↗️, Facebook Messenger↗️, Snapchat↗️ and Discord↗️ as being the most effective for students’ social interaction during the pandemic. According to our evaluation, Whatsapp↗️, Skype↗️ and Telegram↗️ ranked highest (see Table 1 for all system scores).
Generally the results indicate the chat systems students have most experience or were most familiar with were also the most preferred. The main reason students gave for why they preferred WhatsApp↗️ was its ubiquity – the fact that most of their peers also used the app, ease-of-use and wide range of features. Similar reasons were given for the other top systems. Conversely Kik Messenger↗️ ranked the lowest in the evaluation because of its limited features e.g. lacking call functionality which is now a core feature of a majority of chat systems.
Facebook Messenger↗️, Snapchat↗️, Discord↗️, Skype↗️ and Telegram↗️ were mainly identified as highly effective due to unique features such as connecting with others via Facebook accounts, Discord’s server/channel features for group communications and Skype/Telegram chatbots to name a few.
RQ5
We recommend designers/developers to include in chat systems features to facilitate:
- Multimodality: including graphics, voice messages, calls, media files to allow for non-verbal communication such as body language, tone of voice and facial expressions found in in-person interactions, aiding expression and reducing message ambiguity.
- Customisation: providing options for students to customise features can help limit potential issues students face e.g. giving the user the option to enable/disable Read receipts/Online status. Also, features such as options to change chat wallpaper, organise layout of chats and change profile picture or status makes socialising more personal and interactive.
- Group communication: Students can find group communication challenging due to the frequency and volume of messages. Therefore, features such as message reactions, chat-specific replies, muting, servers/channels and a poll can help ease group interactions for students.
Conclusion
Software developers can use our findings to upgrade or create chat systems that better facilitate students’ social interactions. Future work could extend the evaluation with paid and less popular chat systems, and develop a dedicated student social interaction chat system.
You can watch Muminah and Cristina’s conference presentation, ‘Evaluation of Chat and Video Conferencing Systems for Student Social Interaction During the Covid-19 Pandemic↗️, in the video below:
- You can also read about the first part of the study↗️ that assesses which Video Conferencing Systems students used during the Covid-19 pandemic.
Muminah Koleoso
Muminah Koleoso is a recent graduate with a BSc in Computer Science and Mathematics from The University of Edinburgh. Throughout her academic journey, she has developed a strong passion for technology and education. Contemplating further studies in a related field, Muminah currently works as a Web Content Support Assistant within the University’s Prospective Student Web Content Team.
Cristina Adriana Alexandru
Cristina Adriana Alexandru is a Lecturer in the School of Informatics and a Senior Fellow of the Higher Education Academy. She teaches a large-scale, second year undergraduate Software Engineering course, conducts training for academic staff, organises a summer programming course for incoming first year students, and line manages the team of University Teachers and University Tutors. She is passionate about teaching, computer science education, and building usable and useful technology for teaching and learning, which is why many of her undergraduate, Masters and research projects are in these areas. She hopes these projects will have a real impact on her school and community.
