Reframing “Bias” in AI Research

As a PhD student in the School of Informatics, I’ve been researching gender bias in language and language technologies.  Time and again, I’m surprised by how simple people try to make biased language.  As Abeba Birhane stresses in her interview on the podcast The Good Robot, not everything can be conceptualized as a straightforward problem with a straightforward solution.  Our cultures are dynamic and complex.  Our languages evolve slowly over decades, but also change rapidly based on our relationships with the people we are speaking to or writing for.  Moreover, language does not exist in a vacuum.   

In the branch of Linguistics called Critical Discourse Analysis, language is studied in its context of use, considering how it legitimizes and maintains power, and how it incites social change. [1]  Nevertheless, AI research approaches bias as a problem to be fixed, as if bias is an error that can be removed from a dataset, or a mistake that a model can be taught to avoid.  In reality, however, bias is an ongoing challenge. 

Bias changes with time, place, and culture.  Bias will always be with us, because there is no universal, neutral, or objective perspective.  We all are shaped by our own unique viewpoint, by our own experiences of the world.  

We need to reframe research questions about bias in data and technology.  Rather than focusing on removing bias, we need to better understand bias.  We need to study how bias comes through in language and other types of data.  We need to consider the risks bias poses and the harms bias may cause.  Researchers such as Abeba Birhane and Kate Crawford are among a small but growing group of people in the computational research community trying to do this.  There is a wealth of research in the Humanities and Social Sciences that the computational research community can look to; people have been studying and theorizing about language and bias for much longer than the existence of AI as a field.  The School of Informatics has been an exciting place for me to research bias in language technologies because I’ve had the opportunity to talk about new ways to approach bias and ethical AI research with fellow PhD students like Nina Markl and Bhargavi Ganesh.

To reframe questions about bias in data and technology, we need a culture shift in the AI field.  Currently, efficiency, convenience, and quantity drive dataset curation and model creation.  Being the first to publish something is highly valued, so gathering data and building models quickly are done at the expense of critical approaches to dataset and model development.  To gather data quickly, language and images are taken from the Internet without consent from the people who own them or are represented in them.  Datasets are evaluated based on how large they are rather than how representative they are. 

Instead, we need accuracy over efficiency, balance and representativeness over convenience, and quality over quantity.  Then we will realize that bias comes not from the model or the data, but from us, people and society.  Then we can focus on changing the power structures that cause harmful biases. 

References

[1] For more on Critical Discourse Analysis, see Analysing Discourse: Textual Analysis for Social Research (Fairclough, 2003) and Uses of Heritage (Smith, 2006). 

School response to survey results

Our School undertakes a culture survey of all School members every two years.  We don’t do this annually because we feel all this would result in is survey fatigue.  We know that completing the survey takes some time! So why do it?  We think the best incentive for completing the survey is evidence that the responses trigger changes that tackle the issues raised.  Here is a summary of the main issues raised by the 2021 survey  and how the School has responded.

Students

The survey saw 201 responses which was mainly completed by research postgraduate students. Response rates from undergraduate and taught masters students were low.  We’d like to see a significant increase in responses from all categories of student to the 2023 survey.  We hope the brief reports on each of the main issues identified in the survey will encourage more participation in the survey so we can have a clearer view of issues where things have improved and where we still need more work.  The main issues we identified were the following.

Workload

This is the clearest and most pressing issue that comes up in several different contexts and is seen as contributing to other issues identified in the survey.  Issues arise around the number, scale, and coordination of deadlines for coursework:

  • We use information from weekly reps meetings and Staff-Student Laison meeting to identify courses where workload is seen as an issue by students. These are reviewed and several courses have had the number and scale of courseworks reduced as a consequence of these reviews.
  • We have begun to make better use of the academic year by reconsidering the pattern of deadlines. Coursework-only courses can set deadlines beyond week 11 to make use of the early weeks of Semester 2 and the revision period prior to the main exam diet. This reduces deadline congestion and makes better use of the available weeks of study.
  • We are consulting now on reorganising the schedule the final-year project: deciding on a topic, preparing for the project and working on the project. Our goal is to avoid having the project run concurrently with other courses and permit a longer period of full-time work on the project.
  • Issues around deadline congestion are difficult to resolve. One approach we have considered is to have courseworks that span multiple courses to reduce the number of courseworks undertaken simultaneously.
Communication

Many responses point out that the respondents feel like the School is spamming them on multiple channels. Indiscriminate use of whole year mailing lists, multiple emails in the same day, inconsistent use of channels across courses all contribute to this feeling:

  • The move to LEARN ultra has started work in the School on how best to use the new structures. One opportunity is to establish a more consistent policy on the messaging related to individual courses.
  • We are considering the use of stricter moderation on the large and indiscriminate email lists (e.g.,
  • We are also actively considering options to request journaled messages on some of the more active lists.
Community and Caring

This is a somewhat more controversial topic since there is a minority view that questions whether the School should care about community and caring but the majority feel the School should attempt to build a caring community. In this area we have:

  • Initiated the development of basic training in Equality, Diversity and Inclusion oriented to students to help engender a more open, respectful dialogue in the School that will help counter the perceived difficulties some students experience in expressing their views to other students.
  • The variability in sense of community experienced by PhD students is also problematic. The School is considering how best to engender a stronger sense of community across all PhD students.
Timing of Events

Some student respondents raise the issue of the timing of events that assume students are always available. The School will now endeavour to ensure that events are more sympathetically timed.

Bullying and Harassment

Overall the level of bullying and harassment is low in the School.  However, the School will endeavour:

  • To make reporting mechanisms clear and more clearly anonymous to respond to the expressed lack of knowledge on how to report bullying and harassment.
  • The School is aware the EUSA is promoting active bystander training for some societies’ members. The School is exploring how to make such training more widely available to all students.
Mental Health and Wellbeing

This is seen as a major deficiency. The time delay and lack of mental health and wellbeing provision is problematic for most respondents.  These services are provided university-wide so there is little the School can do directly in terms of increasing the supply of services but we are exploring ways we can reduce demand:

  • Exploring how to reduce stress levels among our students. Better management of coursework loads (see earlier) are an important route to reducing stress.
  • Increasing the number of mental health first aiders in the School. This is not a long-term fix but having a wider trained group improves accessibility to prompt help and increases awareness and sensitivity to the issue in the School.
  • Our new expert student support staff will help ensure students receive prompt and consistent support for mental health issues. The School believes this is a significant improvement over the current situation.  The switch to the new system takes place over the summer.

Staff

The survey saw 185 responses which is a significantly higher response rate than the student survey. This has a good spread across all staff categories and levels of seniority.

Workload

This is one of the clearest and most consistent issues across all staff related to students and is related to increases in student numbers. It is seen a major contributor to poor wellbeing, stress and mental health issues.

  • Action to reduce the volume of assessed coursework mentioned above has a direct impact on staff workload. This work is continuing, and consultation on managing the final year project workload is underway.
  • Action has been taken on admissions more effectively to control admissions of taught students and growth in student numbers has been brought under control.
Community and Caring

The School is a large organisation and building an effective and caring community is challenging.

  • Continuing to strengthen the role of the Institutes provides smaller communities for some categories of staff and students that are still evolving, particularly post-COVID.
  • Strong staff networks are also seen as good mechanism to encourage communities with common interests.
  • Individual initiative such as yoga classes and the concert series also provide mechanisms that encourage interaction and socialising across all staff.
Equality, Diversity, and Inclusion

Currently the main focus in the surveys is on gender issues but the School is aware of wider EDI issues.

  • New training in EDI impact assessment will be used to ensure that all new policies are assessed for EDI impact.
  • EDI impacts will be documented and followed up by the People and Culture Committee.
  • Data on EDI impact on promotion will be gathered and analysed systematically to provide a good evidence base for further action.
Bullying and Harassment

Some bullying is experienced, particularly by more junior staff and between academic and other staff.  Bullying is clearly unacceptable.

  • School will establish a confidential channel to report bullying and will publicise policies and reporting channels widely.
Progression and Promotion

There is a feeling that decision taking lacks transparency and support for development to enable promotion/progression.

  • School is organising additional training for Line Managers to enable them better to support the development needs of staff.
  • School is working to provide clearer career route mapping.
Mental Health and Wellbeing

Issues around mental health and wellbeing are closely related to workload.

  • The workload model is explicit and transparently implemented. This is still becoming fully established.  As it beds in we anticipate being better able to identify under-resourcing and the need to recruit to better resource under-resourced activities.
  • School is working with the wider University to increase staff access to mental health services.

Edinburgh at the BCSWomen Lovelace Colloquium

BCSWomen organises the annual Lovelace Colloquium: a day featuring talks, a careers panel, employer stands and a student poster contest. This year, three students from the University of Edinburgh made the trip to Sheffield, and Qiuye Zhang in fact won first place with her poster “Can Artificial Neural Networks Learn like Brains?” in the second year contest! Here is how she experienced the event:

I am excited to share my experiences and insights from the Lovelace Colloquium, where I had the opportunity to present my poster on computational neuroscience and computational psychiatry. It was my first time discussing these two fascinating fields publicly, and I was thrilled to see some attendees express interest in computational psychiatry.

Initially, my abstract didn’t mention computational psychiatry, but after being inspired by Peggy and her course on computational cognitive neuroscience, I decided to include it in my poster. The interest and discussions surrounding my presentation exceeded my expectations. We delved into topics beyond the scope of my poster, such as Hopfield networks, Bayesian models, and reinforcement learning models. The judges of the contest were very encouraging about my current research. Their kind words and support reinforced my passion for the subject and motivated me to continue my work in this field.

The event also allowed me to meet many amazing people who provided warm hugs and support when I felt nervous before my presentation. In addition to my poster experience, the keynote speeches were enlightening. They touched on the biases faced by females, gender-neutral individuals, and disabled people, as well as the use of technology to detect violence.

Going forward, I plan to be more mindful of potential biases in my research, particularly concerning people with psychiatric diseases. I will consider whether they receive adequate support and explore how to facilitate their lives when cognitive control is a challenge.

Overall, the Lovelace Colloquium was an enriching experience that allowed me to share my passion for computational neuroscience and psychiatry, learn from others, and connect with amazing people. I’m grateful for the opportunity and look forward to applying my newfound insights in my future work.

Can artificial neural networks learn like brains?

Qiuye Zhang’s poster “Can artificial neural networks learn like brains?”

What can we do about the gender disparity in Computer Science in the UK?

There are disproportionately few women enrolling for undergraduate degrees in computing in the UK.  Despite constituting 50.5% of the UK population and 57% of college graduates in the UK, only 19% of the technology workforce are women. The statistics for staff within our school align well with the national figures, with women constituting 56% of our professional services staff but only 20% of our academic staff. The disparity is lower in our student population: 27% of taught students (24.2% of undergraduates and 36% of post-graduate) and 22% of research students identified as female.

The decisions made by pupils in the last two years of high school is a key contributor to this disparity. The female to male ratio for first year STEM undergraduates across the UK hovers around 1, but its breakdown across disciplines reveals wide variation across the sciences (Figure 2 shows a detailed breakdown for interested readers). About one in four Computer Science (or Engineering) undergraduates identifies as female.  Those who identified as ‘other’ when given a 3-way choice of gender (about 0.4% of UK’s population) make up about 0.2% of first year undergraduates in the UK; within our school the numbers are significantly better than the national average (0.7% of taught students and 2% of research students).

Most of the interventions designed and delivered in the UK [2,3], to reduce gender disparity in STEM, have been targeted at high school students. Specifically, focussing on female pupils to educate them about the benefits of choosing careers in science, via mechanisms such as the Stimulating Physics Network [1]. It appears that female pupils choose non-mandatory STEM subjects [9,10]  in secondary schools when they:

  1.   believe that they are `good’ at it;
  2.   appreciate the value of science;
  3.   are embedded in a micro-culture that values and discusses science [7]; and
  4.   are exposed to role models provided they do not conform to STEM stereotypes [4,5].

The UK has spearheaded studies related to 3, under the umbrella of ‘science capital’ [6].

Paradoxically, there is  evidence [8] that gender disparity in engineering and technology is inversely related to national gender equality. That is, countries with higher percentages of women engineers (around 40%) tend to have a poor global gender gap index. E.g. Algeria, Tunisia, U.A.E., Turkey, Indonesia, Vietnam.

 

Fig 1. Source: Technation jobs and skills report 2021.

 

In summary, with only 20-25% of undergraduates in Engineering and Computer Science being women, the status quo precludes the majority of women in the UK from gaining the skills needed for lucrative tech jobs (Figure 1). It is a large and complex issue.  What can we at the School of Informatics, as the powerhouse of computer science in the UK, do about this?

My view is that we could aim to overcome this disparity at three different levels:

  1. we are probably large enough to effect change by leading and organising effort at the national level (spawning something like the SPN but specifically for computing) while liaising with government (e.g. Scottish Parliament, Department for Education);
  2. locally, we could identify appropriate [5] role models within the school who connect with schools to keep female pupils (and their teachers) informed of the impact of their choice of subjects to society and their own careers; and finally. For example through the Informatics Tutoring Scheme.
  3. every one of us should actively contribute to an inclusive environment within the school. Although this sounds obvious and trivial, we continue to hear about scope for improvement, in this regard, via our student surveys.

What do you think?

 

Fig 2. Ratios of full-time female to male students in first year undergraduate (left) vs graduate (right) programmes in the year 2020-2021. Data from HESA.

 

 

REFERENCES

[1] https://www.stem.org.uk/secondary/cpd/stimulating-physics-network

[2] https://ffteducationdatalab.org.uk/wp-content/uploads/2022/01/SPN-evaluation-final-Jan-22.pdf

[3] Team, Behavioural Insights. “Applying Behavioural Insights to increase female students’ uptake of STEM subjects at A Level” (2020).

[4] Cheryan, Sapna, John Oliver Siy, Marissa Vichayapai, Benjamin J. Drury, and Saenam Kim. “Do female and male role models who embody STEM stereotypes hinder women’s anticipated success in STEM?” Social psychological and personality science 2, no. 6 (2011): 656-664.

[5] Cheryan, Sapna, Allison Master, and Andrew N. Meltzoff. “Cultural stereotypes as gatekeepers: Increasing girls’ interest in computer science and engineering by diversifying stereotypes” Frontiers in psychology (2015): 49.

[6] Archer, Louise, Emily Dawson, Jennifer DeWitt, Amy Seakins, and Billy Wong. ““Science capital”: A conceptual, methodological, and empirical argument for extending bourdieusian notions of capital beyond the arts.” Journal of research in science teaching 52, no. 7 (2015): 922-948.

[7] Archer, Louise, Julie Moote, Emily Macleod, Becky Francis, and Jennifer DeWitt. “ASPIRES 2: Young people’s science and career aspirations, age 10–19.” (2020).

[8] Stoet, Gijsbert, and David C. Geary. “The gender-equality paradox in science, technology, engineering, and mathematics education.” Psychological science 29, no. 4 (2018): 581-593.

[9] Wigfield, Allan, and Jacquelynne S. Eccles. “Expectancy–value theory of achievement motivation.” Contemporary educational psychology 25, no. 1 (2000): 68-81.

[10] Goulas, Sofoklis, Silvia Griselda, and Rigissa Megalokonomou. “Comparative advantage and gender gap in STEM.” Journal of Human Resources (2022): 0320-10781R2.

 

School Values

Values, with a capital V, are core principles that we, as a School, stand for. They are part of our School strategic plan, and are specific to our community. You may think this is a pointless exercise in bureaucracy.
Why would we have to write these things down? There are three reasons why I think it really is valuable to explicitly record them.

  • Culture is a nebulous thing. Most of us recognise a rotten work environment when we’re in one. But it is much more difficult to pin down how to bring about a pleasant culture, and how to keep it up. Having a common core to tie together fragmented policies helps and sets expectations. Agreeing on, and being reminded of, our shared values lets us all make better decisions about how to behave, staff and students alike.
  • Having values explicit makes it easier for others to appreciate our School culture. In the last rounds of academic recruitment, many candidates commented on how welcoming and collegial they found the School. The same goes for professional services, where staff move around Schools within the University more often, but they come to Informatics because it’s a nice place to work. And the same goes for prospective students deciding whether they want to spend some of their formative years with us. We want to attract the best and nicest people to work with, and having explicit values for applicants to see when they do their homework before applying helps with that.
  • Having a common frame of reference makes it easier stand up to actions that do not live up to our standards. When something happens that you’re not sure was necessarily the best thing, it can be hard to actively say something about it yourself if you’re unsure how others feel. That’s sometimes called the bystander effect. But if we have agreed on our values, you do know to some extent how others feel, making it easier to see what we think is ok and what is not, and so help each other improve and become more effective as a community.

Now, this culture belongs to all of us, not just the 25 or so people who drafted the below list. All of us should agree on it, and revise this living document over time. So we’re going to ask all of you for your input. There is room for improvement on the draft list below. For example, some items cover similar sentiments and might be fused. Or maybe you plain don’t agree with some. Please look at the draft values and other’s thoughts, and contribute your comments.

  • Respect: We value openness and high standards of fairness, always being principled, considerate, and respectful to each other.
  • Inclusion: We are diverse, inclusive and accessible to all, and celebrate our deep-rooted and distinctive internationalism.
  • Collaboration: We have a strong sense of community, work together to achieve our goals, and help to get the best out of each other.
  • Excellence: We aim to achieve excellence in all that we do: teaching, research, societal responsibility.
  • Curiosity: We are open-minded lifelong learners who value freedom of expression.
  • Bravery: We are willing to question norms we take for granted, and call out injustices.
  • Humility: We appreciate that we may never fully understand, but educate ourselves to be as competent as we can be.

Research shows, time and again, that people are more productive when they feel valued and secure. As our Head of School likes to say, “happy chickens lay more eggs”. We don’t all have to be best buddies. In fact we can have passionate disagreements and robust discussions, about academic content, about how to improve working processes, or about teaching approaches. And that’s a good thing. But we do have to be able to work together. Agreeing on shared values explicitly is good hygiene for interpersonal work relationships.

Do you feel these represent (y)our values? Would you change, add, remove, fuse, or reorder any?

 

Why Fairness Matters

People make the School of Informatics. Whether academic, professional services, or student, without people teaching and research would halt. And people are at their best when they feel they belong in the School’s culture. At the root of an inclusive culture lies fairness, which brings out the best in people. Fairness matters, as any child knows instinctively. Because this is so obvious, it can be hard to explain. Social and economic research gives two main practical reasons why we care:

  • First, great ideas can come from anywhere. Working with people from different backgrounds and ways of thinking makes us better. Whether it is scientific debate, improving administrative processes, or tutorial discussions, variety of experience and perspective brings more creative solutions.
  • Second, people are very sensitive to inequality and unfairness. We can detect it instantly. Unfairness increases conflict and tension. When we experience too much of it we tune out, either by physically leaving or by mentally stopping productivity, sometimes involuntarily. Nobody likes conflict. On the upside, a fair culture is not just one in which people genuinely want to work and study. It also nourishes a good reputation and attracts more good things.

So, for everybody to contribute their skills, talents, and ideas, to the benefit of all, fairness matters. Challenging conventions, questioning existing ways of thinking, and sometimes robust debate, only work well when people feel confident that their views will be heard fairly. Not being constrained by what other people think boosts creativity.

Time and again research shows that in most successful teams people trust and respect each other, which allows them to take risks and be vulnerable. Increasing fairness is good for everybody. People who feel fairly treated enjoy themselves more, are more likely to help colleagues, and are more willing to persist in difficult times. Here is a very utilitarian take from Nobel Prize-winning economist Paul Krugman:

Workers are people. Raising the minimum wage makes jobs better; it doesn’t seem to make them scarcer. How is that possible? Workers are not, in fact, commodities. A bushel of soybeans doesn’t care how much you paid for it; but decently paid workers tend to do a better job.

To be fair, you don’t have to go all the way to egalitarianism, where everybody shares efforts and results equally. Some people are naturally better at some things than others, and that diversity should be used to the advantage of the whole community. We want to work with the best colleagues, even if they cannot take credit completely for their talent and effort. But the possibility and support to improve yourself should be open to everyone. People are more prepared to accept unfair outcomes if they feel that the process that led to them was fair. But we balk at unfair process even when it leads to fair outcomes.

Fairness is linked to responsibility and accountability, especially when it comes to diversity. Most of us tend to prefer the familiar. Leaning into differences between us is easier said than done. But research supports the idea that diversity in our backgrounds and beliefs translates into diversity in our ways of thinking.

Fairness matters. Diversity has practical benefits. So be fair, overcome unfamiliarity, and help your fellow people get the best out of themselves to the benefit of yourself and our entire School.

References

 

Black History Month

Our School is a community of people from many different backgrounds, which makes it an inspiring place to learn and work. Members are respected for who they are, and the culture they come from doesn’t matter. So, in a sense, it is weird to spotlight some groups over others. Rather than make a big deal, let’s keep being kind and supporting each other no matter what their background. It’s the small things that matter.

Nevertheless, it is Black History Month! Let’s celebrate the diversity we have, which plays a vital role in our School, including the many other cultural influences in the UK. This is an opportunity to pause and reflect on the priviledges we enjoy as students and staff in the School of Informatics. To see that this is not automatic, you may want to learn more about various people’s experiences or take action. Additionally, the University is holding various events you could attend.

Events:

Actions:

Reading lists:

Books: