The inscrutable nature of AI is often referred to as the ‘black box’ problem because the systems are ‘opaque’, difficult to explain or understand, making it impossible to regulate their use in education, which could have legal and ethical implications (Zednik, 2019).
The European Commission produced ethical guidelines on the use of AI addressing the concerns of AI systems lacking transparency and recommending AI systems deployed should include, ‘‘traceability, explainability and communication’’ (European Commission, 2022 p.18).
Likewise the US department of Education (2023) published their recommendations and identified a lack of ‘explainability’ of AI models, urging industry leaders to work together to foster trust.
‘’So, having open learning environments or inspectable learner models or applications where the stakeholders can understand how these systems make decisions or recommendations is going to be an important aspect in the future of teaching and learning.” —Diego Zapata-Rivera p.36
UNESCO’s (2023) report on ‘Guidance for generative AI in education and research’ suggests providers should explain data used by models and algorithms used to generate content and offer support to understand the technology and data.
However, these guidelines, recommendations and insights do not hold anyone legally accountable for seeking ‘transparency’ or ‘explainability’ in AI systems.
It is questionable whether it is even possible to seek transparency, Bearman & Ajjawi (2022), argue that AI must always act as a ‘black box’ as by definition its judgement on a course of action is untraceable. The practical limitations here pose great challenges for gaining any real insight into the ‘black box’ of AI due to its opaque nature.
Activity 2 Watch Dr. Timnit Gebru talk about AI regulation in ‘How to make AI systems more just with Hilary Pennington #OnWhatMatters from (17.22-18.20) and answer the question below.
What perspective does Dr.Gebru give about why the inscrutable nature of AI need not be an impediment to legislating it?
On What Matters want this video to be accessible to the widest possible audience
The image below, of a cat peering out of the window is an analogy that helps me see the problem from another perspective. The rain soaked window might not appear to bring great insight but what is out there could have great significance.
Featured Image All photos and videos on Pexels can be downloaded and used freely. cat looking out of window with raindrops on it
Activity 3
Examining the environment in which a system is performing in, could reveal more than looking in at how it behaves.
Do you think this is a useful approach to take when considering the use of AI in education?
Go to the Padlet or use the QR code below to respond to the discussion questions.
References
Bearman, M., & Ajjawi, R. (2023). Learning to work with the black box: Pedagogy for a world with artificial intelligence. British Journal of Educational Technology, [e-journal] 54 (5) pp.1160-1173 Available at :Learning to work with the black box: Pedagogy for a world with AI (Accessed: 10 October 2023).
European Commission, (2022).Directorate-General for Education, Youth, Sport and Culture, Ethical guidelines on the use of artificial intelligence (AI) and data in teaching and learning for educators. [online] Publications Office of the European Union, Available at: The European Commission Ethical Guidelines on the use of AI and data in teaching and learning for educators (Accessed 28 October 2023).
UNESCO (2023). Guidance for generative AI in education and research [online] Available at: Guidance for generative AI in education and research – UNESCO Digital Library (Accessed 12 October 2023).
U.S. Department of Education (2023). Office of Educational Technology, Artificial Intelligence and Future of Teaching and Learning: Insights and Recommendations, [online] Washington, DC, 2023. Available at Artificial Intelligence and Future of Teaching and Learning: Insights and Recommendations-US Department of Education (Accessed 28 October 2023).
Zednik, C, (2019). Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence Philosophy & Technology (2021) 34:265–288. Available at: Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence (Accessed: 15 October 2023).
The next section is #3. Futuring will discuss the future and its transformative potential. 3. Futuring
