Participatory design (PD) and co-creation (Co-C) approaches to building Artificial Intelligence (AI) systems have become increasingly popular exercises for ensuring greater social inclusion and fairness in technological transformation by accounting for the experiences of vulnerable or disadvantaged social groups; however, such design work is challenging in practice, partly because of the inaccessible domain of technical expertise inherent to AI design. This paper evaluates a methodological approach to make addressing AI bias more accessible by incorporating a training component on AI bias in a Co-C process with vulnerable and marginalized participant groups. This was applied by socio-technical researchers involved in creating an AI bias mitigation developer toolkit. This paper’s analysis emphasizes that critical reflection on how to use training in Co-C appropriately and how such training should be designed and implemented is necessary to ensure training allows for a genuinely more inclusive approach to AI systems design when those most at risk of being adversely affected by AI technologies are often not the intended end-users of said technologies. This is acutely relevant as Co-C exercises are increasingly used to demonstrate regulatory compliance and ethical practice by powerful institutions and actors developing AI systems, particularly in the ethical and regulatory environment coalescing around the European Union’s recent AI Act.
AI fairness is at the center of many debates, but there are different perspectives on what it entails. Currently, purely technical algorithmic fairness approaches dominate the scene, often neglecting a sufficiently well-rounded view of social implications and ignoring the voices of lay people. With the end goal of overcoming these issues, we investigate and synthesize the points of contact and differences among computer science, sociological and lay people perspectives and move towards a lay-socio-technical view of AI fairness. To this end, we conducted interviews with experts from the computer and social sciences, as well as a survey and co-creation workshops with lay people. Our results show that there are sometimes conflicting views on AI fairness across these perspectives. An integrated view requires two processes of negotiation: (a) between computer and social sciences, and (b) between experts from both disciplines and lay people. We identify strategies for supporting these processes, but we state that they are ultimately possible only if there is a rebalance of current power relations between the disciplines, and between experts and lay people.