>>scientific articles

Abstract. This paper sheds light on the role of digital platform labour in the development of today’s artificial intelligence, predicated on data-intensive machine learning algorithms. Focus is on the specific ways in which outsourcing of data tasks to myriad ‘micro-workers’, recruited and managed through specialized platforms, powers virtual assistants, self-driving vehicles and connected objects. Using qualitative data from multiple sources, we show that micro-work performs a variety of functions, between three poles that we label, respectively, ‘artificial intelligence preparation’, ‘artificial intelligence verification’ and ‘artificial intelligence impersonation’. Because of the wide scope of application of micro-work, it is a structural component of contemporary artificial intelligence production processes – not an ephemeral form of support that may vanish once the technology reaches maturity stage. Through the lens of micro-work, we prefigure the policy implications of a future in which data technologies do not replace human workforce but imply its marginalization and precariousness.

Abstract: Recent studies demonstrate that machine learning algorithms can discriminate based on classes like race and gender. In this work, we present an approach to evaluate bias present in automated facial analysis algorithms and datasets with respect to phenotypic subgroups. Using the dermatologist approved Fitzpatrick Skin Type classification system, we characterize the gender and skin type distribution of two facial analysis benchmarks, IJB-A and Adience. We find that these datasets are overwhelmingly composed of lighter-skinned subjects (79.6% for IJB-A and 86.2% for Adience) and introduce a new facial analysis dataset which is balanced by gender and skin type. We evaluate 3 commercial gender classification systems using our dataset and show that darker-skinned females are the most misclassified group (with error rates of up to 34.7%). The maximum error rate for lighter-skinned males is 0.8%. The substantial disparities in the accuracy of classifying darker females, lighter females, darker males, and lighter males in gender classification systems require urgent attention if commercial companies are to build genuinely fair, transparent and accountable facial analysis algorithms.

Key takeaways: As algorithmic systems become more critical to decision making across many parts of society, there is increasing interest in how they can be scrutinised and assessed for societal impact, and regulatory and
normative compliance.

Abstract: Trained machine learning models are increasingly used to perform high-impact tasks in areas such as law enforcement, medicine, education, and employment. In order to clarify the intended use cases of machine learning models and minimize their usage in contexts for which they are not well suited, we recommend that released models be accompanied by documentation detailing their performance characteristics. In this paper, we propose a framework that we call model cards, to encourage such transparent model reporting. Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information. While we focus primarily on human-centered machine learning models in the application fields of computer vision and natural language processing, this framework can be used to document any trained machine learning model. To solidify the concept, we provide cards for two supervised models: One trained to detect smiling faces in images, and one trained to detect toxic comments in text. We propose model cards as a step towards the responsible democratization of machine learning and related AI technology, increasing transparency into how well AI technology works. We hope this work encourages those releasing trained machine learning models to accompany model releases with similar detailed evaluation numbers and other relevant documentation.

Abstract: The State of Indiana denies one million applications for healthcare, foodstamps and cash benefits in three years—because a new computer system interprets any mistake as “failure to cooperate.” In Los Angeles, an algorithm calculates the comparative vulnerability of tens of thousands of homeless people in order to prioritize them for an inadequate pool of housing resources. In Pittsburgh, a child welfare agency uses a statistical model to try to predict which children might be future victims of abuse or neglect.

Since the dawn of the digital age, decision-making in finance, employment, politics, health and human services has undergone revolutionary change. Today, automated systems—rather than humans—control which neighborhoods get policed, which families attain needed resources, and who is investigated for fraud. While we all live under this new regime of data, the most invasive and punitive systems are aimed at the poor.

In Automating Inequality, Virginia Eubanks systematically investigates the impacts of data mining, policy algorithms, and predictive risk models on poor and working-class people in America. The book is full of heart-wrenching and eye-opening stories, from a woman in Indiana whose benefits are literally cut off as she lays dying to a family in Pennsylvania in daily fear of losing their daughter because they fit a certain statistical profile.

The U.S. has always used its most cutting-edge science and technology to contain, investigate, discipline and punish the destitute. Like the county poorhouse and scientific charity before them, digital tracking and automated decision-making hide poverty from the middle-class public and give the nation the ethical distance it needs to make inhumane choices: which families get food and which starve, who has housing and who remains homeless, and which families are broken up by the state. In the process, they weaken democracy and betray our most cherished national values.

This deeply researched and passionate book could not be more timely.

Abstract: We summarize the potential impact that the European Union’s new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user-level predictors) which “significantly affect” users. The law will also effectively create a “right to explanation,” whereby a user can ask for an explanation of an algorithmic decision that was made about them. We argue that while this law will pose large challenges for industry, it highlights opportunities for computer scientists to take the lead in designing algorithms and evaluation frameworks which avoid discrimination and enable explanation.

Abstract: Data is fundamental to the modern world. From economic development to health care to education and public policy, we rely on numbers to allocate resources and make crucial decisions. But because so much data fails to take into account gender, because it treats men as the default and women as atypical, bias and discrimination are baked into our systems.

And women pay tremendous costs for this insidious bias: in time, in money, and often with their lives.

Celebrated feminist advocate Caroline Criado Perez investigates this shocking root cause of gender inequality in Invisible Women. Examining the home, the workplace, the public square, the doctor’s office, and more, Criado Perez unearths a dangerous pattern in data and its consequences on women’s lives.

Product designers use a “one-size-fits-all” approach to everything from pianos to cell phones to voice recognition software, when in fact this approach is designed to fit men. Cities prioritize men’s needs when designing public transportation, roads, and even snow removal, neglecting to consider women’s safety or unique responsibilities and travel patterns. And in medical research, women have largely been excluded from studies and textbooks, leaving them chronically misunderstood, mistreated, and misdiagnosed.

Built on hundreds of studies in the United States, in the United Kingdom, and around the world, and written with energy, wit, and sparkling intelligence, this is a groundbreaking, highly readable exposé that will change the way you look at the world.

Abstract: Feminist AI: Critical Perspectives on Algorithms, Data and Intelligent Machines is the first volume to bring together leading feminist thinkers from across the disciplines to explore the impact of artificial intelligence (AI) and related data-driven technologies on human society.

Recent years have seen both an explosion in AI systems and a corresponding rise in important critical analyses of these technologies. Central to these analyses has been feminist scholarship, which calls upon the AI sector to be accountable for designing and deploying AI in ways that further, rather than undermine, the pursuit of social justice. This book aims to be a touchstone text for AI researchers concerned with the social impact of their systems, as well as theorists, students and educators in the field of gender and technology. It demonstrates the importance of an intersectional understanding of the risks and benefits of AI, approaching feminism as a political project that aims to challenge various interlocking forms of injustice, social inequality and structural relations of power.

Feminist AI showcases the vital contributions of feminist scholarship to thinking about AI, data, and intelligent machines as well as laying the groundwork for future feminist scholarship on AI. It brings together scholars from a variety of disciplinary backgrounds, from computer science, software engineering, and medical sciences to political theory, anthropology, and literature. It provides an entry point for scholars of AI, science and technology into the diversity of feminist approaches to AI, and creates a rich dialogue between scholars and practitioners of AI to examine the powerful congruences and generative tensions between different feminist approaches to new and emerging technologies. It features original and essential works specially selected to span multiple generations of practitioners and scholars.

These contributors are also attuned to conversations at industry-level around the risks and possibilities that frame the drive to adopt AI. This collection reflects the increasingly blurred divide between the academy, industry and corporate research groups and brings interdisciplinary feminist insights together with postcolonial studies, disability theory, and critical race studies to confront ageism, racism, sexism, ableism, and class-based oppressions in AI.

Abstract: Following COVID-19, there has been an increase in digitization and use of Artificial Intelligence (AI) across all spheres of life, which presents both opportunities and challenges. This commentary will explore the landscape of the gendered impact of AI at the intersections of Science and Technology Studies, feminist studies (socialist feminism), and computing. The Global Dialogue on Gender Equality and Artificial Intelligence (2020) organized by UNESCO highlighted the inadequacy of AI normative instruments or principles which focus on gender equality as a “standalone” issue. Past research has underscored the gender biases within AI algorithms that reinforce gender stereotypes and potentially perpetuate gender inequities and discrimination against women. Gender biases in AI manifest either during the algorithm’s development, the training of datasets, or via AI-generated decision-making. Further, structural and gender imbalances in the AI workforce and the gender divide in digital and STEM skills have direct implications for the design and implementation of AI applications. Using a feminist lens and the concept of affective labor, this commentary will highlight these issues through the lenses of AI in virtual assistants, and robotics and make recommendations for greater accountability within the public, private and nonprofit sectors and offer examples of positive applications of AI in challenging gender stereotypes.

Abstract. On 21 April 2021, the European Commission published the proposal of the new EU Artificial Intelligence Act (AIA) — one of the most influential steps taken so far to regulate AI internationally. This article highlights some foundational aspects of the Act and analyses the philosophy behind its proposal.

Abstract: The public sector is undergoing a transformation characterized by the increasing adoption of digital technologies, commonly referred to as digital government. Within this context, caseworkers who manage citizens seeking benefits from welfare organizations play a crucial role in providing public services to citizens. As digital technologies increasingly shape decision-making processes, their work now encompasses a significant amount of data work, involving understanding, managing, and producing citizens’ data through meticulous documentation. Caseworkers are in charge of providing high-quality services to citizens in need. To do so, they navigate a complex web of regulations and utilize various digital systems that generate many invisible tasks. In this study, we aim to shed light on caseworkers’ data work. We conducted a qualitative case study in a Scandinavian welfare agency, focusing on caseworkers’ utilization and production of documentation. Our findings reveal that caseworkers’ data work is articulated through three work practices mandated by the complex context. We abstracted these practices into two classes of invisible work activities that significantly impact their daily responsibilities. By unveiling these hidden aspects, we contribute valuable empirical insights to policymakers and emphasize the importance of data work and invisible work concepts in the field of digital government.

Abstract: The emergence of the modern state was closely intertwined with the advent of statistics and demographic data. Today, we are witnessing the ascent of artificial intelligence as a new technology of governance. This article seeks to lay the groundwork for a research agenda at the intersection of the state and artificial intelligence, unpacking the notion of “AI” and examining the consequences of the state transitioning from statistics to artificial intelligence as the means of “seeing” its subjects. The first part of the article argues that artificial intelligence represents a fundamental epistemic shift: from variables to patterns, from rules to associations, from surveys to sensors. This transition may transform governance and biopolitics, and with them, the very meanings of concepts such as citizenship, democracy, and population. In the second part of the article, the article draws on the literature on socio-technical transitions to conceptualize the integration of artificial intelligence into state practices, offering a framework to guide empirical research on how artificial intelligence is transforming governance.

Abstract. This article argues that the current hype surrounding artificial intelligence (AI) exhibits characteristics of a tech bubble, based on parallels with five previous technological bubbles: the Dot-Com Bubble, the Telecom Bubble, the Chinese Tech Bubble, the Cryptocurrency Boom, and the Tech Stock Bubble. The AI hype cycle shares with them some essential features, including the presence of potentially disruptive technology, speculation outpacing reality, the emergence of new valuation paradigms, significant retail investor participation, and a lack of adequate regulation. The article also highlights other specific similarities, such as the proliferation of AI startups, inflated valuations, and the ethical concerns associated with the technology. While acknowledging AI’s historical, transformative potential, the article calls for pragmatic caution, evidence-based planning, and critical thinking in approaching the current hype. It concludes by offering some recommendations to minimise the negative impact of the impending bubble burst, emphasising the importance of focusing on sustainable business models and real-world applications, maintaining a balanced perspective on AI’s potential and limitations, and supporting the development of effective regulatory frameworks to guide the technology’s design, development, and deployment 

About. In Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence Crawford reveals how the global networks underpinning AI technology are damaging the environment, entrenching inequality, and fueling a shift toward undemocratic governance. She takes us on a journey through the mining sites, factories, and vast data collections needed to make AI "work" — powerfully revealing where they are failing us and what should be done. Atlas of AI was named one of the best books on technology in 2021 by the Financial Times.

While technical systems present a veneer of objectivity and neutrality, Crawford shows how they are designed to serve and intensify existing systems of power. Drawing on a decade of original research, she shows how the new infrastructures of AI reflect the beliefs and perspectives of a small group of people and serve the interests of the few at the expense of the many. 

Abstract. We are repeatedly told that AI will help us to solve some of the world's biggest challenges, from treating chronic diseases and reducing fatality rates in traffic accidents to fighting climate change and anticipating cybersecurity threats. However, the article contends that public discourse on AI systematically avoids considering AI’s environmental costs.

Artificial Intelligence- Brevini argues- runs on technology, machines, and infrastructures that deplete scarce resources in their production, consumption, and disposal, thus increasing the amounts of energy in their use, and exacerbate problems of waste and pollution. It also relies on data centers, that demands impressive amounts of energy to compute, analyse, categorize. If we want to stand a chance at tackling the Climate Emergency, then we have to stop avoiding addressing the environmental problems generated by AI.

Abstract. From Charles Babbage's Difference Engine (a product of an increasingly global British maritime empire) to Herman Hollerith's tabulating machine (designed to solve the problem of "seeing like a state" in the newly transcontinental American Republic) to the emergence of the modern petrochemical industry, information technologies have always been closely associated with the human desire to understand and manipulate their physical environment. More recently, humankind has started to realize the environmental impacts of information technology, including not only the toxic byproducts associated with their production, but also the polluting effects of the massive amounts of energy and water required by data centers at Google and Facebook (whose physicality is conveniently and deliberately camouflaged behind the disembodied, ethereal "cloud"). This paper grounds the history of information technology in the material world by focusing on the relationship between "computing power" and more traditional processes of resource extraction, exchange, management, and consumption.

Abstract: In the past five years, private companies, research institutions and public sector organizations have issued principles and guidelines for ethical artificial intelligence (AI). However, despite an apparent agreement that AI should be ‘ethical’, there is debate about both what constitutes ‘ethical AI’ and which ethical requirements, technical standards and best practices are needed for its realization. To investigate whether a global agreement on these questions is emerging, we mapped and analysed the current corpus of principles and guidelines on ethical AI. Our results reveal a global convergence emerging around five ethical principles (transparency, justice and fairness, non-maleficence, responsibility and privacy), with substantive divergence in relation to how these principles are interpreted, why they are deemed important, what issue, domain or actors they pertain to, and how they should be implemented. Our findings highlight the importance of integrating guideline-development efforts with substantive ethical analysis and adequate implementation strategies.

Abstract. The progress of AI systems such as large language models (LLMs) raises increasingly pressing concerns about their safe deployment. This paper examines the value alignment problem for LLMs, arguing that current alignment strategies are fundamentally inadequate to prevent misuse. Despite ongoing efforts to instill norms such as helpfulness, honesty, and harmlessness in LLMs through fine-tuning based on human preferences, they remain vulnerable to adversarial attacks that exploit conflicts between these norms. I argue that this vulnerability reflects a fundamental limitation of existing alignment methods: they reinforce shallow behavioral dispositions rather than endowing LLMs with a genuine capacity for normative deliberation. Drawing from on research in moral psychology, I show how humans’ ability to engage in deliberative reasoning enhances their resilience against similar adversarial tactics. LLMs, by contrast, lack a robust capacity to detect and rationally resolve normative conflicts, leaving them susceptible to manipulation; even recent advances in reasoning-focused LLMs have not addressed this vulnerability. This “shallow alignment” problem carries significant implications for AI safety and regulation, suggesting that current approaches are insufficient for mitigating potential harms posed by increasingly capable AI systems.