>>Bias, Transparency, and Accountability of AI (Αγγλικά)

By Giorgos Bouritsas & Yannis Panagakis

Artificial intelligence (AI) is no longer something distant or futuristic. It has become embedded in everyday life: on our phones, in the services we use, in public administration, and in decisions about work and benefits. Many choices that affect us are now mediated by systems we never see.

Yet, most people do not understand how these systems work or whether they treat us fairly. This is not merely a technical issue. It is a question that touches our dignity, our rights, and the quality of our democracy.

Consider three people using AI systems:

• A recent university graduate scrolls through rental listings in Athens, wondering why they only see certain neighbourhoods and price ranges. The platform's algorithm has already determined what to show them based on their search history, estimated income, and browsing patterns, invisibly filtering opportunities before they even know those opportunities exist.
• Someone in their thirties applies for a small business loan at their bank. The decision takes seconds. An algorithm has calculated credit risk based on historical lending data, which might encode decades-old assumptions about which neighbourhoods are "reliable," which professions are "stable," or which types of businesses represent "safe bets."
• A professional in their fifties receives a notice that their tax file has been selected for a detailed audit. They do not know that an automated system flagged them based on patterns and correlations: perhaps their profession, their region, or some combination of factors that the system identified as "suspicious" by learning from past audit data.

Algorithmic Bias and Algorithmic Fairness

In each case, the systems appear neutral and efficient. In reality, though, if these algorithms were trained on data shaped by past biases, inequalities, and human assumptions, they may be replicating those same patterns with mathematical precision. The discrimination is no longer intentional, but it becomes more efficient, more consistent and, perhaps more importantly, harder to challenge.

This is the problem of algorithmic bias. Researchers have documented cases where facial recognition systems misidentify darker-skinned women at error rates up to 34% higher than for lighter-skinned men [1]. Investigative journalists examining a widely used criminal risk assessment tool found it was twice as likely to incorrectly flag Black defendants as high-risk compared to white defendants [2]. Hiring systems have been shown to screen out qualified candidates based on patterns that encode historical prejudices [3].

AI systems are (usually) not designed with a discriminatory intent, while in certain contexts, well-designed algorithms can be more consistent and less susceptible to individual prejudices than human decision-makers [4].   Moreover, what constitutes fairness in algorithmic systems remains contested. A system can be designed to treat everyone identically (formal equality), to produce equal outcomes across groups, e.g. male and female (statistical parity), or to ensure equal accuracy rates across groups (equalised odds), yet these different definitions of fairness can be mathematically incompatible [5]. Choosing among them requires value judgments about justice, not merely technical optimisation. 

Our position:

• The question is not whether to use algorithms, but how to ensure they are designed and deployed responsibly and under which conditions machine predictions are safe to follow.
• Deciding the above is fundamentally a political and ethical procedure, not merely a technical one.

Transparency

The problem is not only that algorithms can be biased. It is also that these systems are often extremely opaque. When someone is rejected, flagged, or filtered by an algorithm, there is often no one they can ask for an explanation. They cannot appeal to fairness, argue their case, or clarify misunderstandings, even though such activities remain central to meaningful participation in social and civic life.

Real transparency requires several elements. 

• First, information about the data [15]: where it came from, what populations it represents, how it was collected and preprocessed. If an algorithm was trained on lending data from banks with historically discriminatory practices, that matters.
• Second, access to the system's design [15]: understanding the model architecture, what variables it considers and how it makes decisions.
• Third, and crucially, interpretability: the right to an explanation about why a specific decision was made. Not simply "the algorithm decided," but concrete reasons one can understand and potentially challenge [6].

Transparency also means knowing where these systems are reliable and where they fail. Every algorithm has limits: situations where it functions well and situations where it does not. Some systems perform accurately with common cases but fail with unusual ones. Some work well for certain demographic groups but poorly for others. This information (the stress testing, failure cases, and contexts where the model is safe to use) should be available, but unfortunately, this is usually not the case. We also rarely know which parts of a process are fully automated, which are AI-assisted, and which still involve human judgment. This full disclosure matters: it is one thing if an algorithm provides a recommendation that a human reviews; it is quite another if the algorithm makes the final decision. People affected by these systems have a right to know [7].

However, transparency faces real tensions. Companies argue that full disclosure of algorithmic systems could reveal proprietary methods, enable gaming of the system, or expose security vulnerabilities. These concerns are not trivial, but they must be balanced against the public interest in accountability and the individual right to understand decisions that affect one's life [8]. 

Our position:

• Algorithmic transparency standards need to be set by regulators and policymakers taking into account all the aforementioned axes (data, system design, interpretability, stress-testing, disclosure of AI use).
• Designing transparency requirements is also a political, rather than a merely technical decision, and policymakers should take into account that protecting fundamental rights through transparency might clash with commercial interests.

Accountability

AI systems’ opacity additionally makes responsibility difficult to assign. If an algorithm makes a harmful or incorrect decision, who bears responsibility? When rights are violated, when biases cause harm, when privacy is breached, who can be held to account? Who has the authority and obligation to address the problem?

Currently, in many cases, the answer remains unclear. Responsibility is distributed across developers, companies, procurement officers, and end users. There exist few clear professional standards, nothing comparable to the codes of conduct that govern engineers or physicians. Further regulation is needed for professional responsibility: defining who is accountable for algorithmic decisions, who must answer for failure cases, and who can be held liable when rights are violated [9]; just as civil engineers bear personal accountability for bridge safety, those who design, deploy, and operate AI systems that affect human lives should bear clear professional obligations.

Our position:

• Transparency and accountability are interlinked. Transparency enables algorithmic audits. Audits can detect failures and violations. Clear accountability regulations then allow enforcement. Systems can be modified or withdrawn, and those responsible can be held to account.

AI Auditing

AI audits constitute systematic examinations of algorithmic systems, investigating whether these systems respect fundamental rights and operate safely; they are the key to enabling accountability.

Different types of audits serve different purposes: bias audits test for discrimination, regulatory inspections assess legal compliance, risk assessments identify problems before deployment, and impact evaluations examine real-world effects [11]. In particular,

• Violations of rights. Audits must assess whether systems violate the right to non-discrimination. Do algorithms produce different outcomes for different groups based on protected characteristics such as gender, ethnicity, or age? Researchers examine whether systems treat groups differently in ways that cannot be justified [6]. Audits also assess privacy rights: Does the system comply with regulations such as GDPR? Is sensitive data properly protected? Is data being used only for its original stated purpose [7]?
• Safety and broader impacts. Beyond individual rights, audits must examine societal consequences. How does the system affect communities, labour markets, and social structures? Does it reinforce existing inequalities or create new ones? Who benefits and who is harmed? Additionally, as AI systems grow more powerful and widespread, their environmental costs (the energy required for training and operation) have become matters of public concern [14].

Just as we conduct safety inspections for buildings or environmental assessments for construction projects, we require systematic examinations of AI systems that affect public welfare. But these audits remain meaningless without enforcement power, and enforcement requires clear accountability.

Addressing these challenges requires more than regulation or technical improvement. It demands public understanding and participation. Citizens need a basic understanding of how AI functions so they can recognise when digital systems are shaping their opportunities and rights. Policymakers need to understand that regulating AI is not only about innovation. It is about ensuring that digital infrastructures respect human rights and democratic values. And developers need professional standards of accountability, comparable to those in engineering or medicine.

 References

[1] Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 1–15.

[2] Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias. ProPublica, May 23, 2016.

[3] Dastin, J. (2018). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters, October 10, 2018.

[4] Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018). Human decisions and machine predictions. The Quarterly Journal of Economics, 133(1), 237–293.

[5] Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017). Inherent trade-offs in the fair determination of risk scores. Proceedings of the 8th Innovations in Theoretical Computer Science Conference (ITCS '17), Article 43, 1–23.

[6] Spanish Data Protection Agency & Eticas Research and Consulting. (2021). Guide to Algorithmic Auditing. Available at: https://eticas.ai/wp-content/uploads/2024/04/Guide-to-Algorithmic-Auditing-EN.pdf

[7] European Parliament and Council. (2016). General Data Protection Regulation (GDPR), Article 13–15.

[8] Wachter, S., Mittelstadt, B., & Russell, C. (2021). Why fairness cannot be automated: Bridging the gap between EU non-discrimination law and AI. Computer Law & Security Review, 41, 105567.

[9] Matthias, A. (2004). The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology, 6(3), 175–183.

[10] Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* '20), 33–44.

[11] Ada Lovelace Institute & DataKind UK. (2020). Examining the Black Box: Tools for Assessing Algorithmic Systems. Available at: https://www.adalovelaceinstitute.org/report/examining-the-black-box-tools-for-assessing-algorithmic-systems/

[14] Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3645–3650.

[15] Mitchell M., Wu S., Zaldivar A., Barnes P., Vasserman L., Hutchinson B., Spitzer E., Raji I.D., Gebru T. (2019). Model cards for model reporting. Proceedings of the Conference on Fairness, Accountability and Transparency (pp. 220-229).