The International Association of Universities (IAU) has asked me to write a short perspective on Generative Artificial Intelligence, which I have been allowed to also post below. It has been published in May 2024 in the IAU’s magazine IAU Horizons Vol. 29 (1), also available in this pdf (scroll to page 28).

Picture: Rick Payne and team / Better Images of AI / Ai is… Banner / CC-BY 4.0

Ban, use, or cite Generative AI?

Should universities ban the use of generative AI (GenAI) in written works or, on the contrary, teach how to integrate it into learning practices? Extractive data practices of many available GenAI platforms support the first stance, whereas the general hype around AI and widespread access may favor the second one. However, neither position does justice to the university’s epistemic mission in teaching. Instead of focusing on banning or imposing new information technologies, universities should more than ever strive to provide the conditions within which humans can learn.

Digital transformation

The narrative of AI as a revolutionary force overlooks the foundational role of digitization and connectivity, with the Internet and web technologies pioneering the changes we now attribute to AI. These earlier innovations have profoundly impacted how information is accessed, consumed, created, and distributed. They have been used by our students from early on: From Google searches about topics or the spelling of words to reading Wikipedia articles, from sharing course notes online to asking for homework help in Internet forums, the university learning experience has already been changing long before the arrival of GenAI. At the same time, students’ learning experience has always included taking responsibility for their work, no matter how it was created.

Common misconceptions

Internet and web technologies have also facilitated unprecedented digital data generation and accumulation that have served to create current GenAI models. Today, few would advocate for a complete ban of access to web search or Wikipedia at universities. I find it therefore curious to see how GenAI starts such conversations anew. Why? Because GenAI is neither source nor author. Attributing human-like thinking or consciousness to it is misleading. GenAI does not provide knowledge. It is a powerful computational tool that generates output based on previous data, parameters and probabilities. These outputs can be used by humans for inspiration, modification, copy-paste, or simply be ignored.

At our university, students do not need to reference the use of thesauri, on- and offline dictionaries, writing correction software, or conversations with others about the topic in their writing. I am not fond of the idea of generically referencing the use of GenAI. Ascribing GenAI the status of a source or author to be cited is a profound mischaracterization of how the technology works and further reiterates the AI hype narrative. Moreover, it may wrongly incentivize students to view GenAI output similarly to other types of sources we already ask them to cite. But because GenAI generates individualized output with each request, hence its name, such output cannot be traced back or reproduced in the future. I fail to see what would be gained by citing it, unless it is for specific educational purposes.

Ethical challenges

Should the use of GenAI be encouraged, then? If it is such a powerful computational tool, harnessing its benefits within universities seems not only justified but necessary? However, as ever so often, it is complicated. Thanks to scholars in the humanities and social sciences, as well as activists and journalists, we know better than to uncritically endorse any of these platforms. There are valid points of criticism that can, and should, be brought up against GenAI platforms, such as illegal data acquisition strategies, veiled data labor, lack of basic testing and missing ethical guardrails, dubious business motives, lack of inclusive governance and harmful environmental impact.

Comprehension beyond the hype

What we cannot do is ignore the existence of GenAI platforms easily accessible to our students. In an article for The Guardian, the eminent media scholar Siva Vaidhyanathan warned us in May 2023 already that we might be “committing two grave errors at the same time. We are hiding from and eluding artificial intelligence because it seems too mysterious and complicated, rendering the current, harmful uses of it invisible and undiscussed.” GenAI, its output, and its implications need to be understood in all fields and contexts. This encompasses not only grasping the technical aspects of these technologies but also critically analyzing their social, political, and cultural dimensions. Our goal should thus be to cultivate a safe, positive learning environment that stimulates critical thinking. Ideally, universities foster the necessary skills that allow students to evaluate information and build on existing knowledge to make informed decisions outside of any hype discourse. Such skills will not become less relevant in times of abundant GenAI content but rather more.

Aufgrund hoher Medienresonanz wurde ich als Forscherin am Alexander von Humboldt Institut für Internet und Gesellschaft (HIIG) am Mittwoch Abend um eine kurze Einschätzung gebeten, welche ich hier in voller Länge wiedergebe:

Der offene Brief des Future of Life Instituts ist Augenwischerei: Er beschreibt eine Phantasiewelt, in der bisherige KI bis auf ein paar technische Updates problemfrei ist und sechs Monate Entwicklungsstopp genügen, um geeignete regulatorische Rahmenbedingungen für die angeblich unausweichliche Superintelligenz zu schaffen.

So werden die Probleme von heute existierenden Systemen nicht nur ignoriert, sondern verharmlost. Das Superintelligenz-Thema gehört in Philosophieseminare und nicht in die Politik.

Auch ist der alleinige Fokus des Briefs auf die Entwicklungsstufe von KI sehr kurzsichtig. Seltsamerweise wird keine Einschränkung beim Einsatz gefordert, obwohl bei KI der Kontext der Anwendung mindestens genauso wichtig ist wie die Entstehung.

Schon fast zynisch mutet die Erwähnung von grundsätzlich sinnvollen Massnahmen wie Audits oder Kennzeichnung an, wenn sich unter den Erstunterzeichner Entscheidungsträger finden, die solche selbst nie eingeführt haben.

Und obwohl im Brief richtigerweise steht, dass wichtige Entscheidungen nicht ungewählten Tech Leadern überlassen werden sollten, erreicht die Publikation nun genau das Gegenteil: Agenda Setting durch Tech Leader.

Honi soit qui mal y pense.

Einige meiner Worte fanden tatsächlich am Freitag ihren Weg in die TAZ, in den Tagesspiegel KI & Digitalisierung, sowie in die Berichterstattung auf Netzpolitik.org.

Ebenfalls am Freitag publizierten übrigens auch die Autorinnen des wegweisenden KI-Artikels “Stochastic parrots” ein Statement, das den offenen Brief des FLI ebenfalls kritisiert und zudem fundiert erklärt, warum die hypothetische Schwarzmalerei von zukünftiger machtvoller KI schädlich ist.

***

1. Blake Lemoine meint im Chatbot von Google ein eigenes Bewusstsein erkannt zu haben. Wäre es alleine aus technischer Sicht überhaupt möglich einer Maschine ein eigenes Bewusstsein und eigene Handlungsfähigkeit zu “geben”?

Ein Bewusstsein wie Menschen kann eine Maschine nie entwickeln. Sie kann natürlich so programmiert werden, dass sie menschliches Sprechen verblüffend ähnlich imitiert. Aber es ist gewollte Imitation, und auch der Datenraum wurde von Menschen definiert. Maschinen handeln nicht autonom. Zwar ist im Gegensatz zu anderen Automaten die sogenannte Künstlicher Intelligenz nicht von A bis Z durchprogrammiert, aber bei der Entwicklung wird ein Lernprozess oder ein Ziel festgelegt.

2. Warum versuchen ForscherInnen Künstliche Intelligenz zu entwickeln, die so menschlich wie möglich ist? Was für einen Vorteil verspricht man sich daraus?

Menschen entwickeln und benutzen seit Urzeiten technische Hilfsmittel zu ihrem Vorteil. Bei sogenannter Künstlicher Intelligenz handelt es sich um Softwareverfahren mit hochentwickelten Datenverarbeitungsmöglichkeiten. Der direkte Vergleich zum Menschen ist jedoch absurd. Wir machen ja auch keine Wettläufe gegen Rennautos oder bewerten Atomkraft wie Muskelkraft. Irreführende Begriffe wie “Intelligenz” verfälschen leider die Wahrnehmung der Handlungsfähigkeit und damit der Verantwortung.

3. Manche ExpertInnen sehe eine Zukunft voraus, in der ein kleiner Teil der Menschen am Computer sitzt und programmiert, während der Rest keine Arbeit mehr hat, weil Maschinen, Roboter und KI effektiver als Menschen geworden sind und alle Aufgaben erledigen. Sehen Sie ein solches Szenario als realistisch an?

Nein, denn zum einen können Software und Roboter nicht alle Tätigkeiten ersetzen, die von Menschen erbracht werden. Zudem unterschätzt ein solches Szenario den Aufwand, der für gut funktionierende Systeme erbracht werden muss. Programmieren ist nur ein Teil davon. Es braucht auch Hardware, Installation, Unterhalt und Wartung. Daten müssen gesammelt, erstellt oder aufbereitet werden, was heute schon oft unsichtbar von Menschen hinter den Kulissen gemacht wird.

4. Welche ethischen Grundsätze sollten bei der Entwicklung von Künstlicher Intelligenz niemals verletzt werden?

Dazu gibt es zahlreiche Vorschläge, aber keine Pauschalantwort, auch weil der Begriff Künstliche Intelligenz mehrere Technologien umfasst, die in den verschiedenen Gebieten unterschiedliche Auswirkungen haben können. Die Diskussion um solche ethischen Grundsätze ist aber zentral, insbesondere mit einem Fokus auf mögliche Anwendungen. Eine wichtige Frage stellt sich übrigens sogar noch vor der Entwicklung, nämlich jene der Verhältnismässigkeit. Eine KI-Lösung benötigt Zeit, Ressourcen, und nicht wenig Energie. Manchmal kann ein Problem oder eine Aufgabe auf alternative Weise zielbringender gelöst werden.

When I was asked to provide ten such links about the ethics of artificial intelligence, I was delighted to do so. Delighted… but also challenged. Because it is impossible to do justice [pun intended] to “AI ethics” via 10 websites.

Therefore, my selection is not meant to be exhaustive but to be just that: a selection. Its aim is to provide a rich overview of crucial issues, perspectives and criticisms in the field of “AI ethics” to people who have not already been studying them. I invite you to not simply evaluate each article linked in isolation, but to see my selection as a collection that derives value from being more than the sum of its parts. I have come up with a list of links where each one contributes a different piece to the puzzle, which should be even more insighful if read in the suggested order.

To get a thorny issue out of the way: I have not been paid by Refind for this work (though I was upgraded to a Premium membership upon asking) and have agreed to participate based on two reasons: 1) Contingent on my availability I am happy to support (Swiss) start-ups whose mission and values I share… and David Bauer, who is in charge of product and content strategy at Refind, contacted me at a time when my calendar pretended to be able to accommodate such requests (thank you for your patience, David). 2) The selection of websites I would put together for the “Ethics of Artificial Intelligence Deep Dive” on Refind could also be shared on my blog. This means that you do not have to join Refind to access my link selection. (Then again, I find the way Refind works very compelling myself and have no problem recommending it to others.)

Before you sign up (for free) on Refind or read on (below, also for free), you might like to know that my collection has been categorized as being at “intermediate level”. The average reading time per article is estimated at 8 minutes. To keep things interesting, I have included various formats: 2 academic articles (yes, you read that right: only 2!), 1 guide, 1 interview, 1 collection of perspectives and positions, 2 long-form media essays, 3 blog articles, and 1 other website.

If you kept track, you will have realized that I cheated: I included 11 articles, not 10. When I started to list AI ethics articles that I thought essential, I started out with several dozens of references from my Zotero library. Then I eliminated, shuffled, and eliminated again until I reached a point where I could not bring myself to eliminate any more links. So eleven it is.

Eleven links. Only eleven links. About the ethics of artificial intelligence. An impossible choice, and chances are that you disagree with my selection. Therefore, if you want to recommend any articles and websites that you feel are unduly left out of the collection, please add your suggestions in the comments. And before you scold me too harshly, please take note that I eliminated my own writing first. Indeed, my Refind collection does not include

• The global landscape of AI ethics guidelines (Nature Machine Intelligence)
• Ethics guidelines galore for AI – so now what? (ETH Zurich Zukunftsblog),
• Ethical Artificial Intelligence — on principles and processes (Digital Transformation and Ethics),
• Towards an inclusive future in AI. A global participatory process (foraus report),
• Why Dr. Timnit Gebru is important for all of us (The Startup)
• or even the most recent AI reflections in 2020 (Nature Machine Intelligence).

Why? Because I figured that if you did not know my work, you would end up checking out my profile anyway after (hopefully) finding my recommendations pertinent… and if you were interested in my recommendations because you already knew my work anyway, there was no need to “waste” a precious spot in my link collection.

So this is the context. Without further ado, below are the eleven articles about the ethics of artificial intelligence I recommend you read in the suggested order.

***

In lieu of an introduction: 1 link that will provide you with an overview of many issues and viewpoints

• Hey Google, sorry you lost your ethics council, so we made one for you

  by Bobbie Johnson and Gideon Lichfield, MIT Technology Review
  April 6, 2019

Instead of a general introduction, I suggest to jump into the cold water via a prominent case study to get an overall idea about the breadth of perspectives. Following Google’s AI failed initiative to establish an Advanced Technology External Advisory Council (ATEAC), the MIT Technology Review asked several experts about what Google should be doing next. The answers provides an overview of many of the perspectives and ethical issues that are at stake in discussions about ethics and artificial intelligence. What is more, this case study also deserves renewed focus based on recent events.

A guide

• The Hitchhiker’s Guide to AI Ethics

  by B Nalini, Towards Data Science
  May 1, 2019

This is an overview in a more classic format: a guide. It is an excellent one at that, providing numerous helpful links and resources I encourage you to check out. Moreover, this is only part 1 out of 3, and it goes without saying that part 2 and part 3 are just as great. So in a sense, this is the link recommendation that allowes me to bypass the initial constraint of ten (or eleven) links by simply also recommending you to check out the links in this guide.

Specific focus points, providing a more in-depth perspective on particular issues

• AI Ethics: Seven Traps

  by Annette Zimmermann and Bendert Zevenbergen, Freedom to Tinker (Princeton’s Center for Information Technology Policy)
  March 25, 2019

Perhaps just as interesting as the ethics of artificial intelligence is the question of what is not. This article explains how not to do AI ethics via seven “traps”, seven perspectives that betray the very idea of AI ethics.

• Four reasons why hyping AI is an ethical problem

  by Dorothea Baur
  November 21, 2020

Is “AI ethics” a hype? Perhaps — but in that case, then so is artificial intelligence. Because one could argue that we are only talking about the importance of ethics for AI because of the importance of AI itself… This short article demonstrates very clearly why hyping AI is an ethical issue in and of itself.

• Don’t ask if artificial intelligence is good or fair, ask how it shifts power

  by Pratyusha Kalluri, Nature
  July 7, 2020

The subtitle of this article is: “Those who could be exploited by AI should be shaping its projects.” It centers on a crucial high-level issue of AI ethics: power. I already told Nature Machine Intelligence why I think this piece is so significant:

Although the illusion that ‘ethical AI’ is simply a technological matter still lingers, 2020 has seen an important push towards broader acceptance of the sociotechnicity of AI. Acknowledging the sociotechnical nature of AI systems requires us, as Pratyusha Kalluri put it succinctly, to centre less on fairness, or on ‘AI for good’, and more on power distribution and power differentials.

More on power, or: AI is not created, deployed, and used in a void

• Ex-Googler Meredith Whittaker on Political Power in Tech, the Flaws of ‘The Social Dilemma,’ and More

  by Alex Kantrowitz
  October 22, 2020

Because “ethical AI” is not simply a technical issue to be fixed, it is important to understand the context in which decisions about artificial intelligence are being made. This interview provides insights into one crucial such context: Silicon Valley. (Bonus point: It explains the flaws of the netflix movie The Social Dilemma — which is an important detail in the context of AI ethics in view of the movie’s success, lest The Social Dilemma and its protagonists are taken at face value. Because they definitely shouldn’t.)

• The Algorithmic Colonization of Africa

  by Abeba Birhane
  July 18, 2019

Speaking of where decisions about artificial intelligence are made : This essay presents a much needed focus on whose values are, or are not, embedded in AI systems. I remember reading it for the first time in summer 2019. A few months earlier, we had wrapped up our analysis of AI ethics guidelines, which demonstrated an over-representation of more economically developed countries in the AI ethics debate via published guidelines. I was struck by this in-depth exploration, which shows just how much the current AI landscape is both rooted in and perpetuating specific perspectives at the expense of others. However, a technology of global importance should be shaped globally, and not impose one set of values unilaterally.

• Giving by Taking Away: Big Tech, Data Colonialism, and the Reconfiguration of Social Good

  by João Carlos Magalhães and Nick Couldry
  2021

Can AI be made ethical, and if yes: how? Simply aiming at designing, implementing and using it “for social good” is not enough. Indeed, “AI for (Social) Good” is no panacea, and there are several good pieces criticizing aspects of AI4good (or AI4SG). I chose this one because it goes beyond questioning the effectivenes of such initiatives: it demonstrates in a well-researched study why and how “AI for Social Good” can even end up doing more harm than good. Click here to get directly to the pdf. (Full disclosure, although I was not aware of this when I picked this article: one co-author has since become my colleague at the HIIG.)

Critiques of AI ethics

• The Invention of “Ethical AI”: How Big Tech Manipulates Academia to Avoid Regulation

  by Rodrigo Ochigame
  December 20, 2019

Any deep dive into the ethics of artificial intelligence worth its name must reflect critically on its own enterprise. This article addresses the topic of “ethics-washing“, in particular: the framing and agenda-setting  in the domain of “Ethical AI” by Big Tech.

• The moral collapse of AI ethics

  by J. Khadijah Abdurahman
  Dec 7, 2020

This is a recent, astute evaluation of failing AI ethics initiatives and, thus, of the failing of AI ethics itself. It is both brilliant and brutal, and adds a necessary perspective to any discussion of ethics of artificial intelligence.

Last but not least

• Anatomy of an AI system

  by Kate Crawford and Vladan Joler
  2018

This is not a link to a typical “AI ethics” contribution, and yet I can think of no better piece to finish this selection on ethics in artificial intelligence with, except perhaps the forthcoming book Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. It masterfully surfaces and interweaves the material, economic, social, and symbolic dimensions of one particular AI system. Ethical issues everywhere.

Après plusieurs prises et montage, j’ai fini par avoir mes 15 secondes de gloire au téléjournal de 19h30, ce qui est évidemment trop peu pour rendre compte de la complexité des enjeux. C’est pourquoi je partage ci-dessous quelques paragraphes issus de mes notes.

Dans mes activités de recherche j’ai rencontré trois problématiques principales qui me semblent importantes par rapport à ce sujet:

La gouvernance privée d’un bien public

Premièrement, il faut reconnnaître que la supression d’une émission journalistique par une plateforme en ligne n’est pas qu’un enjeu juridique. Sous le seul angle légal, il semble logique qu’une entreprise privée puisse agir d’après ses propres termes et conditions. Cependant, l’enjeu est égalemenet social, culturel, et même démocratique. Car la déliberation est un élément constitutif du fonctionnement démocratique, et les bonnes conditions-cadres de la déliberations sont donc un bien public. Or, nos échanges d’information, la communication, le journalisme, nos interactions sociales etc. se passent de plus en plus également en ligne. Pour ce faire, nous sommes tributaires d’intermédiaires, et beaucoup d’entre eux sont des entreprises privée avec comme seul objectif d’être lucratives. Elles fonctionnent donc un peu comme un espace public, ou semi-public, elles font circuler l’information, ou servent comme source d’informations, à l’instar du journalisme “traditionnel” — alors qu’elles n’ont aucune responsabilité publique, ou un modèle de gouvernance à la hauteur de leur importance dans nos vies.

La gouvernance dans un monde globalisé

Deuxièmement, il ne faut pas oublier que ces entreprises ont un siège principal précis, d’où elles agissent à titre international. Et même si elles ont des représentants en dehors de la Sillicon Valley, la manière dont fonctionnent ces plateformes à la base, et les décisions clés, émanent d’un contexte qui est loin des contextes locaux dans lesquels se déroulent nos vies. Il ne surprend donc pas que la gouvernance de ces plateformes ne reflèt pas forcément nos valeurs locales, mais importe plutôt une vision sociale et culturelle, et même légale, venue d’ailleurs. Elle s’oriente, notamment, considérablement au contexte américain, bien qu’il y ait eu des efforts de tenir compte des lois nationales, notamment en ce qui concerne le domaine de la publicité.

Mais voilà: le contexte local n’est pas pris en compte dans l’ensemble de ses spécificités. Il se voit trop souvent réduit à un problème juridique, auquel il suffit de trouver la solution technologique, alors que la réalité est plus complexe.

La gouvernance algorithmique

Troisièmement, la gouvernance des plateformes se fait en grande partie de manière automatisée. D’un côté cla se comprend en vue de l’échelle. D’un autre côté, à chaque fois que la vie humaine est soumise à des règles algorithmiques, on écrase les nuances. Des règles trop strictes sont incompatibles avec les nombreuses ambiguitës que nous rencontrons tous les jours, c’est pourquoi même notre idéal de la jurisprudence vise à prendre en compte les circonstances précises de chaque cas.

Qui plus est, les règles qui se retrouvent inscrites dans ces systèmes algorithmiques on été établies de manière unilatérale et non pas par un système démocratique qui aspire au bien commun.

Pour aller plus loin

Dans le domaine de la recherche, beaucoup de spécialistes se préoccuppent de ces questions depuis longtemps, et de manière approfondie. En anglais, une excellente liste compilée par Tarleton Gillespie(*) du Social Media Collective de Microsoft Research, par exemple, rassemble un grand nombre de livres et d’articles au sujet de la modération des contenus en ligne que je vous conseille vivement.

(*) Et bien qu’il ne se trouve pas sur la liste sus-mentionnée, pour comprendre le contexte actuel de la gouvernance (par) des technologies numériques je vous recommande également le premier livre de l’auteur de la liste: Wired Shut.

***

L’effet Google

Entretien avec Anna Jobin, sociologue du numérique, qui travaille notamment sur nos interactions avec les algorithmes et les dimensions sociales de l’intelligence artificielle. Propos recueillis par Stéphanie Pache.

Peux-tu décrire ce qu’est un moteur de recherche et pourquoi tout un chacun devrait s’y intéresser?

Un moteur de recherche nous assiste à localiser l’endroit où se trouve l’information que nous cherchons. En Suisse, et dans la plupart des pays européens, une majorité de personnes qui cherchent de l’information en ligne utilisent Google pour ce faire. Depuis sa conception il y a plus que vingt ans, le nom de l’entreprise a fini par servir de verbe, “googler”, pour désigner le fait même de faire des recherches en ligne – un reflet de sa place dans notre quotidien. Dans le contexte de l’explosion d’informations se trouvant en ligne dans une forme numérique ou numérisée, Google a réussi à nous rendre un immense service qui est celui de nous diriger vers des sources souvent légitimes qui répondent plus ou moins à ce que nous cherchions.

Deux grands aspects restent problématiques: premièrement, le modèle d’affaire. En effet, Google est une entreprise à but lucratif, donc nos recherches d’informations nourissent également un système publicitaire complexe, qui a rapporté plus que 100 milliards de dollars en 2018.

Deuxièmement, la position de l’entreprise et le manque d’alternatives. Il est impossible de surestimer le pouvoir actuel de Google (et, en occurrence, de quelques autres entreprises du numérique). La multinationale dispose désormais d’énormément de données, de puissance de calcul, d’argent pour racheter d’autres entreprises, et de connexions avec le monde politique. En plus, ses différents services (cartographie, webmail, navigateur web, système d’exploitation pour n’en nommer que quelques-uns) créent des synergies importantes qui ont des effets secondaires considérables.

Comment ce système contribue à la reproduction des inégalités sociales?

Dans le cas de Google Search, il y a au moins trois dimensions qui entrent en compte. D’abord, il y a le niveau de la production de contenus en ligne, car le moteur de recherche ne peut indexer et lister que ce qui existe. Toutes les inégalités reflétées dans et véhiculées par les contenus web se retrouvent alors reproduites. Ainsi, pour donner deux exemples simples, une recherche image du terme “PDG” vous montrera avant tout des images d’hommes blancs, et la recherche image “couple” résultera en une majorité de photos de couples hétérosexuels composés de personnes cis-genres et tout aussi blanches.

Ensuite, il y a une canalisation des usages basée sur les actions passées des personnes utilisant le moteur de recherche: par exemple, l’auto-remplissage, c’est-à-dire les propositions de recherche, s’appuient sur l’ensemble des recherches ayant été effectuées auparavant et tendent donc à reproduire des stéréotypes existants comme propositions de recherche désirables.

Finalement, la programmation et le design du moteur de recherche même agissent sur notre vie et la manière dont nous appréhendons le monde. Non seulement des décisions prises par des humains qui ont une influence, mais aussi des logiques algorithmiques. Un système d’identification de contenus web basé sur des mots-clés risque, par exemple, de perpétuer une stratification de l’accès à l’information en fonction du vocabulaire d’une personne. De plus, les algorithmes de Google favorisent les sites déjà bien connus, qui reçoivent encore davantage d’attention (et inversement).

Quelles actions peut-on imaginer contre ces effets et leur production?

De manière générale il faut davantage de diversité et créer des possibilités pour de vraies alternatives. Cela concerne Google tout autant que notre société, car les deux se nourrissent mutuellement. Davantage de diversité parmi les gens qui créent les outils que nous utilisons au quotidien, dans la production de contenus web, et là où les décisions sont prises. L’appel à de vraies alternatives quant à lui suggère une remise en question de notre rapport à l’information en ligne et de sa gouvernance.

Nos sociétés ont, il y a longtemps, reconnu l’accès à l’information comme valeur fondamental, et le meilleur exemple en sont nos bibliothèques publiques. Aujourd’hui, nous devons repenser comment nous souhaitons façonner non seulement l’accès, mais aussi la participation à la production et à la circulation des informations.

***

Cet entretien a paru en automne 2019 dans la publication Pages de gauche, no 173. Le numéro était dédié au sujet “Toute technique est politique” et est téléchargeable sur le site de la publication.

***

Dr. Timnit Gebru and I do not know each other. I have never met her and she probably is not even aware I exist. And yet, she is very important in my life. Not only in my life, but in your life, too. Indeed, I believe Dr. Gebru is extremely significant for everyone and I will explain why below.

Before that, let me recap briefly what happened last week that inspired me to write this, in case you have never heard of Dr. Gebru before. (It is very well possible that you have not — as I may not have if I weren’t researching the social aspects of algorithmic systems for a living.) However, you certainly know Dr. Gebru’s former employer: Google. You may even have heard of her area of work: at Google, Dr. Gebru used to be the co-lead of the Ethical AI team. And maybe, just maybe, you have even heard of her ground-breaking research (with Joy Buolamwini) demonstrating how face recognition algorithms are most accurate for faces that are male and caucasian and perform terribly for people of color, especially women. _{[Dr. Gebru’s record goes far beyond this one example I picked. Yet it is the one I have heard and seen mentioned most frequently as the go-to example in industry, policy and public discussions of “algorithms are not neutral”.]}

On the morning of December 3, I checked my twitter feed and discovered Timnit Gebru’s tweet, published a few hours prior, announcing her discovery that Google had fired her. She wrote that her employment had been terminated immediately with reference to an email to an internal mailing list.

The news spread like a wild fire and, within hours, disputes over details and definitions broke out. What exactly did her email to the mailing list say? Was this the real reason she was fired by Google? Was she actually fired or did she resign? And even if she was fired: did she deserve to be fired? But here’s the thing: although some of the answers to these questions are informative (and even the questions themselves are very telling!), I will not discuss internal e-mails and information about individual people at Google here. For the point I am trying to make, none of them really matter.

***

Google is still most well-known for the web search engine at the root of its success. But the company is much more than Google Search. Although there is a tendency to underestimate it, it is actually impossible to overstate the power Google/Alphabet has in Europe and North America (and beyond). Arguably, the company has more data on most individuals, industries and societies than any other corporation. Google shapes how people create, circulate, seek out, “consume” and treat information and it is able to identify when and how people sleep, eat, work, travel, socialize, learn, read and write. The company can leverage this knowledge in all domains of our lives for its profit and purposes. Because Google is much more than web search…

There are of course Google Drive, Google Maps, Google Earth, Gmail, G Suite and the other online services, including Google’s advertising infrastructure. But there is YouTube. And the web browser Chrome. And the most popular mobile operating system Android. And consumer electronics such as smartphones (Pixel), laptops (Chromebook) and smart assistants and devices (Google Home, Nest). And Google Pay. And the so-called “other bets.” They were split off as separate entities when, in 2015, Google became your regular international corporation by restructuring its corporate assets and founding Alphabet in the process. These other bets remain nonetheless relevant when talking about Google, notably the transportation venture Waymo, Calico (a biotech company focused on longevity), CapitalG and Google Ventures (private equity and venture capital funds), X Development (an R&D organization), Jigsaw (a technology incubator), Google Access&Energy, and the life science venture Verily, making big forays into the health sector.

Add Google’s financial muscle to the mix — the company reported a 3rd quarter revenue record of 46.02 billion U.S. dollar in 2020 and has currently reached its all-time high of 1.23 trillion(!) U.S. dollar in market valuation — as well as a trove of lobbyists and policy-shapers close to governments and within consultancies, and you may start to realize why I wrote that Google’s power cannot be understated.

***

Artificial intelligence (AI) is central to many products and services of the contemporary information technology industry. When the company announced their AI principle on June 7, 2018, CEO Sundar Pichai wrote:

We recognize that such powerful technology raises equally powerful questions about its use. How AI is developed and used will have a significant impact on society for many years to come. As a leader in AI, we feel a deep responsibility to get this right. So today, we’re announcing seven principles to guide our work going forward. These are not theoretical concepts; they are concrete standards that will actively govern our research and product development and will impact our business decisions.

We acknowledge that this area is dynamic and evolving, and we will approach our work with humility, a commitment to internal and external engagement, and a willingness to adapt our approach as we learn over time.

(from https://blog.google/technology/ai/ai-principles/, emphases mine)

Google’s seven principles for AI were widely reported on by media outlets, although the company was neither the only organization to publish ethics principles for AI nor the first. _{[In fact, at the end of the decade there had been so many publications of AI ethics principles that it became hard to see the forest for the trees. Therefore, at my former workplace (the Health Ethics & Policy Lab at ETH Zurich), we decided to provide an overview and analysis of such publications.]}

These ethical principles are not even Google’s only initiatives in the domained called “AI ethics”. Browsing around the company’s website you may stumble upon suggestions and documentation of “responsible AI practices“. You can find research papers, tools, datasets, links to initiatives aimed at fostering a diverse workforce, and even novel approaches such as model cards.

Most prominent was also Google’s creation of ATEAC: the Advanced Technology External Advisory Council. The ATEAC has colloquially been called Google’s “AI ethics council” and was announced in spring of 2019 with the stated aim of “consider[ing] some of Google’s most complex challenges that arise under our AI Principles, like facial recognition and fairness in machine learning, providing diverse perspectives to inform our work”.

The fact that Google has already had an internal “Ethical AI” team since 2017, founded by Meg Mitchell, is not widely known outside the industry. Until last week, Dr. Timnit Gebru was the technical co-lead of the Ethical AI team.

***

Ethical is not the same as legal: not everything that is legal is also ethical, and not everything that is illegal is also unethical. Legal is about what is and is not lawful. Ethics is about what should and should not be, from a moral standpoint. There are many overlaps and, ideally, laws are also ethical. But a discussion about what is ethical, or not, cannot be limited to what is legal or not.

***

Ethical AI is a widely discussed and disputed notion for several reasons. Untangling this would be a whole other article, but highlighting just two tensions will help illustrate the issue present: one is about the technological solutions to “achieve” ethical AI vs the processes that will enable the existence of “ethical AI”, and another one is about the question whether it is even possible for a corporation such as Google to be ethical. They are, of course, related.

I will not expand on the first tension here but refer to two important articles on the topic instead: this excellent Nature piece by Dr. Pratyusha Kalluri, and the award-winning article “Technology Can’t Fix Algorithmic Injustice“.

The second tension brings me back to a Friday night in May 2018. I had the priviledge to attend a panel discussion between Tarleton Gillespie, Ben Tarnoff and the Tressie McMillan Cottom. To be honest, I do not recall the exact quote nor the exact circumstances — a discussion about the power of big tech companies? about the lawmakers’ inaction? about the futility of consumer boycotts? probably all of them — but I vividly remember one particular affirmation Dr. McMillan Cottom made that night: workers are vectors of change.

At that time, I did not fully understand what she meant in the context of the panel (despite the fact that a few weeks earlier there’d been big internal protests at Google against the company’s involvement in “the business of war”, a.k.a project Maven). Then, three months later, in the beginning of August 2018, the public, including many people working at Google, learned about the existence of project Dragonfly, a web search engine for China that censors in compliance with the Chinese government. Google employees of the company got organized. And the implications of Dr. McMillan Cottom’s statement finally made sense to me.

Employees can be vectors of change. They can nudge, prod, force, inspire, encourage their company towards doing the right thing. Workers may wield some influence in economic settings where end-users have none.

The list of companies behaving unethically is long. When it comes to big tech and AI, it has even been suggested that “ethical AI” itself is an unethical farce, a corporate tactic aiming at avoiding regulation. Yet, if companies behave more ethically even if it is for the wrong reasons, are we not all better off anyway? Well, it depends on who the “we” is. If the premise is that a company actually behaves more ethically, this must include how the company works internally and it must comprise employees. Behaving ethically cannot be done with outwards facing principles and panels alone. It cannot be done with ethical entities alone. An ethical company also needs to be based on ethical processes, which make room for hard questions about what should be.

***

Companies benefits from the people it employs. Such benefit can take many forms, for example labor, expertise, connections, or reputation. By “company” I do not mean the people it is comprised of but the economic, structural entity in whose name people act.

Timnit Gebru holds a a PhD from the prestigious Stanford Artificial Intelligence Laboratory, where she studied computer vision under Fei-Fei Li. Prior, she had also worked at Apple, and after graduating she did a postdoc at Microsoft’s FATE (Fairness, Accountability, Transparency and Ethics in AI). Dr. Gebru also co-founded the Black in AI community (together with Rediet Abebe) and the FAccT Conference. Her research is both innovative and important, widely recognized and appreciated.

It is evident that Google could benefit immensly from employing someone with the expertise of Dr. Gebru. In addition, by hiring a leading AI researcher who happens to be a woman of color, the company takes a small step in correcting its abysimal diversity statistics (only close to one third of Google employees happen to be women, and in Google’s U.S. workforce, black women only make up 1.6%). _{[In Sillicon Valley, Google is unfortunately not an outlier in this regard.]}

There is a third, more indirect benefit for Google, though it is mainly reputational. Dr. Gebru and her work aim at making this world a better place. If she agrees to work for Google, the company cannot be that bad, now can it. Indeed, a company that Dr. Gebru is willing to work for simply cannot be entirely unethical… it must at least be trying to do better, right?

The fact that Dr. Gebru is not part of Google anymore is a loss for the company because Google loses both expertise and reputation: it is hard to keep even an ounce of good faith in Google if the company is not willing to retain a brilliant black AI expert who is trying to make it more ethical.

But most of all, the fact that Dr. Gebru is not part of Google anymore is a a loss for all of us: because both Google and AI are powerful in today’s world, we are all worse off if Google so very clearly does not value what Dr. Gebru has to offer.

***

Disputes over what happened and who did when what exactly seem unproductive to me in this regard. Even if they were resolved, they do not change the outcome and, thus, the conclusion: Google very clearly does not value neither Dr. Gebru herself nor her expertise.

But I want to make explicit that I, of course, believe Dr. Gebru. I am horrified that she was effectively fired from one day to the next for doing her job, which is: asking hard questions about what should be, and pushing the company to do better and be more ethical. I have seen people point to Google’s supposed rationale for firing her, but these arguments are legalese sugarcoating of the fact that Google wanted to look like it was doing ethics without actually doing ethics.

Behaving more ethically requires more ethical processes. Well, Dr. Gebru spoke out against unethical practices and processes and demanded accountability; her recent research warns of severe risks of AI for people and the planet — I can hardly think of a better opportunity for an AI company to behave more ethically than to do listen to, learn from, and empower someone like Dr. Gebru.

Google, however, clearly has no interest in behaving more ethically, for whatever reason. Absent stringent regulation, only organized internal pushback at great personal cost seems to positively impact the company’s unethical policies. _{[You may want to keep in mind that Google is not necessarily doing worse than other technology company in this regard, on the contrary. Some of Google’s behaviours like its comparatively transparent research outputs may even make it fare better compared to other coporporations. But the the bar is low and Dr. Gebru has just been fired, so please excuse my specific focus here.]}

Remember Google’s “AI Ethics council”? Its make-up, so blatantly motivated by politics, incited an internal protest petition and ATEAC ended up being disbanded a few days after it was announced. What should Google have done differently? The MIT Technology Review published several relevant recommendations, including mine (stop treating ethics like a PR game!), but it is Prof. Anna Lauren Hoffmann’s suggestion I want to highlight here: “Look inward and empower employees who stand in solidarity with vulnerable groups.”

Remember the Google walkouts of 2018, sparked by the $90 billion pay-out to a high-ranked employee accused of sexual assault? The organizers paid a great individual cost. What should Google have done differently? Probably everything, as the U.S. National Labor Relations Board found that the company illegally surveilled and fired employees for organizing. Ironically, these news broke the same day Dr. Timnit Gebru announced she had been fired…

***

Now what?

For a start, I have signed my very first online petition asking for accountability regarding Dr. Gebru’s termination (I even signed twice due to an e-mail error). Although I am not overly worried about where she will land, I am very sorry for what she has been experiencing.

This is of course about Dr. Gebru. Yet this is also about so much more. It is also about the power of corporations. It is also about underrepresented people in the workplace. And ultimately, it is about all of us. The way Google treated the person they hired to do ethics highlights the profoundly undemocratic and unjust configuration between people and corporations, which may behave in authoritarian ways without consequences. What happened illustrates the unilateral power of tech companies, which significantly shape our world and our lives without ever having to take into account the slightest amount of critique.

This is not how things should be.

This is not ethical.