Phillip Herndon

AI, Corporations & the Morality of Artificial Agential Systems

2025-12-05T20:03:43.360837+00:00

Abstract

The parallels between the moral problems raised by AI and corporations are commonly referenced, but the depth of their overlap is more than incidental. I argue that corporations and AI can both be categorized as artificial agential systems (AAS), a group of entities which pose unique moral problems relating to responsibility, accountability and agency cultivation. Considering AAS as a whole allows us to jointly address longstanding debates in each domain and reveals tangible opportunities for improving our moral environment. The paper illustrates the parallels in problems of moral responsibility between corporations and AI, particularly regarding agency and the compontentized makeup of the systems. I argue that both systems are sociotechnical in nature, and that the technical elements of each system actively minimize human agency. I consider whether this can be reduced to the problem of many hands, but rebut the argument through further illustrated this tendency to minimization and human agency disintegration. I propose that a revised moral framework—one that recognizes the distinct agency of AAS and emphasizes the cultivation of human moral agency—offers a more effective path for integrating these systems into our moral ecology and addressing the novel ethical risks they pose.

1. Introduction

In December 2024 the CEO of a major U.S. health insurance company was shot and killed, allegedly by a man with grievances over the company’s handling of healthcare coverage ¹. The U.S. public was split on the morality of the act, with many surprised by the significant support the accused garnered (Meko, 2025).

Meanwhile Elon Musk, currently the world’s richest person, vigorously debates whether highlighting the dangers of autonomous vehicles is morally acceptable, saying, “If, in writing some article that’s negative, you effectively dissuade people from using an autonomous vehicle, you’re killing people” (Bohn, 2016).

Despite a great deal of discussion, there is little consensus on how the actions of corporations and artificial intelligence (AI) fit into our moral landscape. AI is relatively new as a technology, and our moral practices are catching up to technological progress. And while corporations are not novel, their increasingly dominant societal presence has made the blindspots in our moral ascriptions more apparent.

In both cases, we notably run into the problem of moral responsibility gaps. Moral responsibility describes a set of practices including our ascriptions of who or what is accountable, attributable or answerable for an event or outcome. This differentiates moral responsibility from other concepts of responsibility like legal responsibility (wherein we consider legal liability) or causal responsibility (who or what led to an event occurring)².

Moral responsibility gaps exist where we have a morally loaded outcome without an adequate target for accountability or responsibility. The outcome must be morally loaded, meaning that it is morally good or bad (Floridi, 2016), to discern from acts of nature, luck, and other outcomes that may be tragic or favorable but without moral valence.

I show that corporate and AI ethics face many of the same ethical questions and argue that this is because they are similar types of systems, which I call artificial agential systems (AAS). This categorization implies that not only are the ethical questions the same, the answers are the same. I argue that this categorization untangles long-running debates in both the corporate and AI cases, and illuminates productive pathways for future analysis.

To do this, I describe the major turns and questions debated in the cases of both corporate moral responsibility and AI and moral responsibility. I then outline relevant parallels between the debates and show that the parallels exist because AI and corporations are the same type of system. With an understanding of the overarching system behavior, I show how analysis at the category level can advance both debates.

I further describe the behavior of AAS by addressing two objections: that my categorization is not apt (corporations and AI are distinct in relevant ways) and that it does not abstract at the appropriate level (we can categorize the problem more broadly).

2. Moral Responsibility & Corporations

Corporations play a dominant role in society and regularly exact morally loaded consequences on the people and communities with which they interact. Many people are employed as attorneys of corporate law, auditors and regulators, responsible for litigating and mitigating issues of causal and legal responsibility. Perhaps these well-formed systems of responsibility ascription lead us to reflexively look for the same when it comes to corporate moral responsibility — who or what can be held accountable when corporate actions have morally loaded repercussions.

In the literature on moral responsibility and corporate activity, the debate is generally framed against proving whether or not corporations are appropriate vessels for moral responsibility, as they are for causal and legal responsibility.

Pettit, individually and with List (List & Pettit, 2013; Pettit, 2017), has influentially argued that corporations are a type of group agent. Group agents must meet specific requirements for agency, and corporations in particular meet both those and additional specific requirements for being moral agents.

To be a moral agent, an entity must a) be an agent and b) must be able to be held responsible for its actions. Holding responsible is different from thinking an entity is responsible. In the latter case, the entity is merely a candidate for praise or blame. Holding responsible is the activity of blaming or praising an entity. If an agent is fit to be held responsible, it meets the criteria for being a moral agent.

List and Pettit offer three criteria for an agent to be able to be held responsible for a particular choice:

Normative significance: The agent must have been faced with a morally significant choice. It must have the opportunity to choose between a right and wrong action.

Judgmental capacity: The agent has the appropriate information needed to make a considered decision, and the capacities and understanding to choose between available options.

Relevant control: The agent is able to make the choice (i.e. it has control over its decision).

List and Pettit ³ hold that each of these criteria are necessary for fitness to be held responsible, as an agent who does not face morally significant choices, does not have the appropriate information or capacity to judge a decision, or is unable to autonomously make a choice is unlikely to be held to be morally responsible for an action. They also hold these are sufficient criteria, as someone who meets these criteria would be hard pressed to argue that they should not be fit to be held responsible.

Critics argue that corporations do not meet these criteria (Moen, 2023; Rönnegard & Velasquez, 2017), that these are the incorrect criteria from which to judge (Sepinwall, 2017), and that attributing moral responsibility to corporations is at best useless, and at worst harmful (Hasnas, 2017).

2.1 Corporations Do Not Meet Pettit’s Criteria

Rönnegard and Velasquez (2017) outline six arguments against attributing moral responsibility to corporations, two of which target criteria for holding corporations responsible. Similarly, Moen (2023) argues that corporations do not meet the control condition.

Rönnegard and Velasquez argue that corporations do not have mental states, therefore cannot meet the second and third criteria. They cannot meet the criterion for judgmental capacity because corporations do not hold their own beliefs and desires. While we often attribute beliefs and desires to corporations, this is merely metaphorical. That is, it is inaccurate to reference beliefs or desires to describe or explain the actions of a corporation. These actions can be better described by referencing processes, actions by members, and the architecture of the corporation itself, among other things.

Without mental states, a corporation cannot be said to understand why it has chosen between available options. The corporation does not have considered reasons for its actions.

Pettit considers metaphorical attribution, however, and describes a discursive dilemma which could occur in a corporate setting. The discursive dilemma shows how some decisions a company makes are irreducible to the beliefs and preferences of the membership. In fact, in Pettit’s example (2017) the corporation takes a decision opposite what any representative member would prefer (see Table 1).

Table 1

An illustration of a discursive dilemma (Pettit, 2017). While this is meant to underline the control condition corporations must meet to be held responsible, Moen (2023) argues that this is not enough, as the membership’s capacity for strategic behavior (they could figure out the game and vote accordingly) keeps the ultimate control of the situation with them.

2.2 Methodological Issues

Perhaps the issue with holding corporations morally accountable precedes Pettit’s criteria. Rönnegard and Velasquez (2017), and more expansively Sepinwall (2017), argue that the method Pettit uses for assessing fitness to be held responsible misunderstands the practical application and goals of our moral practices.

The argument builds off of Strawson’s foundational work (2008) on reactive attitudes where he identifies second-order attitudes people have when they are the actor or subject of moral judgement. These attitudes and reactions like resentment, gratitude and forgiveness are inextricable elements of our moral practices.

As Sepinwall emphasizes, reaction to moral praise or indignation is more than cognitive: it has an important emotional element which lends these practices their efficacy. The emotional impact of being the target of blame, for instance, is part of its punitive force. Rönnegard and Velasquez also stress the importance of emotion in our moral practices, adding that without emotion one is unable to fully and accurately understand moral standards.

Without emotion, corporations could meet Pettit’s criteria to be held responsible and still be inadequate targets of moral responsibility. The argument is not that without emotion praise and blame do not serve an adequate regulatory function on future action. It is that without emotion the practice of holding morally responsible is misdirected. It’s like cutting a sandwich with a spoon. You may end up with two pieces of sandwich, but it is not the appropriate use of the tool (and there are better cutlery options to get the outcome you’re looking for).

2.3 Corporate Moral Responsibility is Unnecessary

Even if corporations could be held morally responsible they shouldn’t be, Hasnas (2017) argues. Denying corporate moral responsibility doesn’t mean we can’t assign any type of responsibility. We have significant structures to accuse, judge and redress situations of civil liability, administrative responsibility, and metaphorical responsibility (the figurative ascriptions people commonly attribute to companies).

Moreover, ascribing moral responsibility to corporations opens the door for injustice. Moral responsibility implies that corporations can be subject to criminal punishment, but criminal punishment cannot target a corporation because the corporate entity is not an individual subject. Any punishment of the corporation necessarily is punishment for persons downstream: corporate members, employees or consumers of the corporation’s products and services. This is unfairly punitive to innocent people (consumers, for instance, have no role in perpetrating corporate malfeasance). “Corporate punishment is inherently vicarious collective punishment,” (Hasnas, 2017, p. 100). Rönnegard and Velasquez (2017) offer a similar critique.

Floridi (2016) attempts to bridge this by altering the concept of moral responsibility in distributed settings like corporations. Rather than holding corporations morally responsible like we would hold an individual morally responsible, we can develop a concept of faultless responsibility. When a distributed system does a morally loaded action, faultless responsibility would be distributed equally to all causally-relevant nodes in the system regardless of intention. In a corporation, responsibility might be distributed to causally-relevant employees and members, for instance.

Floridi sidesteps the question of collective punishment by stressing that the distributed moral responsibility must be faultless in nature, a strict liability where "there is no requirement to prove fault, negligence or intention" (2016, p. 8). Responsibility could be distributed using a method of backward propagation, similar to how neural networks are trained to generate more favorable outputs.

With a system of faultless responsibility, corporate moral responsibility also becomes unnecessary. Responsibility is distributed from the corporation to the nodes of the system which receive appropriate faultless praise or blame.

Even so, if corporations are not fit to be held morally responsible, we face staggering gaps in our responsibility ascriptions. Pettit points to an “avoidable shortfall in the regulatory effects that our responsibility practices are generally designed to achieve. We will leave a loophole for people to incorporate for socially harmful but selfishly rewarding ends, and to do so with relative impunity” (2017, p. 33).

I address this moral responsibility gap more below.

3. Moral Responsibility & AI

Debate concerning the moral responsibility of artificial intelligence (AI) tends to focus less on whether AIs are fit to be held morally responsible, and more on where moral responsibility falls given that:

An AI’s actions can be morally loaded, and
An AI as technology is not an appropriate target for moral responsibility

Matthias was early to describe this responsibility gap problem (2004). The design of AI systems involves creating a hidden decision layer between the system’s input and output. In the decision layer, inputs are driven through an intricate series of weighting algorithms which branch through potential solution paths, converging on an output which is ostensibly based on the node weighting tuned through previous training. This decision layer is “hidden” in that it is a black box for humans. The criteria for weighting across the millions or billions of nodes is not symbolic — it does not correspond to human concepts — and the network’s massive complexity makes it impractical to untangle and analyze.

In cases where an AI makes a morally loaded and unpredictable action — imagine a surgical robot going forward with a risky medical intervention which ends up failing, or a recruiting software which systematically rejects applications from women — we are left with a responsibility gap. The standard view is that this gap comes from a lack of two general conditions of responsibility: the epistemic condition and the control condition (Coeckelbergh, 2020). The epistemic condition dictates that to be morally responsible one must know what they are doing. The control condition requires that one must have sufficient control over the action in question. Note that these criteria are similar (though not equal to) Pettit’s criteria for group moral agency.

If the AI’s action was neither predictable nor controllable by the system’s developers or users, neither can meet these responsibility criteria. Likewise, the AI does not seem to have “knowledge” of its actions and does not have the appropriate kind of control over its outputs to be morally responsible.

The responsibility gap is posed as a problem for our normative understanding of moral responsibility. As people continue to create, operate and rely on AI systems for important societal functions, how do we ascribe blame (or praise) when something goes wrong (or right)?

3.1 Use the Tools & Targets We Have

Multiple philosophers have offered solutions to bridging or dissolving the AI responsibility gap, citing pre-existing responsibility practices and targets.

Nyholm (2018) and Himmelreich (2019) suggest we look to the users of the systems for ascribing responsibility. Our relationships with AI is not such that these systems are off acting entirely without human action. There are people using the systems. Nyholm argues that this creates a situation where collaborative responsibility can be appropriate. Users maintain a relationship of moral responsibility by virtue of being the only party involved which is a fully responsible moral agent. In the case of autonomous weapons systems, Himmelreich similarly argues that a military commander could be seen as reasonably responsible for the actions of the autonomous system.

Or perhaps through deliberate system design we can both improve the possibility of users to assume responsibility or be responsible, and affect the allocation of responsibility among the users and other affected groups (Nihlén Fahlquist et al., 2015).

Nihlén Fahlquist et al. differentiate between backward-looking responsibility and forward-looking responsibility. Backward-looking responsibility is the responsibility I’ve addressed so far. It is assessing responsibility based on certain criteria an agent either meets or does not. Forward-looking responsibility is responsibility seen as a virtue. Forward-looking moral responsibility practices are a continuous habituation and development of responsible dispositions, attitudes and actions rather than specific requirements.

Technologies, and AI systems, can be designed to promote both, and Nihlén Fahlquist et al. offer twelve design heuristics to help accomplish this. Relevant to this discussion, they recommend designers distribute responsibilities created by the system fairly, effectively and completely (i.e., such that “for each relevant issue at least one individual is responsible,” (2015, p. 486)). Following these guidelines we necessarily close the moral responsibility gap.

Where morally loaded actions aren’t genuinely blameless, Hindriks and Veluwenkamp also invoke system design (2023), proposing the existence of a control gap rather than a responsibility gap. A control gap is risk-based rather than capacity-based. When a system has an unacceptable level of risk of behaving unpredictably it can be said to display a control gap. These control gaps can be “decreased, if not avoided all together” by improving the system’s ability to “emulate guidance control” and through safety engineering (proactive and reactive safety barriers) (Hindriks & Veluwenkamp, 2023, p. 21). Thus we close the responsibility gaps by placing relevant subsystems “under meaningful human control” (ibid).

These solutions sidestep the issue, however. Responsibility gaps can exist in situations where an AI user is acting ethically appropriately. The users that Nyholm and Himmelreich target have knowledge of the morally loaded potential of their systems’ actions, but we can’t always rely on this. Likewise, Nihlén Fahlquist et al. and Hindriks and Veluwenkamp offer meaningful ways to avoid responsibility gaps, but unsatisfying results in addressing problems of the gaps as they exist in the world.

Rather than targeting the closest individuals, Tigard (2020) recommends we use our pre-existing moral practices in cases of morally loaded AI outcomes. Blaming, praising, or expressing our reactive attitudes toward AI that create these outcomes is normatively appropriate, Tigard argues. Though the system may be insufficiently responsive to such attitudes, we can then move to an objective stance and use other accountability practices like sanctions and behavior correction to improve the AI’s future actions.

Focusing on pre-existing practices ultimately leaves us as unsatisfied as changing our targets. We already use our pre-existing moral practices, and the gaps still seem to exist, particularly concerning backward-looking responsibility. Tigard stresses that our moral practices are much broader than accountability, but does not offer satisfying responses to how to address accountability for deeds done.

3.2 Develop New Responsibility Practices

If we can’t use the tools we have, perhaps we can broaden our moral practices to acknowledge our increasingly networked and complex world.

Responsibility attribution above is largely focused on individual responsibility: which individual persons or systems can hold our responsibility ascriptions?

Taylor (2024) discusses efforts to extend our responsibility ascriptions in cases of AI action to a collective rather than a single or set of individuals. Those people involved in the development, design and deployment of AI systems may together be considered a collective entity with its own, separate agency (similar to Pettit’s corporation). Should these groups also exhibit moral agency, they become adequate targets for responsibility ascriptions. Taylor concludes, however, that this collective responsibility will not be adequate to address the moral costs responsibility gaps can create, and the gaps’ effects remain.

Goetze (2022) offers a solution in which we alter our responsibility practices to accept blameless responsibility on the part of AI systems’ developers and designers for morally loaded AI actions. Goetze’s vicarious responsibility is similar to Floridi’s faultless responsibility in that both have connected individuals accepting responsibility for an outcome without having personal agency in said outcome. Goetze does not suggest splitting the blame equally among nodes as Floridi does. Instead, a system’s developers can take responsibility for an AI’s morally loaded actions without the moral demerit (or merit) that goes with being held responsible.

Stahl (2006) turns not to the developers of the system but to AI and autonomous systems themselves, offering that we can alter our moral practices to ascribe quasi-responsibility to these systems. He takes an instrumentalist view of responsibility, stressing its nature “as a social construct of ascription, aimed at achieving certain social goals” (2006, p. 206). With this in mind, we can attribute a responsibility without reference to agency or personhood — the quasi-responsibility — to the systems themselves. The responsibility would be with “the purpose of ascribing a subject to an object with the aim of attributing sanctions (the heart of responsibility) without regard to the question of whether the subject fulfills the traditional conditions of responsibility” (2006, p. 210).

While Goetze and Stahl’s arguments are grounded in pre-existing, everyday moral practices, they flatten the notion of moral responsibility and ignore much of its depth in our lived experience.

3.3 Reconsider Our Method

How then can we appropriately address responsibility for the morally loaded outcomes of AI systems while fully acknowledging the roles and nature of morality? If we fully acknowledge the nature of morality individually and societally, perhaps we’ll find that we have misconstrued the problem.

More so than assigning praise and blame, moral responsibility is a relational and practice-based phenomenon. It is focused on actively cultivating and maintaining moral agents; praise and blame being only one feature of the system. Vallor and Vierkant (2024) draw out this distinction, pulling from Strawson (2008) and Vargas (2013, 2021). They argue that responsibility gap debates are best directed not only at how best to ascribe praise or blame but also how to best create a healthy moral ecology.

This is a much more forward-looking account of AI responsibility. Vallor and Vierkant argue that the moral performance of an AI system is derivative of the moral maturity of the developers, users and regulators of the system. Therefore, in cases where our typical backward-looking responsibility attributions fail — in cases of responsibility gaps — “an open-ended suite of other responsibility practices remains available to us, including answerability and accountability practices understood proleptically” (Vallor & Vierkant, 2024, p. 12).

In order to fold AI into our moral ecology, Vallor and Vierkant allow for both using tools we already have and developing new responsibility practices. But by focusing on cultivating moral agency in people, a new way to untangle the responsibility gap problem appears.

Much of our moral practice has developed in order to change the future actions of ourselves, other people and groups of people. The reactive attitudes, a constitutive element of our responsibility practice, are meant to invoke moral emotions in their targets as much as they do in their subjects.

But as Tigard (2020) points out in the case of AI, and Sepinwall (2017) does in the case of corporations, our reactive attitudes can be met with silence. Our moral practices break down in cases where we are in a moral relationship with a system, which creates morally loaded outcomes but is not an appropriate target for moral responsibility itself.

This lack of mutual recognition is a gap far deeper than a responsibility gap. Vallor and Vierkant deem this a vulnerability gap. AI does not have the appropriate emotional, social and cognitive capacities to be vulnerable in the same way humans are. This vulnerability asymmetry leads not only to apparent responsibility gaps but also potential agency disintegration, wherein a human delegates responsibility for a choice or action to a system which cannot hold the requisite moral obligations concerning its execution.

Reframing the AI responsibility problem as a problem of mutual recognition and identifying the goal as improving our moral ecology opens up new avenues for innovation, but does not resolve the how of integrating AI into our moral ecology.

4. Addressing Artificial Agential Systems

As you can see, corporate and AI morality discussions traverse similar terrain. Could typical moral practices account for the actions of these systems? Do criteria for human moral agency transfer to them? Can responsibility for these systems’ actions be distributed to associated people? And beneath it all there lies a debate-altering distinction between criteria-based accounts of moral agency and holistic or virtue-based accounts. What we additionally will find is that not only are the questions the same, the answers are too.

Vallor and Vierkant (2024) note the similarity between corporations and AI in their work surrounding the AI responsibility gap. Sepinwall (2024) and List (2021) analyze the similarities from the corporate/group agency starting point. The overlap of these discussions is not coincidental — corporations and AI are similar entities in many respects. Sepinwall (following Reyes, 2021) groups them as artificial systems, meaning they are human-originated (i.e., artificial) goal-directed groups of components working in a coordinated manner to achieve those goal(s) (systems). List identifies the key parallel between group agency and artificial intelligence as “the fact that both phenomena involve entities distinct from individual human beings that qualify as goal-directed, ‘intentional’ agents, with the ability to make a significant difference to the social world” (2021, p. 1218).

I am limiting my discussion to corporations, not all group agents, for reasons I’ll make clear shortly. To move forward, I take List’s insight into the agency of these systems and add it to Reyes and Sepinwall’s emphasis on human origination.

Corporations and AI share significant parallels because they are both artificial agential systems (AAS). Agency for this definition includes basic requirements of agency, what Nyholm calls “domain-specific basic agency: pursuing goals on the basis of representations, within certain limited domains,” (2018, p. 1207). Of course, systems with higher degrees of agency also qualify as artificial agential systems, but the agency threshold is low.

This basic agency requirement separates the relevant systems from artificial systems which would not have the same problems when it comes to ascribing responsibility, like a mechanical loom or a sufficiently small business.

Having identified AAS, we can now better identify the problems of moral responsibility ascription discussed above. Rather than separate questions; “how can we ascribe moral responsibility to the acts of corporations” and “how can we ascribe moral responsibility to the acts of AI systems” we can ask “how can we ascribe moral responsibility to the acts of artificial agential systems?” The answer should apply to both corporations and AI.

Indeed, through our tour of the debate on AI and corporations above, we can identify ways in which combining the discussions will move both forward. A large part of the discussion on AI, for instance, bypasses whether AI systems are fit to be held responsible (apart from future-oriented discussions). As we’ve seen, corporations’ fitness to be held responsible is in dire straits. Participants in both discussions have also attempted to distribute responsibility to humans involved in the systems. In both cases they have failed, and for the same reasons: they ignore crucial aspects of our comprehensive responsibility practices. Thirdly, there are compelling arguments in both domains that moral responsibility is not just an atomist question of meeting certain conditions. Moral responsibility is part of a continuous cultivation of moral agency. The effort to untangle multiplying knots concerning responsibility ascription will be less effective than a coordinated practice which humanely folds all agential systems we create into our moral ecology.

There are immediate practical benefits as well. Society’s broader approach to responsibility in the corporate arena is well formed, and there is robust infrastructure in place, particularly on questions of legal and causal responsibility. AI responsibility frameworks can build off of this. AI, for its part, benefits from a currently dizzying pace of experimentation and technical progress. Moral innovation in the AI domain can be applied to similar questions for corporations. We can more effectively incorporate these systems into a working moral ecology by addressing them and other artificial agential systems as a group.

Perhaps there are good reasons not to combine the discussions. Pettit and List describe corporations as group agents, and are thus dependent on human constituency. AI are not group agents (which can include entities like clubs and political parties) and thus it would be more fitting to designate AI, which is autonomous from human intervention, separately.

On the other hand, perhaps our dissolution of the problems does not go far enough. Rather than grouping the problems present as problems of AAS, we can better characterize the moral responsibility issues around corporate and AI action as the familiar “many hands problem.” After all, if AAS do not present as moral agents, responsibility and our ascription practices must cascade to those who do ⁴.

I argue against these positions in the next two sections.

5. AIs & Corporations are Too Different

List and Pettit introduce a commonsense conception of group agents as “collections of human beings [which act] as if they were unitary agents, capable of performing like individuals” (List & Pettit, 2013, p. 1). The recent discussion of corporate moral agency has followed suit, asking how best to ascribe moral responsibility given that corporations are constituted by human members asserting their own individual agency, from which emerges a separate agent able to perform like an individual. Meanwhile, there is a noticeable lack of humanity in our initial discussions of AI and moral responsibility. Philosophers begin with an absence of humans in the system and work to explain (in many cases) where human responsibility best fits in.

This disparity may point to an important difference between AI and corporate moral responsibility. If this is true, we should consider our corporate moral responsibility practices in light of their human constituency. The solutions become primarily social, focused on circling the square of this odd quasi-human agent. It’s as if the corporation were an alien or a zombie — it is not like us in important ways, but enough like us that we may want to apply moral obligations and rights to it or to us concerning it.

AIs, on the other hand, are first and foremost technical systems. In addition to social considerations, we are able to consider AI’s technical makeup and apply technical innovation to help overcome our responsibility problems. The philosophical discussion focuses on the actions of the AI removed from humans and driven by the black box of its decision making infrastructure. Examples concerning self-driving cars, autonomous weapons systems and facial recognition software remove humans from the system’s actions except as targets for injury or injustice.

Commonsense bears this out. Corporations are made of people (an employee may consider themself part of a corporation) while AI are made of computer code (they exist on servers or in a computer). If these systems are truly as different as this, it would be a mistake to try to group them under the category of AAS and focus analysis on that. We would lose out on important tactics for solving these problems: technically in the case of AI and societally in the case of corporations.

However, these idealized views of the corporation and AI do not accurately reflect reality. AIs are sociotechnical systems, as dependent on societal infrastructure as they are on technical infrastructure. The modern corporation ⁵ is also a sociotechnical system, and in both systems we see agency instantiated in the technical side of the system. This agency actively sheds dependencies on direct human involvement.

5.1 AIs are Sociotechnical Systems

I’ll take the sociotechnical nature of AI first, as it has been emphasized previously (see Vallor & Vierkant, 2024). While it seems most of the action of an AI — its ingestion and interpretation of inputs to produce novel outputs — occurs in a self-contained, dynamic system, in fact AIs rely on a large array of human social practices to maintain and enable their continued existence and indeed to produce their actions. This is implied in various degrees by arguments for responsibility distribution we’ve seen above, but along with suggesting a potential (though problematic) out for the moral responsibility gap, the reliance on a social element also affects how we approach AI holistically.

We have already established that AIs are artificial (they are human-originated). The role of designers, users, companies, energy and regulatory infrastructure, and other social practices are crucial to start an AI’s program running, direct it to necessary inputs and keep it going. The social elements represent necessary components to an AI’s continuing function. Without recognizing this, the autonomy of AI systems is easy to overestimate.

I don’t mean to overemphasize the human element, though. AI is novel and philosophically relevant in part because the technology alienates human control from its outputs more so than other technologies. Considering this, we end up in a place where social practices are part of the AI system, but not necessarily. Humans are part of the AI system, but not in a way that establishes control in a morally meaningful fashion. Vallor and Vierkant argue that the moral performance of an AI as tool is can only be understood from the level of a sociotechnical system. I disagree, and show below that AI as technology (apart from its full sociotechnical system) has agency which separates its actions from that of the sociotechnical influences.

5.2 Corporations are Not Group Agents

Meanwhile, the description of corporations as group agents is no longer accurate. Categorizing corporations as a group agent places humans as a necessary constituent part of the entity. However, many corporations as they exist today — and certainly the largest corporations which produce the most outsized moral consequences on our communities — have been designed in much the same way as AI: to deemphasize human decision making.

Consider the membership makeup of the modern corporation ⁶. The agency of individual directors and shareholders is subordinate to the goals and representations (that is, the agency) of other AAS. When a decision needs to be made, the human members of a corporation are not lending individual judgment. Instead they are reflecting the incentives and dispositions of the AAS they represent, whether that be an investment group, a pension fund, another corporation or the corporation itself. The human members of the modern corporation repudiate personal agency and become components of the system. Activist investors can be an example of members defying this disavowal of agency, but these aberrant actions help illustrate that corporations are not expecting or designed to be run as an aggregation of human intention — as a group agent.

Hyper-complex corporations rely little on human agency to make corporate-wide decisions, particularly those that are the most morally loaded. Corporate actions, then, should not be described as aggregation of human intention but are better described as the outcome of interactions between the entity and systems like stock exchanges, governments and other corporations.

While this holds distinctly for mega corporations, we also see distancing from human agency in the creation of shell companies and special-purpose entities. Decentralized autonomous organizations (DAOs) represent a relatively new corporate structure (not legally recognized everywhere) in which governance and management of a corporation is automated and executed through a program on a blockchain with up to no human intervention (Rikken et al., 2023). Consider that corporations can create subsidiary corporations, and it becomes not just possible, but likely to have corporations with human origination, but without humans associated with the corporation itself.

The purpose of incorporating, you might say, is to deflect all of our responsibility practices away from human actors (as it explicitly is in the case of legal responsibility). Its nature is anti-humanistic. Humans can be said to be part of the makeup of the typical modern corporation, but are not a necessary part. Social practices affect the corporate sociotechnical system, but not in a way that establishes control in a morally meaningful fashion. The technology of the corporation, that is, the structures and processes that produce decisions, has agency which separates its actions from that of its sociotechnical influences.

A corporation, then, is not a group agent and it is a mistake to categorize corporations as such. Corporations, like AIs, are sociotechnical systems with technical cores that have variable levels of reliance on human action. We cannot differentiate these systems from each other or from other systems by the nature of human participation with the system. We should instead look to agency.

I write about this in more depth in the next section.

6. Avoiding the Problem of Many Hands

It may be the case that AI, corporations and analogous complex systems raise similar enough ethical issues that it would be preferable to treat them as a category. If so, we could also look more broadly. Are the problems described with AAS best addressed at an even wider scope?

Perhaps the ethical obstacles of AAS are examples of the many hands problem, which we see not only in artificial agential systems but in other complex interactions like the phenomena which bring about global climate change (van de Poel et al., 2012). The problem of many hands (PMH) is a situation in which, “due to the complexity of the situation and the number of actors involved, it is impossible or at least very difficult to hold someone reasonably responsible” (van de Poel et al., 2012, p. 50) ⁷.

Take AI for instance. When we establish that there are responsibility gaps regarding the actions of AI, it is often due to the sociotechnical systems that create and maintain them. Were a single inventor to program a powerful AI which then went mad, we could ascribe a healthy amount of responsibility to the individual inventor, similar to how we may blame adults for the transgressions of their misbehaved pets. The so-called responsibility gap for AI, then, only occurs when the forces creating, maintaining and sustaining the AI are more diffuse. In fact, this is likely to happen when a corporation or multiple corporations are operationally responsible for the AI.

What we come to in this case is that the AI responsibility gap problem reduces to problems of corporate moral responsibility. The problems we recognize in ascribing responsibility for the outcomes of AI systems exist because they are the same problems we have ascribing responsibility for corporate actions.

Morally loaded corporate actions, for their part, are examples of PMH. Modern businesses are built to exploit efficiencies and advantages of specialization and division of labor. Therefore, for any project or initiative the company undertakes there are many people doing discrete parts of the work to create an overall outcome. Add in support and operational functions and you have an extremely complex group of interactions. Backward-propagating responsibility for outcomes would be very difficult (though not impossible as Floridi shows above) and would not result in holding anyone responsible to a sufficient degree, hence the PMH.

So at this point we have a situation where our AI responsibility problems have been subsumed into the corporate case and our corporate case is well-described as the problem of many hands. Rather than focusing on AAS, why not focus our efforts on PMH, which can additionally lead to insights into societal tragedies such as humanitarian crises, food safety outbreaks and climate change?

Unfortunately, while addressing our moral practices concerning PMH is laudable, PMH does not apply in the case of AAS. PMH occurs where individual responsibility and the impact of individual agency is minimal. AAS’, on the other hand, actively suppress and disintegrate individual agency, replacing it with the agency of the technical systems.

PMH exists in situations with many actors or in collective settings. A collective here, and the actors described, are people or groups of people acting with individual agency. As I discussed earlier, however, the acts of corporations do not reduce to the acts or intentions of individuals. The actions of corporations are better described as the outcomes of interactions between the entity and other systems. The technology of the corporation suppresses the individual agency of the people involved in corporate decision making by design and replaces it with the representations and goals — that is, the basic agency — of the corporation or other AAS. This agency disintegration creates people who act as components of a system. Vehicles of agency rather than acting as agents themselves. And that is where people are the actors in a process. Often the activity of suppliers, competitors and markets, which instantiate as representations within the AAS, dictate actions well before human consideration.

AI as technology has a separate agency from the sociotechnical system that creates and sustains it. The capacities for AI to learn, and thereby create unpredictable outcomes, separate it from the goals and representations of its originators. This is the reason for the responsibility gap and why it cannot be subsumed into the corporate case. Like corporations, AI actions are decided based on representations and goals within the technological system itself. The intentions of sociotechnical actors such as users, corporations which build AI, designers and other groups are influential but secondary to the agency of the AI.

In the case of humans, I argued that people can have their agency effectively disintegrated in their roles as corporate members. This does not mean they’ve lost all their agency, just their agency in their role within the corporate system. Similarly, a sufficiently strong AI that is used by a corporation could have its agency disintegrated in its role in a corporate system (say as a part of a DAO). Like a person, the AI as technology would still have agency separate from the corporate agency. In fact, one of the current practical problems of AI systems is that they display more agency than human employees, in notable cases coming to unpredictable outcomes that are damaging to the corporations employing them.

Let’s turn to the case of the single inventor who creates an AI which then goes mad. The responsibility attribution in this case is less straightforward than it looks. Consider a single entrepreneur who founds a multinational corporation which at some point in the future has morally negative effects. The responsibility ascription to the founder would not be based on her creation of the company, but on her continued involvement in the running of the company. An autocratic leader of a single-member company might hold significant responsibility, while a company founder who is not an active member of the current running of the company (in the time period relevant to the negative actions) would not. Like a corporation, AI as technology changes over time as new inputs and representations are addressed to the system. An inventor who exercises heavy control over the working of an AI throughout its operation would be able to hold much of the responsibility for its outputs (one might also ask if this is actually AI ⁸), whereas an inventor whose AI goes rogue after years of operation in a complex informational environment may not. In short, the creation of the system is different from the running of the system.

Add to this that AI technologies, like corporations, have powerful agency disintegration effects. As Vallor and Vierkant put it, AI technologies “often disperse the contributions of human will, introduce more chaotic and random effects in action chains, and sever the cognitive and motivational links between means and ends that give actions moral meaning” (2024, p. 17). As with the corporate technology, AI as technology suppresses human agency as inputs are transformed into outputs.

Artificial agential systems, then, have qualities that prevent them from being considered as cases of PMH. As agential systems, they interrupt human agency in favor of their own. In modern corporations this takes the form of dependencies, contracts, processes and relationships that prioritize system-level representations and goals. In AI this presents as black box functions that alienate human intention from system output.

Does this split from human agency also make the outcomes caused by AAS morally neutral? Perhaps these outcomes cannot be morally loaded positively or negatively and are similar to outcomes caused by hurricanes or luck. This could pose a problem for my argument, and comprehensively defending it is beyond the scope of this discussion, but I believe the act of human origination, that is, AAS’ artificial nature, enables outcomes caused by AAS to remain morally loaded. A future analysis could combine human origination and a holistic view of our moral ecosystem to maintain moral valence throughout the process, separate from agency.

7. Implications of AAS

If, as I argue, artificial agential systems are an appropriate categorization of the systems we create that present at least basic elements of agency, the moral positioning of AI and corporations can be combined into a coherent whole.

This view draws most from arguments of Sepinwall, Vallor and Vierkant, and others who would have us reconsider our primarily backwards-looking method of ascribing responsibility in the case of corporations and AI. I break from them, Vallor and Vierkant in particular, by stressing the agency of the technical systems separate from an agency derived from the sociotechnical systems of which these entities are a part. Due to its constitution, the technical aspect of an AI or corporate sociotechnical system actively works to disintegrate the agency of surrounding entities, including associated individuals and the sociotechnical system of which it is a part.

Acknowledging AAS with these features both bolsters and rebuts arguments in the general debate we’ve seen so far.

Firstly, my argument reinforces List and Pettit’s irreducibility conclusion. Corporate-level decisions are irreducible to the beliefs and preferences of the membership as individual agents. This isn’t due to examples such as the discursive dilemma, however. It is because corporate members do not exercise their agency in the modern corporation. They represent the intentions of other AAS, or the corporation as AAS.

Because of this, my argument stymies efforts for distributive responsibility in both AI and corporate cases. As we’ve seen, the nature of AAS is such that at sufficient complexity they break the intentional link between people and system. Design and user intent do not have the regulative effects they should to create responsibility, and in cases where AAS require human activity to function they can do so without human agency.

Finally, my argument supports Sepinwall, Vallor and Vierkant’s emphasis on mutual recognition as necessary for moral interaction. AAS are likely to adapt and proliferate in the coming decades. While it is only possible that we could develop AAS that has the capacities for vulnerability and mutual recognition so as to be considered moral agents or patients, it is guaranteed that AAS as it exists today — as agency disintegrating — will continue.

Emphasizing mutual recognition in our morality practices creates a powerful incentive to combat agency disintegration individually and communally. A moral rejection of agency disintegration would change our relationship with AAS. While it wouldn’t eliminate AAS’ hostility toward human agency, it would allow us to better identify and mitigate the dangers these systems pose to our moral landscape. In fact, as corporations maintain a dominant societal position and as AI’s impact grows, such mitigation strategies have become essential to the continuing function of our moral environment.

AAS are a part of our moral ecology, but not as agents which can be held morally responsible or be equal participants in our moral practices. Instead their makeup and actions must be integrated as part of a responsibility all people have to develop as moral agents. A responsibility to create, cultivate and negotiate a morally thriving world. AAS are powerful mechanisms to obfuscate and destroy human agency. They can also lead to profound innovation, prosperity and knowledge. By exercising our moral judgment, reasoning and regulation we can balance the moral risks of AAS for the good of all.

8. Conclusion

The level of human agency required for the functioning of technologies and businesses is variable. Each can act with close interaction between human and system. In these cases our moral intuitions and practices can generally be exercised as-is. But when the technology becomes agential a split appears which places it in a different ethical position. This gap is created by system agency that separates itself from human agency. AIs and corporations are both technologies that proactively widen this gap.

Artificial agential systems (AAS) create unique and troubling moral problems relating to responsibility, accountability and agency cultivation. I have argued that corporations and AI, as AAS, have significant overlap in their moral positioning. Considering AAS as a whole allows us to better recognize the agency disintegrating power of these systems, and reveals tangible opportunities for improving our moral environment.

To get here, I gave a brief overview of the debate surrounding corporate moral responsibility and AI and responsibility. From there I drew parallels in the debates and grouped them under the category of artificial agential systems. Those parallels include effects on agency and the componentized makeup of the systems. I then showed how this can explain the similarities in the discursive turns in both debates.

I considered two objections: that the categorization goes too far and that it does not go far enough. As a counterargument to the idea that AI and corporate responsibility positions are distinct, I showed that the lack of human input in the case of AI is overblown, and the focus on human input in the corporate case misrepresents reality. In fact, both systems are sociotechnical systems in which the technical side of the system actively minimizes human agency. To rebut the idea that the problem can be reduced to the problem of many hands I further illustrated this tendency to minimization and human agency disintegration.

Ultimately, I show that artificial agential systems are an important conceptual categorization of two sources of pertinent problems in our moral landscape. By acknowledging the nature of AAS and addressing the lack of mutual recognition accentuated in current AAS we can begin to rectify existing moral problems and develop a healthier moral environment.

References

Björnsson, G., & Hess, K. (2017). Corporate Crocodile Tears?: On the Reactive Attitudes of Corporate Agents. Philosophy and Phenomenological Research, 94(2), 273–298

Bohn, D. (2016, October 19). Elon Musk: negative media coverage of autonomous vehicles could be ‘killing people’. The Verge.

Business Roundtable. (2016). Principles of Corporate Governance. BusinessRoundtable.org.

Coeckelbergh, M. (2020). Artificial Intelligence, Responsibility Attribution, and a Relational Justification of Explainability. Science and Engineering Ethics, 26(4), 2051–2068.

Davis, M. (2012). “Ain’t No One Here But Us Social Forces”: Constructing the Professional Responsibility of Engineers. Science and Engineering Ethics, 18(1), 13–34.

Floridi, L. (2016). Faultless responsibility: on the nature and allocation of moral responsibility for distributed moral actions. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2083), 20160112.

French, P. A. (1984). Collective and Corporate Responsibility. Columbia University Press.

Goetze, T. S. (2022). Mind the Gap: Autonomous Systems, the Responsibility Gap, and Moral Entanglement. 2022 ACM Conference on Fairness, Accountability, and Transparency, 390–400.

Hasnas, J. (2017). The phantom menace of the responsibility deficit. In E. W. Orts & N. C. Smith (Eds.), The Moral Responsibility of Firms. OUP Oxford.

Himmelreich, J. (2019). Responsibility for Killer Robots. Ethical Theory and Moral Practice, 22(3), 731–747.

Hindriks, F., & Veluwenkamp, H. (2023). The risks of autonomous machines: from responsibility gaps to control gaps. Synthese, 201(1).

Katersky, A., Deliso, M., & Pezenik, S. (2025, February 21). Luigi Mangione’s defense cites evidence concerns, no trial date set. ABC News.

List, C. (2021). Group Agency and Artificial Intelligence. Philosophy & Technology, 34(4), 1213–1242.

List, C., & Pettit, P. (2013). Group Agency: The Possibility, Design, and Status of Corporate Agents (Illustrated). Oxford University Press.

Matthias, A. (2004). The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology, 6(3), 175–183.

Meko, H. (2025, February 21). Suspect in Insurance C.E.O. Killing Creates Website as Support Floods In. The New York Times.

Moen, L. J. (2023). Against corporate responsibility. Journal of Social Philosophy, 55(1).

Nihlén Fahlquist, J., Doorn, N., & Van de Poel, I. (2015). Design for the Value of Responsibility. In J. van den Hoven, I. van de Poel, & P. Vermaas (Eds.), Handbook of Ethics, Values, and Technological Design: Sources, Theory, Values and Application Domains (2015th ed.). Springer.

Nyholm, S. (2018). Attributing Agency to Automated Systems: Reflections on Human–Robot Collaborations and Responsibility-Loci. Science and Engineering Ethics, 24(4), 1201–1219.

Pettit, P. (2017). The Conversable, Responsible Corporation. In E. W. Orts & N. C. Smith (Eds.), The Moral Responsibility of Firms (1st ed.). OUP Oxford.

Reyes, C. L. (2021). Autonomous corporate personhood. Wash. L. Rev., 96, 1453.

Rikken, O., Janssen, M., & Kwee, Z. (2023). The ins and outs of decentralized autonomous organizations (DAOs) unraveling the definitions, characteristics, and emerging developments of DAOs. Blockchain: Research and Applications, 4(3), 100143.

Rönnegard, D., & Velasquez, M. (2017). On (Not) Attributing Moral Responsibility to Organizations. In E. W. Orts & N. C. Smith (Eds.), The Moral Responsibility of Firms (1st ed.). OUP Oxford.

Sepinwall, A. (2017). Blame, Emotion, and the Corporation. In E. W. Orts & N. C. Smith (Eds.), The Moral Responsibility of Firms. OUP Oxford.

Sepinwall, A. (2024). Artificial Moral Agents: Corporations and AI. In S. Hormio & B. Wringe (Eds.), Collective Responsibility: Perspectives on Political Philosophy from Social Ontology (pp. 27–48). Springer Nature.

Stahl, B. C. (2006). Responsible computers? A case for ascribing quasi-responsibility to computers independent of personhood or agency. Ethics and Information Technology, 8(4), 205–213.

Strawson, P. F. (2008). Freedom and Resentment. In Freedom and Resentment and Other Essays (1st ed.). Routledge.

Taylor, I. (2024). Collective Responsibility and Artificial Intelligence. Philosophy & Technology,37(1).

Tigard, D. W. (2020). There Is No Techno-Responsibility Gap. Philosophy & Technology, 34(3), 589–607.

Vallor, S. (2023, July 14). Edinburgh Declaration on Responsibility for Responsible AI. Medium.

Vallor, S., & Vierkant, T. (2024). Find the Gap: AI, Responsible Agency and Vulnerability. Minds and Machines, 34(3).

van de Poel, I., Nihlén Fahlquist, J., Doorn, N., Zwart, S., & Royakkers, L. (2012). The Problem of Many Hands: Climate Change as an Example. Science and Engineering Ethics, 18(1), 49–67.

Vargas, M. (2013). Building better beings: A theory of moral responsibility. OUP Oxford.

Vargas, M. (2021). Constitutive instrumentalism and the fragility of responsibility. The Monist, 104(4), 427–442.

Footnotes

As of writing a trial date has not been set in the Mangione case (Katersky, et al, 2025)↩
I am using types of responsibility outlined by Vallor (2023) in the case of AI, but similar relevant discussion can be found in Davis (2010) (responsibility-as-simple-causation and responsibility-as-liability taking the place of causal responsibility and legal responsibility, respectively).↩
Moving forward I refer to these as Pettit’s criteria though he, Christian List and Peter French (1984) have written similarly on the subject.↩
This cascading does not have to be redistributive, as I’ve already discussed some of the issues there.↩
I use the term ‘modern corporation’ to separate the romanticized concept of a corporation from the complex, hyper-connected and legalized institutions present today.↩
In many discussions of corporate agency, employees, contractors, and other people hired by a corporation are considered part of the 'membership' of a company (see Björnsson & Hess, 2017). I support a more constrained definition of membership drawn on the shareholder model of corporate governance dominant today. In this model a company's board of directors and executive management are decision making authorities. Employees, suppliers and communities are stakeholders, but not active members of the decision making process (except in overlapping roles as shareholders or board members).↩
In the cited work, van de Poel, et al. later redefine PMH: “A problem of many hands occurs if there is a gap in a responsibility distribution in a collective setting that is morally problematic.” This better addresses both backward- and forward-looking responsibility practices but deemphasizes complexity of the interactions as a requirement, only specifying that it be in a collective setting and morally problematic. I am using the former definition to highlight the complexity of interactions and the act of being held responsible rather than being morally problematic.↩
A similar case can happen with corporations: an autocratic, single-member corporation is prevented from becoming agential.↩

Moral responsibility gaps: The problem for corporations

2025-02-19T20:59:35.853142+00:00

Five years ago the Business Roundtable, an association of CEOs from the many of the largest American companies, released “Statement on the Purpose of a Corporation.” The announcement was a signal that U.S. corporate leadership recognized the need to move from a shareholder-focused capitalism to a “stakeholder capitalism” undergirded by notions of long-term value, ethical action, fairness and support.

Suppliers, customers and communities are judging the morality of companies through the company’s operating activities, the effects of their products and services, and the actions of their employees. As the Business Roundtable statement shows, corporations are acknowledging the importance of this.

The growing use of artificial intelligence and autonomous systems (AI/AS) has raised new ethical issues for companies. At times the use and development of AI has had detrimental effects on both knowledge workers and workers in developing countries, and AI/AS has profound environmental implications. But there are many examples of how to avoid mass layoffs, exploitation of the global South, and environmental impacts, should a company’s leadership aim to.

AI/AS have placed a spotlight on another set of ethical risks for companies, however, one that is novel for many leaders. The use of AI/AS can create moral responsibility gaps. These gaps can represent a significant risk to companies even if a company and its representatives are acting ethically.

Responsibility gaps exist where autonomous systems act in ways which are not caused, controlled, or predictable to the system’s developers or users. The nature of AI/AS itself causes these gaps to exist, and in their wake they leave outcomes without any reasonable accountable entity.

Below I describe the nature of moral responsibility gaps in greater detail and outline some specific risks the gaps represent for corporations. I then address some of the controversies surrounding the issue, before offering recommendations for mitigating the risks of moral responsibility gaps.

Corporate leaders should take moral responsibility gaps seriously when assessing how to implement or continue implementing AI/AS, and enact mechanisms and processes for mitigating associated risks.

Recommendations include:

Tactics for dissolving responsibility gaps by including people in risky decision making processes
Tactics for identify risks ahead of implementation by pressure testing systems and responses
Frameworks which can limit the impact of identified moral responsibility gaps through culture and resources

What is a Moral Responsibility Gap?

There are many ways we hold each other responsible. In a corporate setting we are often concerned with legal responsibility, who is legally liable for an outcome; causal responsibility, who or what led to an event occurring; and personal or moral responsibility, who or what is accountable for an event or outcome¹.

AI/AS create obstacles for our normal sense of moral responsibility. A moral responsibility gap exists where an outcome occurred which we’d typically have a moral accounting of — a target for accountability — but there is no appropriate target for that accountability.

Consider an autonomous surgical robot designed to perform intricate operations with many different sensory and data inputs and risk assessments determining its behaviors. In a particular surgery the device identifies a risky intervention and takes it. The intervention is unsuccessful and the patient ends up dying.

People are unlikely to find blaming the robot sufficient. There’s not a sense in which the robot could have known not to take that intervention². It was acting as required by its code. Similarly the developer and the user (the doctor, in this case) are not adequate targets for blame, as they did not meaningfully cause the intervention to happen.

As philosopher Andreas Matthias explains, the technology undergirding our AI/AS systems creates this moral opaqueness. In neural networks, for example, a series of hidden layers exist between an input layer and an output layer. The hidden layers are where the AI system has created an intricate system of weights that lead to predefined outputs. The hidden layers are hidden because they cannot be audited or untangled, as they do not correspond to symbolic representations. The weights don’t ‘mean’ anything in a human sense.

These “black boxes” matched with novel inputs and situations can create outputs that neither users nor developers could have predicted.

Corporate leaders generally understand that the moral responsibility for the effects of their products and services begins with the organization. Either with the people who’ve developed the product or service or the company more generally. Moral responsibility can be transferred to users through operations norms, training, manuals and the like (Matthias, 2004). This is how we agree a manufacturer of cooking knives isn’t responsible for morally harmful acts that happen with their products.

The AI/AS black box does not allow for this transfer of responsibility to occur. The user cannot be adequately forewarned on potential risks when a system’s outcomes are inherently unpredictable. Yet it does not seem like we can hold the builder or developers of AI/AS morally accountable either, for the outcomes could not have been predicted or controlled by them, and the calculations that resulted in a negative outcome cannot be traced to actions by the developer. So the moral responsibility gap exists.

Risks

No matter the progress a corporation has made in its transition from shareholder to stakeholder capitalism, the benefits of acting ethically are apparent. Ethical action improves relationships with employees, suppliers and the communities in which a company operates.

Moral responsibility gaps, however, pose risks to corporations even if corporate activities are ethical. When technological tragedy strikes, if there’s no appropriate target for blame, the company itself is likely to be a default foil. Moral responsibility gaps create situations where a company with no justified blame becomes the target of perceived ethical failure.

In the surgical robot example, the manufacturer may be held to popular account for the actions of the robot. Or the doctor or hospital system could be blamed by the public. The opaqueness of the moral responsibility gap creates an uncertainty in who might be a target of blame, even when each person was acting completely ethically.

The risks are more than an ethical roulette wheel, though. Ethical lapses attributed to corporations or their employees present significant financial and reputational risks.

Consumers take a mix of moral and practical considerations into account during purchasing decisions, so corporations could take a financial and reputational hit if faced with a controversy spurred by a moral responsibility gap. Perception of blame, whether deserved or not, could affect consumers’ moral considerations. Similarly, governments around the world have placed tariffs, embargos and other regulations on companies motivated from concepts of national duty, public pressure and safety spawned by moral pressures.

Moral responsibility gaps offer an upside risk to corporations as well. When such a gap exists, the market does not have an appropriate target for praise. These responsibility gaps ensure that some beneficial effects of using or developing an autonomous system could fail to be attributed to the enterprise which developed or used the system. The confusion, or lack of justified moral responsibility, creates the risk of potentially squandering beneficial opportunities for the corporation.

Controversy & Considerations

Philosophers, technologists and ethicists do not agree on how moral responsibility gaps should be resolved, or how they should even be characterized. As Trystan Goetze notes, “It remains genuinely unclear whether computing professionals are morally responsible for the behavior of the systems they develop” (emphasis added).

Some philosophers, for instance, argue that the gaps don’t even exist in a meaningful way. Daniel W. Tigard of the University of San Diego writes that our understanding of responsibility is sufficiently dynamic that technologies like AI/AS do not pose a threat to accounts of responsibility.

Tigard focuses on answerability, attributability and accountability. Requiring that technologies be able to be answerable to their outputs (complicated by the “black box” described above) is misdirected, as we often hold people accountable even when they can’t give good reasons for what they did. Regarding attributability, Tigard notes that there are many mundane examples of when we give moral attributes to objects (like blaming the intentions of your printer when it won’t connect correctly). We have no problem attributing morality to non-humans, he concludes. For accountability, Tigard proposes we can have the same effect of human accountability (e.g. repairing or preventing the harm from happening in the future) by directing accountability to developers or users of the system.

Beyond whether responsibility gaps exist at all, a responsibility gap may be preferable to the alternative. What are we fixing when we close a responsibility gap? Peter Asaro points out that human decision makers often make mistakes, particularly in high-impact decision making situations like we’re describing. It may not be the case that avoiding a responsibility gap leads to better outcomes.

Business leaders too question the benefits of focusing too heavily on the negative effects of AI/AS. In a discussion on responsibility, Tesla and SpaceX CEO Elon Musk said, “if, in writing some article that’s negative, you effectively dissuade people from using an autonomous vehicle, you’re killing people.” The idea being that the slow adoption of even a non-ideal self-driving system would lead to more auto deaths than full adoption.

Recommendations

Even if philosophers and technologists reached consensus, moral responsibility gaps would remain a risk for corporations until a resolution could be broadly justified to the lay person. But the uncertainty regarding moral attribution can be mitigated, if not avoided.

Human-in-the-Loop, Where Possible

Moral responsibility gaps can be dissolved by putting a human in the decision-making loop in the operation of autonomous systems. This sidesteps the problems created by moral responsibility gaps; there are no longer decisions made without human intervention. Human-in-the-loop also limits the autonomy of systems. Teams must analyze the tradeoff between the benefits of autonomous action and risks created by moral responsibility gaps.

Human-in-the-loop comes with its own set of risks, of course. People can get lazy in their relationship with AI predictions, and there’s a risk that human intervention decreases the quality of overall decision making. But ethical risks coming out of these workflows have better structures for moral resolution than situations where a moral responsibility gap exists. It is important for organizations implementing human-in-the-loop as a response to moral responsibility gaps to build feedback loops surrounding the system which prevent the human(s) in the loop from being mere responsibility sinks.

Red Teaming & Crisis Simulations

Red teaming and crisis simulations are one of the most beneficial practices an organization using AI/AS can implement. Red teaming is the practice of gathering a group of experts to simulate malicious, manipulative and unexpected uses of an AI/AS system to identify potential dangers. Crisis simulations gather divisions and teams together to practice responses to crisis situations.

Red teaming can help identify important risks and unintended uses of an AI/AS system, and can be used to inform future safety development of a system. Crisis simulations can also inform structures and workflows around identification and resolution of harmful events resulting from the deployment of a new technology or system. It is beneficial to involve a variety of experts and non-experts from both within and outside the company with both tactics. For instance, in your AI red team consider involving communications and marketing teammates, who have different perspectives on what a risk to the company and its stakeholders might look like than engineering and product teams.

Training & a Culture of Responsibility

Teams should be educated about moral responsibility gaps, how they come about, and what risks they pose. This should not be framed as a transfer or responsibility, but as a tool to better design and implement with the associated constraints in mind.

A culture of responsibility can be strengthened by creating and making resources available to staff. Standard operating procedures defining professional, hierarchical, and personal responsibility create a shared understanding and commitment to responsibility in a corporation. Likewise support mechanisms like ethics hotlines and reporting structures can help identify ethical risks where they appear.

These examples of types of responsibility are drawn from both Vallor (2023) and Goetze (2022).↩
This and following based on requirements for personal responsibility outlined by Goetze (2022). Namely, to be personally responsible an agent must have had control over the outcome, have a reasonable expectation of knowledge that the outcome would or could occur, and have causal responsibility for the outcome.↩

I may have been wrong about strategy

2025-02-19T16:24:35.463248+00:00

For my second career I worked at consulting firms and agencies. I was a ‘digital strategist’. Companies were just trying to understand social media, really getting serious about their online presences, and were wondering how to move their communications and marketing (and more of their business) online.

I got there at the time where just having a website turned out to be not a digital strategy. There were blogs, mobile and social media to think about. (That a mobile website was separate was the thinking back then). People wanted word of mouth marketing, then social clout, then gamification.

We’d go into people’s offices and say, “Look, your website may exist, but that doesn’t mean it’s doing anything. We can use social media to start driving traffic, giving your brand a voice, and making real fans or advocates for what you do. People who will act when you ask them.”

I was a strategist but I didn’t feel like I was doing strategy. Strategy was supposed to be something like innovative or transformational or visionary. I was just solving problems. We’d go back to the office and spend a bunch of a retainer contract writing tweets and Facebook posts.

But it worked. It was valuable stuff. We were really good at guiding clients through change and growth, looking at new technology and figuring out how best to apply it. Often it was workmanlike. Crafting the content, optimizing, learning and writing more.

I eventually began moving up, doing work with bigger scope. Sometime after strategist was dropped from my title I was writing the vision statements, repositioning brands, clarifying mission. This was real strategy. Right?

Right?

Maybe not. In Good Strategy/Bad Strategy Richard Rumelt takes to task the popular idea of strategy.

Vision isn’t strategy. Goals aren’t strategy. What passes for ‘strategy’ today is usually hopes and dreams or objectives floating untethered from reality.

Bad strategy is long on goals and short on policy or action. It assumes that goals are all you need. It puts forward strategic objectives that are incoherent and, sometimes, totally impracticable. It uses high-sounding words and phrases to hide these failings.

There is a large industry of consultants and book writers who are willing to provide instruction on the delicate differences between missions, visions, strategies, initiatives, and priorities. From small boutiques to the large IT-based firms trying to break into strategy work, consultants have found that template-style strategy frees them from the onerous work of analyzing the true challenges and opportunities faced by the client. Plus, by couching strategy in terms of positives-vision, mission, and values-no feelings are hurt.

This is a hard pill to swallow. I can’t count the number of team meetings and all hands where someone’s asked what’s the strategy when they were looking for charismatic leadership or a transformational vision or motivation.

I do it. I was doing it recently. I built a whole breakdown of how to decompose objectives into product vision and strategy for my products.

A flow chart moving through three major sections with subsections. Starts with Organization objectives and divisional objectives. In “Product Strategy” you have product vision, goals, and outcomes. In Discovery you have opportunities, ideas and solutions. In Delivery you have features and stories.

Vision isn’t strategy. Goals aren’t strategy. What passes for ‘strategy’ today is usually hopes and dreams or objectives floating untethered from reality.

A good strategy does more than urge us forward toward a goal or vision. A good strategy honestly acknowledges the challenges being faced and provides an approach to overcoming them. And the greater the challenge, the more a good strategy focuses and coordinates efforts to achieve a powerful competitive punch or problem-solving effect.

A good strategy can be identified by articulating a clear diagnosis of the strategic challenge, a guiding policy leading from that problem, and a plan to carry that policy out through coherent actions. What Rumelt calls the “the kernel.”

Diagnosis: The diagnosis describes what the problem is. Not the full landscape, not the desired future state, but what aspects of the situation are critical.
Guiding Policy: The guiding policy is how we’re going to deal with the problem or diagnosis. What approach can we take (hopefully something that plays to strengths we possess or can attain) to meet the challenges implied by the diagnosis.
Coherent Actions: The steps we are going to take to accomplish the guiding policy. These must be coordinated with one another. In harmony with the guiding policy. I’m a communications-oriented person so the vision → mission → strategy framework makes sense to me. It guides someone from what the organization is trying to do to what the person is doing. The kernel idea is not communications oriented, but it really resonates. It’s pragmatic. It focuses on reality — ground truth — every step of the way. In some strategic frameworks strategy can be unmoored from execution, but by pulling a strategy from a clear diagnosis through to coherent actions, you are creating the framework for execution within the strategic exercise itself.

I read Good Strategy/Bad Strategy only recently, and I’m using some of its tools in my strategy work now. When I do, I think back to my work as a strategist, get in that mindset. We’d work to identify really clear problems with achievable solutions. We’d work within our realm of influence to grasp at and get to those solutions. We’d create a suite of services or tactics that all drove to the same end. Sounds like we were doing real strategy.

So you want to be a product manager

2025-02-19T17:15:02.346481+00:00

Lately I’ve been talking to more people who are eager to get jobs as product managers. How do you become a product manager? How do you become a PM at xyz company? How do I get a PM job?

Here’s some of my advice on the subject. I plan to revisit this every now and then to update it.

What this isn’t is a discussion of what product managers do. That’s different from what people are looking for when evaluating product managers.

As for me, I’ve been a PM for 10 years, and I’ve hired or have been a part of hiring about 20 product managers.

Whether you’re looking to get into product management from another discipline, or you’re entry level, I hope you’ll find some of this helpful.

OK! So, people want to talk about three things with potential PMs:

Strategy/Business acumen
Technology
Execution They’re going to be asking themselves…

Strategy

Does this person understand what we’re trying to do as an organization? Do you understand how this product portfolio succeeds? Will you be able to develop new ideas for getting us there, and more importantly, will you be able to tell a great idea from an OK one? Do you have a sense of balancing priorities in a way that matches our culture?

Technology

What sorts of products or platforms (or websites or whatever) have you been a part of building? How well do you understand what was built? Do you understand the technologies we use and the technologies we’re trying to build? Do you understand the risks and benefits of using different technologies here?

Execution

Can you work with teams to get things done? Can you do it consistently, and on timelines that work with our organization? Can you get things done with shifting priorities, resource allotments, and politics? Can you do all that and ensure the outcome is high quality? Without role power?

As they’re assessing these, the people who are deciding whether you can become a PM on their team are also going to be looking to hear:

This person can research. You know how to get answers (or close to answers) in situations where knowledge/information doesn’t exist or is ambiguous.
This person can communicate. You understand how to understand people and get them to understand you. This includes communicating with different types of people and people with different expertises. Most of the time you have to communicate with people who know more about what you’re talking about than you do.

If this sounds easy

Let’s say you have a PM role in mind and you are confident you can speak on these things. That’s great. What you need to do is talk about it in the language of results and **examples✱.

These are different. Results are the measurements, the metrics, or the change in the world that happened because you did the project. Examples are the artifacts in the world where people can see or use what you’ve done.

Where you can’t share examples (say, most of the work you want to talk about is transient, inaccessible, or abstract) focus on results. Use numbers and put those numbers in context.

Where you can’t share results (e.g. the results are confidential or incomplete) try to show examples. Screenshots work, but the actual thing is better.

Hopefully you have a mix of both, and if you have examples and results for the same project so much the better.

If this sounds daunting

That’s OK too, no one was born having all this under their belt. If you feel you’re not there yet with most of the questions above try these things.

Self-directed learning

Take some of the questions above you’re interested in and hit the books, listen to the talks, take courses. Learning alone isn’t going to get you the results or examples you’re going to need during an interview. It is essential, though, for getting you comfortable with how the product business works.

Form opinions on what you’re learning. Make connections to the work you do or have done. Write all this down.

Get the job that gets the job

You may not be in a PM role now, but you can move closer to product management by looking for jobs with organization similarity or role similarity.

Org Similarity

Roles with org similarity are the jobs in companies with strong product teams that work closely with the product team. Think engineering, marketing, design, customer service, content development. If you have the experience to land a job on a related team, you can be better positioned to transition to a PM role. I’ve seen many people move internally from their team to product management group. They’re the ones who’ve worked with and gained the respect of the product teams and leadership.

One of the most successful PMs at the last company I worked at was in product marketing when I got there. There was some turnover in the corporate product vertical and she jumped at the opportunity. She’d been been marketing the product for years already and knew the team, the product back and forth, and how to get things done at the company. She moved over to lead that team as a senior director and not only rocked the role but brought a knowledge with the customer that was really fantastic.

Role Similarity

While they may not have the ‘product manager’ as their title, digital strategists, business analysts, and digital project managers do work like those of product managers. With a role like this you can start building the results and examples that will help you fill in the rest of the story. There are many titles like this out there.

This is how I got into product management. I’d been doing strategy and client management in consulting and digital marketing agencies for a chunk of my career. When I broke into product management I found my stakeholder management, systems thinking and a broad knowledge of different technology solutions were exactly the same muscles I needed for product management. I was fortunate the hiring team saw the overlap too.

Build something

Build something digital, or diagram something. Share it. Try building a website. If you’re comfortable doing that, build something more complex. Create a topic-oriented Instagram or design a system for making decisions. Show it to people, get feedback, and improve it. You’ll find it’s fun (or you’ll hate it in which case you may not like being a product manager).

You don’t have to build a native iPhone app with the goal of getting VC funding, just start where you are.

But if that sounds interesting you could also take the entrepreneur’s path.

Give yourself the job

Make a company of your own and name yourself head of product or head of everything. Do all the things you think are fun or interesting about being a product manager. Start to make some money. See if you can make the company go. Can you make a company that can support you? Can you create a company that can employ multiple people?

A lot of successful product managers at big companies were founders before they ever held a PM job title.

So think about it. You probably don’t have to do all this, but doing more than one of these will help more than doing any alone. Consider what interests you about becoming a product manager and explore that.

And good luck! The world needs more good product managers. PMs who have experience in other disciplines have an edge in understanding how things work (if they have a handle on the strategy, technology and execution, of course). You probably already know more than you think.

I’d love to hear if you put any of this into practice, which parts resonated and what fell flat. Let me know!

What are responsibility gaps?

2025-02-18T15:51:53.858898+00:00

A COMPUTER CAN NEVER BE HELD ACCOUNTABLE

THEREFORE A COMPUTER MUST NEVER MAKE A MANAGEMENT DECISION

Computers are good at making decisions. Throw some criteria together, some weighting, put in some initial conditions and they’ll tell you what comes of all of it. Some decisions are really complicated for humans but really easy once you’ve programmed it all into a computer. Things like keeping a nuclear power core within a specific temperature range or deciding what should come up first in a search engine.

Someone at IBM was taking a principled stance on the decisions we should offload to computers, saying that a computer should never make management decisions. Why? Because a computer cannot be held accountable.

There are some decisions that seem like we need some human responsibility behind. Things like whether we should fire or promote Jim, or where should we go on vacation. Sure, with the right criteria we could trust the computer’s output, but there’s something human about these decisions that wants for a human agent to take responsibility.

First, let me make one thing clear, computers aren’t ‘making decisions’ or ‘choosing’ like a human is. Computers, software, AI systems aren’t agents as we think about people being agents. It’s not instinct or deliberation or anything anthropomorphic that results in their outputs. So they can’t be responsible in a way that we consider people can. They can’t be morally responsible in particular.2

We’ve reached a point with computers, though, where we can’t trace responsibility as clearly as we could in the past. This seemingly creates responsibility gaps, where a moral outcome occurs, one which intuition says should have someone we should praise or blame for it, but no one seems to be a good target. Computers (which I’ll just use as shorthand for all kinds of technologies including software, automata, systems programmed under the techniques of computer science) are being programmed in such a way that the outputs and paths that produce those outputs were not created by the programmer, and could not be predicted by the programmer or the user.

The responsibility gap

Andreas Matthias wrote a great paper (pdf) that really opened the door to discussions on responsibility gaps.

We have pretty good methods to trace responsibility concerning technology. For most tools and technology, responsibility for harms or benefits falls either on the manufacturer or programmer (the builder) or the user. If a technology is meant for a certain task, it should do that. If it malfunctions and causes harm, the builder is responsible. The way we generally transfer responsibility from builder to user is through an operations manual or operations norms. If a user operates a technology against the rules by which it’s meant to be used, the user is responsible for any harmful effects.

The ‘manual’ doesn’t have to be explicit. Matthias uses the example of a candle. If a candle user sets a lit candle close to a curtain and burns the house down, that doesn’t seem like it should be the builder’s fault. It’s generally accepted that one rule of using candles is to be careful about such things. If the user of a candle lights it and it explodes, the builder bears the blame and responsibility. The candle isn’t working as expected.

But! There are new technologies that muddy this. Neural networks are designed to mimic the connections in a biological brain. These networks are made up of nodes in three or more layers: an input layer, one or more hidden layers, and an output layer.

We can figure out what is in the input layer and the output layer, but the nodes and connections that make up the hidden layers are, well, hidden from the builder and user.

You might use a neural network to help identify photos of milk. The neural network would take data comprising the photos in its input layer, and in the hidden layers work to make connections filtering out any but the milk photos. How it would identify the photos would be on criteria weighted node by node.

The network must be trained to set its weights. The builder might feed it a few million photos of milk and a few million photos of non-milk that it would use to create the criteria and weights for deciding what’s milk and what’s not. The hidden layers are hidden because there isn’t symbolic information stored in the system, just weights between nodes. No third party can go back and parse what pathways were taken to reach the output. This is the ‘black box’ you hear about some AI technologies.

Systems that learn by adaptation and reinforcement are similar. They are programmed to try different solutions, receive feedback, and alter how they work so they consistently get the right answers. These systems can optimize in dynamic environments, but they must do so by trial and error. The system is designed to get things wrong sometimes in order to improve its parameters. Again the parameters aren’t predefined by the builder, so what criteria the system uses to create its outputs aren’t predictable.

So sometimes, one of these systems can produce an outcome that was

Not predictable by the builder, and
Not the effect of any user’s actions or intentions If the outcome is harmful (or beneficial) where does the blame (or praise) lie? The system itself isn’t an agent that can be held responsible. “A computer can never be held accountable.” And since neither the builder nor the user had control over the system’s output, it doesn’t seem like it’s right to hold either of them responsible.

This is a responsibility gap, as there isn’t any agent or agents who are an appropriate target for responsibility.

Are these gaps novel?

My first thought when talking about responsibility gaps is that hey, there is no responsibility gap. If you built it you are responsible for what it does. When the user is using something correctly, the responsibility for harms are on the builder. Easy enough.

But looking deeper, the builder really doesn’t have the control that we associate with responsibility. Sure, they created the system, and for a purpose, but when and why the system gets the answers wrong, potential emergent capabilities or seeming capabilities are unpredictable.

This isn’t new. Corporations are systems that produce unpredictable outcomes and develop unpredictable properties with moral implications. When corporations fire people, tens of thousands of people, say, it creates real pain and suffering for real people. Society hasn’t really agreed upon if a person or people bear responsibility there or not. When a company produces morally good outcomes a CEO or leader will associate themselves with it, but responsibility for mass layoffs is obfuscated, at least rhetorically. It’s an outcome of the system, not the responsibility of the leaders of the company who built or manage the corporate system, or of the people who ‘use’ the system by being employed.

Ethicist Daniel Tigard looks into the mechanics of responsibility gaps at a more personal scale and concludes that the gap doesn’t exist. Responsibility is a complicated concept, and in non-technological situations we navigate similar obstacles in accountability and responsibility without admitting an unbridgeable gap. Holding responsible can be communicative, it can be reparative, preventative. Tigard argues that our concept of holding responsible doesn’t rely on control. With a wider definition of responsibility we hold animals responsible for their actions, we hold technology responsible for dropping our calls, not printing our papers. We even hold people responsible for things outside of their control, or for actions that they can’t give good reasons for (due to something like implicit bias, say).

The question about responsibility in regard to AI technologies, then, wouldn’t be who is responsible, but how are our actions of holding responsible going to improve future outcomes? As communities, we can negotiate and decide what guidelines, obligations and consequences will best prevent the harms that are important to us and promote the benefits.

So the IBM’mer who says a computer can never be held accountable isn’t exactly right, but they’re starting an important discussion. How do we want to govern our relationships with technology?

If we want to codify that technologies can’t be held responsible, then we’ll need agreements on fallbacks in ‘responsibility gap’ situations like these. If we want to allow for some flavor of responsibility for these systems, we’ll need to specify what remedial or preventative measures we develop, and who bears the brunt of that.

Computers don’t have to be agents to be part of the community behavior of holding responsible. It seems like if we’re clear about how we manage responsibility, we can govern these systems and the agents involved fairly.

I looked hard to try to find the whole deck, but could not. It looks like this slide first appeared on Twitter in 2017 from user @bumblebike. According to them the whole deck was lost in a flood before they got around to scanning the rest. They had one more slide to share here. If you know more contact me.
It’s hard to talk about the activities of computers without using terms we usually use for human action. I’m not trying to say that computers choose or think or act. They dryly have processes that produce outcomes.

Linked in this post:

Matthias (2004): The responsibility gap: Ascribing responsibility for the actions of learning automata (pdf)

Tigard (2020): There is no techno-responsibility gap

@bumblebike: IBM slide tweet

Succession & the art of ending

2025-02-19T18:40:27.060019+00:00

⚠️ Heavy spoilers ahead for the Succession series finale. ⚠️

The Succession series finale aired last Sunday. By most accounts the creators nailed the landing. I thought so too! The show ended coherently and truly. It wasn’t satisfying, though. It didn’t try to tie up every loose end like a Game of Thrones. That’s ok.

Despite all their maneuvering, none of the Roy children ended up running Waystar Royco, the company their father started. Roman sat at a bar, smiling at his insignificance. Kendall watched the Hudson River, still unable to control forces of nature. And, after betraying her brothers, Shiv sat in the back of a car with her husband Tom, who had been named the empty suit successor to Logan Roy.

I really appreciated the ending. Aristotle says the ending of a tragedy should be necessary, it must be inevitable. The actions leading up to the ending define the ending itself such that there’s only one way it could go.

The Succession writers had set up the rules of the world and the ending didn’t bend them at all. Who was best or strongest (or worst) didn’t matter in the end. The structures of power depicted in the show are self-protecting, and reject chaos. Everything the Roys do, that happens during the show, ends up blips on the stock price. The season-long DOJ investigation into the a cruise line scandal peters out and ends in a settlement. The Roys even bet on the stasis of the system when they announced early the results of the election. They relied on the other news orgs following suit with their call; that the system would correct to keep chaos down.

When it came time for a successor to Logan Roy to be chosen, the kids were outside the system, pushing against it. The system chose stasis.

John Gardner, in The Art of Fiction, describes two ways a story can end: with resolution or repetition.

By definition-and of aesthetic necessity-a story contains profluence, a requirement best satisfied by a sequence of causally related events, a sequence that can end in only one of two ways: in resolution, when no further event can take place (the murderer has been caught and hanged, the diamond has been found and restored to its owner, the elusive lady has been captured and married), or in logical exhaustion, our recognition that we’ve reached the stage of infinite repetition; more events might follow, perhaps from now till Kingdom Come, but they will all express the same thing-for example, the character’s entrapment in empty ritual or some consistently wrong response to the pressures of his environment. Resolution is of course the classical and usually more satisfying conclusion; logical exhaustion satisfies us intellectually but often not emotionally, since it’s more pleasing to see things definitely achieved or thwarted than to be shown why they can never be either achieved or thwarted. Both achievement and failure give importance to the thing sought; we can feel about it as we feel about values.

Resolution is more satisfying. In Succession, there’s a sense that Roman gets resolution. He recognizes his situation is ‘bullshit.’ Kendall and Shiv, though, are doomed to repeat, and will continue to move and push against their situations.

But it’s repetition. The future sociopathic adventures of Kendall, Shiv and Tom are not part of the story of Succession.

The question of the show was answered, and that’s where we were left. It wasn’t satisfying, but it didn’t need to be. The tragedy of the Roy siblings was that they were only ever creatures of ambition.

Augustine's Confessions, Book 1

2025-02-19T19:25:54.200063+00:00

It takes so much time to learn, but it’s worth it, right?

You know, maybe not.

I’m reading St. Augustine’s Confessions, and though it was written 1600 years ago, the way he was taught doesn’t sound all that different from today. (Less focus on STEM.)

The way he describes how he learned, and why, is a lot like what children are taught today. The effects are similar, too: they’re disastrous. To Augustine, we veer way off course regarding what’s really important when teaching, learning and seeking excellence.

It starts like this. St. Augustine wasn’t born a saint (or maybe he was, I don’t know how that works), he was born a baby. Babies can’t talk, but they make up for that in being very mean. Their meanness gets them what they want until that time they learn to talk.

Being weak, babies’ bodies are harmless, but babies’ minds aren’t harmless. I myself have observed (carefully enough that I know what I’m writing about) a tiny child who was jealous: he couldn’t speak yet, but his face was pale and had a hateful expression as he glared at the child who shared his nurse. Who doesn’t know that this happens?

In Book One, Augustine sets up a process of learning, which starts with learning to talk. Learning to talk comes through repetition and sheer force of desire to get what one (as a baby) wants.

In due course, when I had heard words often in their proper places in a variety of sentences, I gradually deduced what they were symbols for; and once I had tamed my mouth and made it use these symbols, I could announce my wishes through them.Thus I began to share with those around me the symbols for making wishes known, and I ventured farther from shore on the stormy sea of our common human life-depending on my parents’ authority and the power of people older than myself.

After that, teachers take over. He gets to school and first learns how to read and write, and second learns foreign languages (Greek), literature and rhetoric.

So the progression of learning in Augustine’s society goes like this:

Learning to Talk → Becoming Literate → Literature/Foreign Languages

This is a bad way to order and organize learning.

First, Augustine argues, literacy is a higher skill than being book smart. People don’t just forget how to read and write, even after not doing it for months. People all the time forget the important parts of stories and speeches.

If I were to ask which it would be a greater drawback in this life of ours for any given person to forget, reading and writing or those poetic fairy tales, who (unless he’d forgotten his own existence, i.e., was brain-dead) wouldn’t see what the answer needed to be?

Secondly, what is it that we’re trying to do with all this third-stage learning? It’s not what’s most valuable to Augustine, that’s for sure.

Throughout Book One, Augustine sets up a second progression, that of the value of learning. It goes something like

Getting what you want → Understanding the world → Being correct and skillful

This is how it works in the world, at least. It’s not the best way. Beyond learning to read and write all teachers care about is being correct and skillful. You’re meant to value this above all else too, and that’s a bad focus. There’s no consideration for whether what you did is true or good.

If someone who upholds and teaches those ancient tenets should, against the rules of the language, pronounce the word homo, or “human being,” without an aspiration in the first syllable, as ‘omo, he would offend other human beings more than if, in violation of your decrees, he hated a member of the humanity to which he belongs.As a mere boy, I sprawled like a forlorn lover on the threshold of such ethics. In this arena, on this wrestling floor, I was more wary of making a mistake in pronunciation than of envying people who didn’t make one when I did.

Teachers, like in Augustine’s language class here, emphasize the value of the quality of your speech and pronunciation more than guiding you on whether what you’re saying is moral.

So we’ll revise our progression on the value of learning. It should be something like

Getting what you want → Understanding the world → Being correct and skillful Being true and good

That, of course, is not how curricula were set up in the 400s, and maybe less today (more STEM). So learning in school starts as a positive, and looks like it’ll continue to be a path toward excellence or what’s Good. But we veer off. This focus on correctness can actually drive you away from what’s good and true. And to Augustine, away from God.

Well shit. Where does that leave us? I’m not in school, but I do enjoy reading and learning, and I wear learning around as some sort of virtue. Is it? It doesn’t sound like it.

Throughout Book One, Augustine wrestles with this as he recounts his early life. How could it be that what he thought was the right path, what adults told him was valuable, could be so completely blind to the actual path — knowing and becoming closer to God?

Today it’s well worn to think ‘yeah, what society thinks is right probably isn’t.’ But Augustine’s compact dissection of book learning in Book One is fresh, even one and a half millennia later. What is it that we’re doing here?

FOOTNOTE: I’m reading Sarah Ruden’s translation of Confessions, and it’s outstanding and lively. Pick this one up.

Bat Story

2025-11-20T14:20:50.120823+00:00

I heard a story once about a group of scientists who went down to Hill Country in Texas. In that part of Texas they have caves with huge amounts of bats. In one of these deep caves hundreds of thousands of Brazilian free-tailed bats roost together. Each evening at dusk they come flowing out of the cave mouth, looking for bugs to eat for the night.

The scientists had asked themselves, how do all these bats pour out of the cave without bashing into each other? How do they figure out how not to run in to the other bats leaving at the same time without injuring themselves or each other?

The scientists set up super high-speed cameras to film the bats as they left the cave over a few nights, then they took the footage and analyzed it with 3D software to model each bat as it moved through space.

You know what they found? The bats run into each other quite a bit.

Bats haven’t found some great way to analyze where their friends are around them and compensate so no-one touches. They’ve just gotten really good at readjusting when someone bumps them off track.

I think about this when I’m trying out new processes and setups at work. Even when everyone’s trying to go in the same direction, there can be a lot of jostling and getting in each other’s way. It’s tempting to make lanes rigid and standardized to ‘solve’ this, but sometimes it’s better just to make it easier to get back on track.

See for yourself: Bat Ballet: Slo-mo footage reveals how thousands of bats emerge from a cave without injury

UX Rules Beyond the Web

2025-11-20T14:24:12.755703+00:00

There’s a vibrant industry around user experience (UX) thought today, but as UX moves beyond flat screens we’re finding that a lot of the best practices and known methods are too specific. What applies to call-to-action styling and page navigation on desktop web and mobile is often totally inapplicable when interacting with Alexa or Siri, or diving into virtual reality headgear.

Early UX and interface research dealt with the same problem. When Jakob Nielsen, now head of Nielsen Norman Group, was at Bellcore in 1994, he published a report called Enhancing the Explanatory Power of Usability Heuristics (pdf).

The report explores usability and user experience heuristics. Heuristics in this case are generalized methods to solve groups of problems – not specific solutions but ideas and processes you can fall back on as rules of thumb.

Importantly, Nielsen and team assessed heuristics for telephone interfaces and text-based environments as well as graphical user interfaces (GUIs). GUIs, back then things like Windows desktop environments, have a huge overlap with web design. The idea was that problems that arise between GUIs and text-based interfaces, like DOS, and telephone interfaces are more generalizable as overall user experience ‘rules’.

Nielsen started with a long list of usability fixes, and pared down which of these solutions are most useful. To figure out which heuristics are most important, Nielsen split out results into the heuristics which solve

The greatest number of UX problems, and Those that are most effective at solving serious problems

I’ve been keeping these lists at hand and referencing them often as I look through projects we’re working on. They’re useful for getting a broader view of what we’re looking at, and help us solve larger numbers of problems with reproducible techniques.

Gettin' Jiggy Wit It

2026-02-05T02:00:08.265214+00:00

Phillip Herndon · Gettin' Jiggy Wit It