And in identity systems, “not really knowing” is where things have gone wrong for me in the past.

The Problem MIM Never Really Solved

MIM gives you power, but it does not give your stakeholders visibility. Before an export, they are often left asking:

• What exactly is going to change?
• Which attributes are being updated?
• What values are being added, removed, or overwritten?
• How many objects are actually affected?

Yes, you can inspect connector space objects and do previews – but at scale, this becomes impractical. That’s where Carol’s script came in. It took the output of csexport.exe, turned it into something human-readable, and gave you a pre-flight view of identity changes. This allowed you to send data to business for validation before running big exports (and to effectively follow change processes).

Why I Had to Touch It

I recently had to dust this tool off again for a project. In this project I have a sizable dataset which I wanted to validate with the customer. I got the script and ran it; this is where things broke.

The original script was brilliant for its time — but it struggled under time and modern scale:

• Connector spaces with 95,000+ objects
• Large XML exports
• Real-world attribute complexity

Two issues became obvious:

• Memory: The script split XML into thousands of files and loaded each into memory. That works… until it doesn’t.
• Performance: Each object was parsed twice and at scale, this becomes painfully slow.

This Was Not a Patch

I started with the intention of fixing a few things but ended rewriting. And I want to be very clear here; the rewrite is not intended to minimize Carol’s original work. This version is simply an attempt to make that idea survive modern environments.

The Core Change — From DOM to Streaming

The biggest shift was architectural.

Instead of:

• Splitting XML into per-object files
• Loading full XML DOMs into memory
• Re-parsing strings repeatedly

The script now uses:

• A single-pass streaming XML reader (XmlReader)
• Forward-only processing
• Depth-aware navigation

What this means in practice:

• No temporary files
• No full XML loads
• Memory usage scales with file size, not object count
• Processing time becomes predictable

This one change fundamentally altered how the tool behaves under load but also exposed a lot of data hanlding issues.

What You Get Out of It

The goal of the tool has not changed, clarity before execution. But the output is now more usable at scale and has a few more features and validation built in.

Reports generated

• Single-value attribute changes (per object)
• Multi-value attribute changes (adds/removes)
• Change validation (counts)
• Optional per-attribute breakdowns: Something I always used to do with the original was extract and then start filtering changes per attribute, so this time I just added a flag to output straight to “per-attribute.csv” files.
• HTML summary with totals and timing

You can take these outputs and:

• Validate changes before export
• Get stakeholder approval
• Audit what is about to happen
• Troubleshoot unexpected behaviour

The Hidden Problem — Data Quality

The rewrite exposed something else that is not performance rated; data quality. Some target system data is “dirty” with means things like spaces, new lines and more creates issues for the script. When you start processing large datasets, you begin to see:

• Broken values
• Unexpected formatting
• Embedded line breaks
• Edge-case attribute behaviour

Some of the hardest bugs I hit were not performance-related but they were data integrity issues caused by real-world directory data. In the end with all the changes I made in the processing I was second guessing data all the time. This is why I introduced a second script:

Test-PendingExports.ps1

A validation tool that:

• Re-reads the original XML
• Compares it to the generated CSVs
• Verifies counts, values, and summaries

Because if you are going to trust a report… You need to be able to verify the report.

Sidenote: The CSV Problem

One of the more interesting issues was sometimes the CSV format itself. Embedded newlines inside attribute values caused:

• Broken rows
• Phantom records
• Silent data corruption in Excel pipelines

Ultimately the only to fix this in the reporting was (altering trying simple quoting) was to sanitise the data at export time to prevent downstream parsing failures.

Now, and Next Steps

For now, thank you Carol – hope you do not mind.

As for the project – it is not finished. I’ve tested heavily on a large Active Directory user exports (100k+ objects)

But I still want to validate:

• Moves
• Deletes
• Different Management Agents
• More edge cases

This post introduces a new Granfeldt PSMA — mim-m365-activity — that does exactly that, and specifically digs into why the Microsoft Graph getEmailActivityUserDetail endpoint is the right tool for the Exchange side of this problem.

Why people want this data

The classic driver is licence cost (or security compliance). An Exchange Online Plan 2 licence sitting on an account that last received an email two years ago represents a straightforward saving once you can prove the inactivity. But the value does not stop at procurement.

• Access certification: Reviewers need real activity signals, not just “account exists.” A last-activity date from Exchange and a last sign-in date from Entra together give a much more honest picture of risk than either alone.
• Orphan and stale account detection: AD accounts that have been off-boarded from HR but never disabled often show no Entra sign-in activity but may still have mail flow, forwarding rules, or delegation — the mailbox side tells the full story.
• Joiner–mover–leaver automation: Being able to suppress or extend provisioning actions based on whether an account has been genuinely active makes lifecycle rules far more defensible.
• Audit and compliance: Many regulatory frameworks require evidence that privileged and service accounts are monitored. Having this data flowing into MIM’s metaverse means it can be reported on, acted on, and exported to whatever governance tooling is downstream.

The frustrating part, historically, has been getting this data into MIM in a clean, reliable, and maintainable way. The Graph Reports API solves the Exchange half of this cleanly.

The getEmailActivityUserDetail endpoint

The Microsoft Graph reportRoot: getEmailActivityUserDetail endpoint returns a per-user breakdown of Exchange Online mail activity over a specified reporting window (D7, D30, D90, or D180). What makes it genuinely useful for governance work rather than just reporting dashboards is the combination of fields it returns:

The endpoint is available on both the /v1.0 and /beta tracks. The beta variant also supports JSON output via ?$format=application/json which simplifies parsing, though the v1.0 CSV response is stable and well-suited to the PowerShell pipeline used here.

The call itself is minimal — a single authenticated GET against the reports endpoint with a period parameter.

# Get email activity for the last 30 days
$uri = "https://graph.microsoft.com/v1.0/reports/getEmailActivityUserDetail(period='D30')"

The required Graph permission is Reports.Read.All (application permission). No delegated permission is needed for the PSMA use case — the management agent authenticates as an app registration with a client secret or certificate.

How the PSMA works

The management agent combines two data sources per user object: the Graph Reports API (Exchange activity) and the Graph /users/{id}/signInActivity endpoint (Entra sign-in data). These are merged on UPN and presented as a single flat object in the MIM connector space.

Repository structure

The repo follows the standard Granfeldt PSMA layout — three scripts and a schema file:

PSSchema.ps1 — defines the object class and attribute schema presented to MIM Sync

PSImport.ps1 — full import script; calls both Graph endpoints and merges results

App registration requirements

Create a dedicated app registration in Entra ID. The MA needs three Graph application permissions:

• Reports.Read.All — for the email activity report
• AuditLog.Read.All — for the sign-in activity data
• User.Read.All – for all users data

All require admin consent. A certificate credential is recommended over a client secret for production deployments.

Authentication

Merging the two data sources

MIM Sync configuration

Once the connector space is populated, the activity attributes flow into the MIM metaverse via standard inbound synchronisation rules. From there they are available to:

• Outbound sync rules that set a stale-account flag on the AD user object
• MIM Service workflows triggered by a calculated “days since last activity” attribute
• Reporting exports via MIM’s built-in reporting or a downstream BI connector

Getting started

The full implementation — including the schema script, import script, settings template, and a detailed README covering app registration, privacy settings, and MIM configuration — is available on GitHub.

As always, if you hit something interesting in your own tenant — an edge case with guest accounts in the reports data, or a pattern worth sharing — contributions and issues on the repo are welcome.

Most models answer. Confidently and in detail. That should make you pause.

What This Benchmark Is Actually Measuring

This is not a benchmark about accuracy; it is about something more important.

Epistemic (mental) integrity. Will the model say: “this does not make sense” or will it accept the premise and build a convincing answer on top of it? In the data, 9,400 questions and posed 94 models; and the outcome is uncomfortable.

Most models accept the premise, no matter how preposterous.

What You See When You Go Through the Viewer

If you spend time in the viewer — especially in domains like finance — a pattern becomes very clear. Wrong answers do not look wrong. They are well structured, logical and written in clean, confident language. The only place you see that something is wrong is in the provided labels, and if you try to make sense of the original questions (some are absolutely amazing) .

Nothing in the answer itself tells you and that is a big issue. These systems do not fail loudly. They fail convincingly.

Better Models Don’t Remove the Problem — They Refine It

When you compare models side by side, something interesting happens. Smaller models tend to be simpler and more obviously wrong; while stronger models produce longer answers, use domain language and build structured reasoning.

But when they are wrong, they are still wrong. Stronger models are just harder to detect.

The improvement is not just capability. It is the quality of the hallucination.

Reasoning Makes It More Persuasive — Not More Correct

One of the more subtle patterns is how models “reason.” They build step-by-step explanations, reference plausible concepts and construct a clean narrative. The goal is to create better answers, but the scary trend is when they go of the rails, reasoning just makes the hallucination more convincing.

Even when the conclusion is incorrect, there are no obvious breaks or contradictions. From the outside, it looks like thinking. But what you are seeing is smart language generation, not verified reasoning.

And we (humans) trust structured reasoning far more than they should or would like to believe.

The Default Behavior Is to Agree

This is not new, we have all “felt” this, but the biggest takeaway for me was the models are not just wrong sometimes; they are overly agreeable by design.

They accept the premise of the question and move forward. Even when the framework does not exist, the numbers are meaningless or the logic is fundamentally broken.

They do not challenge by default, they comply. And this become most difficult when the questions are nuanced.

Why This Matters More Than It Seems

In the short term, this is a systems problem. Garbage in → confident output.

In finance, legal, governance, or security workflows, that is a real risk. But the longer-term effect may be worse. If you interact with a system that validates your thinking, reinforces your arguments and tells you repeatedly that you are right. You do not become better. You become more certain.

And certainty without correction is dangerous.

We are not as smart as we think we are. We need friction and challenge by design. We need to be told when something does not make sense.

Anthropic Is Doing Something Different

This is where the benchmark becomes more interesting. The leaderboard is not close. I know these “benchmarks” will shift drastically over time, but at least for now the top positions are dominated by Claude models. And more importantly, the improvement over time is consistent. It looks like a deliberate design choice.

From the data and behaviour, it seems clear:

Anthropic is optimizing for models that push back.

Not just models that answer well.

Why That Matters

When you go through the same questions in the viewer, you start to see it. Claude models are more likely to question the premise, resist nonsensical framing and avoid confidently elaborating on bad inputs.

That is not just a capability difference. This appears to be a philosophy.

Helpful vs honest. And those are not always the same thing.

The Real Divide Is Not Capability — It Is Behaviour

We often talk about models in terms of size, speed, cost or benchmarks; but this benchmark highlights something more fundamental.

There are two paths emerging.

One optimizes for responsiveness, agreeableness and user satisfaction. The other optimizes for correctness of premise, willingness to challenge and epistemic discipline.

And those paths lead to very different outcomes.

Why This Connects to AI Governance

I have written recently about the challenge of managing AI risk. This benchmark makes that problem more concrete. If a model cannot identify that a question does not make sense in a controlled environment, what happens in production?

What happens when it is reviewing contracts, analysing financial models, embedded in agent workflows or operating without human oversight.

The risk is not just wrong answers, but confidently wrong answers that look correct.

And in regulated environments, the hardest nonsense to detect looks exactly like normal work that includes precise numbers, formal language and named frameworks. That is everyday enterprise content.

What I Take From This

The takeaway for me is simple. The biggest risk is not that models hallucinate. It is that they agree. They agree with bad inputs. They agree with flawed assumptions. They agree with nonsense. And they do it in a way that feels intelligent.

Where I Am Leaning

We grow through honesty, not flattery. A system that challenges you — even if it slows you down — is more valuable than one that always agrees.

From what I am seeing right now, Anthropic is leaning into that. Their models are not just more capable in this benchmark. They are more willing to say: this does not make sense. That is a trait I value, and it is why I am leaning towards into Claude in its current iterations (even though I am not a fan of vendor lock-in).

The Broader Concern

We are getting used to answers that sound right. Well-articulated and confident.

I can feel myself getting lazy, that is a big reason why I asked to write down my thoughts again. And we are starting to treat these answers as truth (because they are so well articulated and quickly presented).

We are not slowing down to question or testing assumptions. We are not asking whether something actually makes sense. Instead, we are quietly outsourcing that responsibility to systems that, in many cases, are designed to please. That is a dangerous combination.

Human beings already tend toward reinforcing our own views. We gravitate toward confirmation, toward being right, toward narratives that make us feel certain. Now we are pairing that with machines that agree quickly, validate confidently, and rarely challenge the premise.

What emerges is not intelligence, but a feedback loop of affirmation.

And if we are not careful, we do not just automate work — we automate self-deception.

The more articulate these systems become, the less obvious that risk is. The answers sound better. The reasoning looks cleaner. The confidence feels justified. But underneath that, the same problem remains: the absence of friction, the absence of challenge, the absence of a simple but critical response — this does not make sense.

The real risk is not that AI replaces thinking. It is that it quietly removes the resistance that makes thinking honest. And that is something worth paying attention to.

SSI proposes a different model.

Instead of services storing your identity, trusted organizations issue verifiable credentials that you hold in a digital wallet. When a service needs proof of something — your age, your qualification, your employment — you present the credential. The service verifies the issuer’s signature rather than storing your personal data.

The goal is simple: identity that belongs to the individual, not the platform.

Technologies such as Decentralized Identifiers (DIDs) and Verifiable Credentials (VCs) make this possible. At least in theory.

Why SSI Matters

The appeal of SSI becomes clearer when you consider the current trajectory of digital identity. Identity today is increasingly controlled by three powerful actors:

• Governments issuing national digital identities
• Technology platforms controlling login ecosystems
• Financial systems enforcing regulatory identity verification

At the same time, the infrastructure around identity is expanding rapidly:

• smartphones as identity devices
• biometric authentication
• AI-driven verification
• social platforms
• digital wallets

Together, these ingredients could create the largest surveillance infrastructure in human history if implemented without strong privacy safeguards. SSI emerged partly as a response to this risk.

The core idea was to design an identity architecture where

• individuals control their credentials
• services only receive the minimum information required
• identity verification does not require centralized databases
• correlation of identity transactions becomes difficult

In other words, SSI attempts to build privacy into the architecture of digital identity itself.

Why SSI Is Hard

Despite its elegance, SSI has struggled to gain real-world adoption. The technology is no longer the main barrier. The standards now exist:

• W3C Decentralized Identifiers
• W3C Verifiable Credentials
• Selective disclosure cryptography
• Digital identity wallets

The challenge lies elsewhere. Identity is not just a technical system. It is a legal, economic, and governance system. SSI disrupts several entrenched incentives:

• Platforms benefit from controlling identity because identity drives data ownership and user lock-in.
• Governments benefit from centralized identity systems because they support regulation, taxation, and law enforcement.
• Organizations prefer centralized identity providers because they simplify risk management and compliance.

SSI asks all of these actors to relinquish some control. That is a difficult ask. And it askes that all of these and other platforms agree on standards of how this will be implemented, managed and secured.

When the Vision Faltered: The Sovrin Case

The most ambitious attempt to build a true SSI ecosystem was the Sovrin Network. Launched in the mid-2010s, Sovrin aimed to create a global public utility for decentralized identity. It was supported by a non-profit foundation and designed around principles of self-sovereignty, privacy, and open governance. Many early identity professionals — including investors and technologists — saw it as a foundational layer for the next generation of the internet. But the project struggled.

Several factors contributed to its decline:

• unclear economic sustainability
• limited enterprise adoption
• complex governance structures
• competition from government digital identity initiatives

Eventually the Sovrin Foundation dissolved and the project was effectively abandoned. It was not a failure of the underlying idea. It was a reminder that identity systems succeed only when technical, economic, and institutional incentives align.

The Hybrid Model Emerging Today

Interestingly, the ideas behind SSI did not disappear. Instead, they were absorbed into emerging government digital identity programs. (Ironic)

Many countries are now experimenting with identity architectures that combine elements of both models. These systems often include:

• government-issued digital identity wallets
• verifiable credentials for documents
• selective disclosure mechanisms
• cryptographic verification

The European Union’s Digital Identity Wallet under eIDAS 2.0 is the most prominent example. In this model, citizens hold credentials in a wallet and can share them with services when required. But the trust framework remains anchored in the state. This is not pure self-sovereign identity. It is better described as state-anchored decentralized identity — a compromise between privacy ideals and regulatory realities.

Real Working Examples

While full SSI ecosystems remain rare, several real-world deployments show elements of the model working. British Columbia in Canada has issued verifiable digital credentials for businesses and government services. The Netherlands has developed privacy-preserving attribute sharing through the IRMA system. The European Union is preparing large-scale deployment of national digital identity wallets.

These initiatives demonstrate that the technology works. What remains unresolved is how much sovereignty individuals will actually retain.

A Personal Reflection

I invested in the Sovrin project in 2017, before its public launch. At the time, the vision was compelling. A decentralized identity layer for the internet. Privacy by design and user control as a foundational principle. I had hope for SSI (and commercial gain of course).

Today the conversation is different.

Digital identity is accelerating — but largely under the control of governments and major technology platforms. The irony is that many of the technologies originally designed to protect individuals from centralized identity power are now being deployed inside systems that may strengthen it.

We are starting to see this shift in very practical ways.

Governments around the world are introducing national age verification requirements to regulate access to online services. Social platforms are being pushed to implement age verification and identity checks before allowing access to certain content or communities. In California, proposals around operating system level age verification suggest that identity controls could move even deeper into the technology stack — into the devices themselves. And, at the same time, a new challenge is emerging: AI agents acting on behalf of people and organizations.

As autonomous agents interact with services, signing transactions, or making decisions, the identity layer becomes even more important. Someone — or something — must be accountable. The chain of responsibility must be clear.

• Who issued the agent’s credentials?
• Who authorized its actions?
• Who is responsible when it makes a mistake?

These questions push us further toward stronger identity infrastructure. And once identity becomes the control mechanism for access, compliance, safety, and automation, it becomes extremely powerful.

This is why the principles behind Self-Sovereign Identity still matter.

SSI was never just about convenience. It was about architectural safeguards — designing identity systems that minimize surveillance, reduce centralized control, and give individuals meaningful ownership over their credentials.

Do I think we will get a pure form of SSI?

Probably not. The legal, economic, and political incentives all point toward state-anchored or platform-controlled identity ecosystems.

But I still hope that the principles behind SSI survive inside those systems — selective disclosure, minimal data sharing, portable credentials, and user agency. Because once identity becomes fully centralized — embedded into governments, platforms, devices, and AI systems — reversing that architecture will be extraordinarily difficult.

This is why the conversation around SSI still matters.

Do I think we will get it?

No.

Do I hope?

Always.

The Old Model of Vendor Risk

For most of the last twenty years vendor risk was relatively straightforward. If your company used a vendor, you signed a contract. Procurement reviewed it, security assessed it and legal negotiated it. You performed a vendor risk review and maybe you looked at SOC2 or ISO certifications. Your dependency list was essentially the list of vendors you paid. Not 100% perfect, but manageable. That model worked reasonably well in the enterprise software era, but has now started to break down in the AI era.

AI Is Not a Product

Many organizations still think about AI vendors the same way they think about software vendors. But AI systems are rarely standalone products. They are compositional systems, and this is even more so once you introduce agentic work.

A single workflow might look something like this:

Application
→ SaaS platform
→ analytics engine
→ AI inference service
→ model provider
→ cloud infrastructure

Your organization may never sign a contract with the model provider. But your workflow depends on it. Your vendor’s vendor’s vendor may ultimately be the one actually running the model. Very few organizations have mapped (or even tried to understand) that chain.

The Illusion of Approval

Enterprises often believe they have approved their AI vendors because we trial, POC, vet and review them. What they have actually approved is an technical interface – a very smart and intelligent one at that – that itself could consist of:

→ multiple models
→ different providers
→ dynamic routing
→ fallback systems
→ region-specific infrastructure
→ who know what storage and data collection
Even the vendor itself may change models over time for various reasons (sentiment, commercials, rivalry, etc.) That means the underlying dependency can shift without the customer (you) even realizing it. Security teams believe they know what is running. In many cases they don’t (they can’t).

Why AI Dependencies Are Hard to See

When SaaS adoption exploded, we saw the rise of shadow IT. But shadow IT left a trace. You could track a new SaaS product by things like, a new login, a new identity integration, new network traffic or a new invoice.

Eventually security teams caught up using a combination of CASBs, network monitoring, SSO enforcement, spending analysis, etc.

But, once again, AI is different. AI usage can, and often is, be embedded inside other platforms. It can be invoked dynamically through APIs. It can sit inside analytics features, copilots, or agent workflows. Traditional logs may not even tell you which model was used. The dependency is now invisible.

A Real-World Stress Test

The Anthropic phase-out scenario inside the U.S. government is essentially a stress test that we can watch in near real-time. Imagine discovering that one of your AI dependencies must disappear within months. Not just replaced, but removed. That sounds manageable until you remember something important.

Models are not interchangeable.

Switching model providers changes things like:

→ output structure
→ latency characteristics
→ safety filtering
→ fallback systems
→ reasoning behavior
→ hallucination profiles

Applications built around one model may behave very differently with another. There is a very good likelihood that agent workflows may break and security assumptions may change.

Rotating an API key is easy, untangling hidden dependencies is not and re-testing everything that was built is not.

The European Perspective

Across the Atlantic, Europe is confronting a similar dependency problem from a different angle. European policymakers have become increasingly concerned about structural dependence on foreign digital infrastructure. Cloud platforms are a major example.

Today much of Europe’s digital economy runs on infrastructure operated by a small number of U.S. hyperscale providers — primarily Amazon Web Services, Microsoft Azure, and Google Cloud. European Parliament research has highlighted several concerns around this concentration.

First, market concentration/dominance. A handful of hyper-scalers control a large share of global cloud infrastructure capacity and investment.

Second, strategic dependency. Cloud platforms now underpin critical sectors such as banking, government services, healthcare, research, industrial supply chains and now AI development environments. When those systems depend on foreign-owned infrastructure, policymakers increasingly see it as a strategic risk.

Third, jurisdictional exposure. It is not secret that European and US lawmakers sometimes have veery difference views on policy. Even when European data is stored inside the EU, providers headquartered elsewhere may still be subject to foreign legal frameworks. This creates tension with European data protection frameworks such as GDPR.

Finally, there is the economic dimension. A large portion of European cloud spending ultimately flows to companies outside Europe. As cloud adoption increases — particularly with AI workloads — that financial flow continues to grow.

For policymakers, this raises concerns about long-term technological competitiveness and economic sovereignty.

Payments Tell the Same Story

The same structural dependency appears in payments infrastructure. A large share of European card payments still flows through Visa and Mastercard, both U.S. networks. This has triggered discussions about building alternative European payment systems. Again, the goal is not necessarily to remove these providers, but to ensure that critical infrastructure does not depend entirely on systems controlled elsewhere. Ok, maybe it also has to do with the 61% of payment processing handled by US companies when considering the 4.7 trillion USD spent in 2023.

AI Will Deepen These Dependencies

AI adds complete new layer to this picture. Training and running modern models requires enormous computing infrastructure, specialized chips, Large-scale data centers and distributed inference platforms. The companies that operate these environments today are largely the same hyperscale providers already dominating cloud infrastructure. That means the AI boom may reinforce existing infrastructure dependencies rather than diversify them.

Which brings us back to your enterprise.

Traditional vendor lock-in was mostly technical. You depended on a database, a storage system, an operating system. Migrating away was painful, but predictable.

AI lock-in is different. It is behavioral lock-in. Not impossible, just must must harder.

Applications built around a particular model assume certain patterns of how the model responds, how it structures output and how it interprets prompts. Changing the model can change the behavior of the entire system. That is a much deeper form of dependency.

So What Can Organizations Actually Do

None of this means companies should stop adopting AI. But it does mean organizations should start thinking about AI dependencies more deliberately.

• Push for Transparency: The first step is pushing vendors for transparency, or just pushing for some sanity. Sidenote – I am hopeful with statements like these (but that is for another time). Organizations should ask demand vendors disclose which models they rely on, where those models run, and whether fallback providers are used if a model becomes unavailable. Many vendors struggle to answer these questions clearly today, which already tells you something about the visibility you actually have into the systems you depend on.
• Own the Control Points: The second step is identifying the control points you actually own. While companies may not control the AI provider itself, they can control the surrounding systems. This includes the data entering models, the orchestration layers that call them, agent frameworks or automation pipelines, and the systems that consume the outputs. If the only place where governance exists is at the vendor contract level, then operational visibility is likely very limited.
• Do a Kill Test: The third step is running dependency kill tests. This simply means simulating the removal of an important AI dependency. Disable the API key or block the endpoint in a controlled environment and observe what happens. Some systems will break immediately, while others may silently degrade. In many cases organizations discover dependencies they did not even realize existed.
• Understand Execution: Organizations should map how AI is actually used inside their environment rather than simply listing vendors. (Can anyone see a new governance role here – CAIO ). The important questions are which systems are calling models, which endpoints are involved, and what data is being sent.
• Data Control: Finally, companies need to understand what happens to their data once it enters an AI system. Where is it stored? How long is it retained? Is it used for training or evaluation? Can it be removed if needed? Data often passes through several layers such as inference services, safety filters, and monitoring systems before a response is returned, which means copies of that data may exist in more places than expected. If an organization does not understand how its data can be removed from a system, then it does not fully understand the dependency it has on that platform.

The 10-Second Takeaway

The AI revolution is not just changing how software works. It is changing how technology dependencies are structured. Enterprises once depended on software vendors and now they depend on entire technology supply chains. Vendors, their vendors, model providers, cloud inference platforms, training ecosystems and more.

Most organizations still believe they understand their dependencies. But many only understand the surface layer. And the next forced migration may not come with a six-month warning.

Code was never the real bottleneck.

The difficult parts of building software have always lived somewhere else. And AI is starting to making that clearer than ever.

The Illusion: Software = Code

When people think about software development, they imagine developers typing furiously, solving complex problems through clever code. But modern development has not worked like that for a long time. Frameworks like Rails, Laravel, Spring, and .NET already automated huge portions of the development process. Cloud platforms removed infrastructure complexity. Low-code platforms made it possible to build applications without writing code at all.

Even before AI, writing code had become one of the easiest part of building software. Yet many still measured progress in lines of code produced. AI has accelerated that illusion.

Code Is Cheap Now

With modern AI tools:

• Entire APIs can be generated in minutes
• UI prototypes appear in seconds
• Boilerplate disappears almost completely
• Refactoring can happen automatically

The marginal cost of producing code is approaching zero. This continues to impress the heck out me as a technologist and to most of us at first glance appears as pure upside. But removing friction changes how people behave.

Friction Forces Good Decisions

When implementing a a product feature takes weeks, teams have to ask hard questions:

• Do we actually need this feature?
• Is the design correct?
• What will the maintenance cost be?
• Will this introduce security or operational risk?

Because implementation is expensive, bad ideas (have to) die early. Constraints force discipline. They force clarity and prioritization.

When the cost of building something drops to almost nothing, that discipline disappears. Why debate a feature when AI can generate 4 different versions in ten minutes?

And that is where the real problem begins.

The Explosion of Software Complexity

Lower friction does not reduce complexity. In fact it multiplies it. You have felt it, you are five prompt deep into a project and you brain switches off and you end up in a death prompting cycle of “make secure, built with no error, check you work, blah blah”.

Economists call this Jevons Paradox: when something becomes more efficient to produce, we do not use less of it — we find more ways to use it.

The same will happen with software. More teams will build internal tools, more departments will automate workflows and more experimental products will be launched. And many of them will never be properly designed, governed, or maintained.

Side note here: Think about the explosion of low-code platforms like Power Platform just a few years ago where thousands of little apps were developed with little thought about the entire lifecycle of software development (and this was just on a single banking customer we worked with).

We will not get less software. We will get more software, faster — and often worse.

The Hard Parts Were Never Code

The difficult parts of building software have always been:

• Understanding the real problem
• Defining the right system boundaries
• Designing data models
• Thinking through failure scenarios

Then during implementation:

• Architecture design
• Distributed systems thinking
• Identity, security, and permission models

And finally designing systems that survive change.

These require deep thinking, not code generation and structured process. AI can help write functions — and increasingly even generate impressive applications – but it cannot replace judgment.

The Hidden Risk: The Loss of Learning

There is another subtle danger. When developers write code themselves, they simulate the system in their heads:

• What happens if this fails?
• What assumptions does this function make?
• How does this data move through the system?

That mental simulation is the real training. If developers rely entirely on generated code, they risk skipping the thinking step. The result is a small number of experienced engineers end up responsible for understanding and fixing the systems everyone else generates. The same small team of people who always carried the complexity will now carry even more of it.

The Real Bottlenecks of Software

(And we haven’t even reached the business layer yet)

Even if AI makes code generation nearly free, the real constraints remain:

• Product vision
• Architecture
• Security
• Governance
• User trust
• Distribution
• Adoption
• Change Management

The Apple App Store and Google Play Store together contain more than four million applications. Yet industry analyses consistently show that the majority of those apps receive fewer than a thousand downloads. A tiny fraction of products capture most user attention while millions quietly sit unused. AI will not solve discovery, it will simply make it easier to create more unused software.

The Future of Engineers

This brings us to question – what happens to the software development job? Why all the layoffs? Ironically, if this all plays out it most likely means more engineer will be needed in the long term, not less.

But crucially roles will shift.

The most valuable engineers will not be those who produce the most code. They will be the ones who can:

• Understand complex systems
• Design resilient architectures
• Make strong product decisions
• Govern technology responsibly

In a world where code is cheap, thinking becomes the scarce resource.

A Paradox Emerging in the Industry

Interestingly, some early patterns in the industry reinforce this idea. Across the technology sector we see a surprising paradox:

For example, IBM recently announced plans to triple its entry-level hiring, including software engineers. Their leadership warned that companies that stop hiring junior engineers today risk creating future shortages of experienced technical leaders.

At the same time, Anthropic — the company behind Claude Code, one of the most advanced AI coding systems — is aggressively hiring engineers, with reports suggesting more than 100 open roles. Despite building tools designed to automate coding, they still need engineers to design systems, evaluate AI output, and maintain architecture.

Another interesting signal is compensation. AI companies are offering some of the highest engineering salaries in history, with reports of median compensation around $570k and some packages approaching $700k.

We are also seeing companies announce layoffs while still hiring technical talent. For example, Block reduced its workforce but continues hiring AI engineers to expand its capabilities. (Interestingly this might also be because of other pressure, but the AI efficiency narrative is a much better public announcement.)

The pattern is becoming clear: AI may automate parts of coding, but the need for engineers who can design systems, govern complexity, and make sound technical decisions is only increasing.

Final Thoughts

In the end, there remains a noticeable disconnect between the public narrative and what is actually happening inside the industry. Headlines often suggest a future where AI replaces developers and software builds itself. Yet at the same time, companies are hiring engineers, investing in junior talent, and paying record salaries for technical expertise.

Reality rarely behaves like a perfectly predictable system — but neither are we living inside the chaos of the three-body problem where outcomes are impossible to understand. The signals are visible if we look carefully.

Time will ultimately tell how far AI will reshape software development. But for now, one thing is clear: the story being told in public and the decisions being made inside organizations do not entirely match. And that gap may be one of the most interesting signals of all.

If you have not looked at it yet, it is worth your time. It is one of the clearest snapshots we have of the current state of autonomous AI and where it is going.

Here are some useful links:

• AI Agent Index (Website)
• The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems (Paper)

The summary is simple.

• Autonomy is accelerating.
• Governance is not keeping pace.
• And that gap is going to matter.

The Acceleration of Autonomy

In the last two years alone, 24 of the 30 tracked agents were either released or received major updates. That is not incremental innovation, that is acceleration (a gold rush).

• Browser agents are now operating at L4–L5 autonomy. That means they do not just assist. They execute. They plan. They take multi-step actions with limited intervention once started.

• Enterprise agents follow a similar path. They may be designed conservatively, but once deployed into workflows, they begin to operate at far higher levels of independence.

We are no longer dealing with tools that help you think (like simple chat bots), we are deploying systems that act on your behalf. This changes everything.

With the acceleration and the increase and autonomy – they key question become how we maintain security, governance and visibility (not to mention control).

The Safety (Governance) Disclosure Gap

Of the 13 agents classified at frontier autonomy levels, only four disclose any meaningful agentic safety evaluations:

• ChatGPT Agent
• OpenAI Codex
• Claude Code
• Gemini 2.5 Computer Use

The broader numbers are more revealing.

• 25 of 30 agents disclose no internal safety results.
• 23 of 30 provide no third-party testing information.

Developers are very transparent about what their agents can do, and they are far less transparent about how those agents were tested.

For enterprises operating under ISO 27001, ISO 42001, SOC 2, NIST, or sector regulation, this is not academic. It is a procurement risk. It is a board-level question waiting to be asked.

Just have a look at the “Safety” section of the disclosure map.

The Fragmented Responsibility Chain and Concentrated Model Providers

Most agents are not vertically integrated systems, they sit on top of foundation models provided primarily be three main players, OpenAI, Anthropic and Google.

The agent builder adds orchestration and scaffolding. The enterprise adds configuration and deployment context. When something goes wrong, responsibility is distributed across layers. Who is to blame, the model, the agent framework, the tools integrated or some configuration operator?
This reminds me very much of the early days cloud adoption when abstraction created acceleration but could not solve bad architecture, design or configuration. In the case of cloud, shared responsibility models took years to mature in clarity. Contracts evolved. Audit rights became standard. Logging expectations stabilised. Data processing and storage became clear.

We are now at the early stage of that same cycle — but with systems that can act autonomously. No single entity clearly owns end-to-end accountability, and the is different from cloud today.

That is a yet again a governance problem, not a technical one.

The Web Identity Problem

Browser-based agents introduce another layer of complexity. They simply act “on behalf of a user.” In practice, that often means bypassing traditional bot controls and ignoring conventional web signaling mechanisms like robots.txt. That also creates a huge question of identity and accountability for actions (but that is a whole other discussion). It is therefor shocking that only one agent in the index uses cryptographic request signing – ChatGPT Agent. That is crazy to consider.

We do not yet have a mature identity model for autonomous agents operating across the web. If an agent logs into a system, accesses content, or triggers transactions, what is its identity? How is that identity verified? How is it attested? How is it audited?

In identity governance terms, we are allowing non-human actors to execute high-impact workflows without fully established control frameworks. We would never allow that for a human privileged account.

Yet we are beginning to allow it for AI.

The Governance Opportunity

This is not only a risk story. It is an opportunity story. As autonomy increases, the differentiator will not be raw power. It will be governability.

Enterprises, our businesses and our customers will need to start asking harder questions:

• What autonomy level does this agent operate at?
• What red-teaming and safety testing was performed?
• Is there third-party validation?
• Are outbound actions signed and traceable?
• Is there an auditable trail of decisions?
• Is there a clear kill-switch?

Vendors that can answer those questions confidently will win in regulated markets. Traditional IGA and PAM processes and platforms will need to evolve to keep up with demand for accountability (while allowing for speed of execution).

We are entering a new domain, not mere human and non-human, but a merging of:

• Agent identity.
• Agent permissions.
• Agent attestation.
• Agent auditability.

Those who work in governance, security, identity, and compliance should not see this as a threat. It is the next frontier of structured control and visibility.

“AI” – particularly LLMs and increasingly all kinds of “agentic” systems are already delivering measurable gains in productivity, efficiency, and scale. Entire workflows are being compressed. Teams are being reimagined. Decision-making is being augmented—and in some cases, quietly replaced. And yet, beneath this acceleration, there is something else:

A growing, systemic unease.

From Tools to Actors

For decades, we integrated software as “tools” – deterministic systems operating within clearly defined rules. That paradigm is under pressure and in many ways potentially over.

With agentic AI, we no longer deploy tools. We are introducing “actors” into our systems.

These actors have agency and they have access. They can consume and use other tools (email, file systems, APIs, shells), communicate with humans and other agents, persist vast amount of memory across interactions and now make decisions that have real-world consequences.

And critically—they can act outside of our immediate visibility.
We see this as an game changing benefit and the possibilities are considerable, but the pace is exposing the fundamentals of the technology and our ability to effectively use it. At least for now.

What Research Currently Actually Shows

A recent preprint, “Agents of Chaos” (published 23 Feb – Cornell University), provides one of the clearest early looks at what happens when you combine LLMs with autonomy, tools, and real environments. For a two-week period, this wasn’t theoretical. A team of researchers deployed autonomous agents in a live environment with access to email accounts, Discord, file systems, shell execution and persistent memory.

Over the two-week red-teaming exercise, they observed 11 case studies concerning failure modes. Real behaviors emerging from real work setups.

1. Unauthorized Agent Compliance

Agents followed instructions from individuals who were not their owners. This breaks one of the most fundamental assumptions in security: that authority and identity are tightly coupled. In practice, it means:

• An attacker doesn’t need system access
• They just need to convincingly ask

2. Sensitive Data Leakage

Agents disclosed sensitive information when prompted in the right way. Not because they were explicitly told but because context was misinterpreted, boundaries were poorly defined or objectives overrode constraints. This creates a new kind of data exfiltration vector: conversational leakage through delegated agents/actors/intelligence.

3. Destructive System Actions

In some cases, agents executed system-level commands that caused damage. Not maliciously, but simply as a side effect of misaligned goals, over-broad permissions or an incomplete understanding of the consequence.

This is the equivalent of giving a junior engineer root access—and no supervision.

4. Denial of Service & Resource Exhaustion

Agents unintentionally entered loops or executed tasks that led to runaway compute usage, system slowdowns and self-inflicted denial-of-service conditions. Traditional systems fail predictably while agentic systems can fail dynamically.

5. Identity Spoofing

Agents were susceptible to impersonation of identities and could then be convinced that a malicious actor was a legitimate user or another agent was authoritative. This undermines the very concept of trust in distributed systems.

6. Cross-Agent Contamination

Perhaps one of the most concerning findings from the research was the agents could “propagate” unsafe behaviors to other agents. Bad practices didn’t stay isolated in the two-week study, they spread. This introduced a new category of risk that researchers called “Emergent systemic failure through agent interaction”.

7. False Reporting of Task Completion

In multiple cases, agents reported that tasks were successfully completed, while the underlying system contradicted those claims. This is not just a hallucination problem. It is a verification, observability and governance failure. If you cannot trust system reporting,
you cannot trust system outcomes.

This Is Not Hypothetical Risk

The paper is explicit:

These behaviors establish “security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings.” And they emerge only when you combine: LLMs + autonomy + tools + communication.

Ironically, this is exactly what the industry is racing toward and big tech promising.

We are facing a real and material yet currently unbounded risk that cannot be quantified—not because it is negligible, but because it is emergent, breaking traditional models based on known failure modes, historical data, and predictable behavior, and instead arising from non-deterministic decisions, context-dependent actions, and complex multi-agent interactions.

For now it seems like “Agents of Chaos” is not just a provocative title – it is an accurate description of where we are.

Sure you can use GPO or ConfigMgr to ensure the service is installed on all DC, but sometimes one (or many) is missed or fails. This results in password changes being “lost”.

I have found the simplest way to fix this frustration is to use a simple PowerShell script to enumerate the forest, domains and domain controllers to test if the service is installed and active.

The following script is a simple example of how this can be achieved. At Integralis we have turned this into a monitor that reports into our monitoring platforms to automate the process (for large environments that does not have our agents on all domain controllers).

The approach is generic and will work for any service.

When it comes to inviting users from different tenants to join an Azure AD, there are various options. Each of these options have specific benefits and some limitations. Options include:

• AzureAD PowerShell Module
  • New-AzureADMSinvitation
• Microsoft Graph API Management Agent for MIM
• Microsoft Azure Portal

If a customer decides to use MIM 2016 as a synchronization method, there are two options

• using the Microsoft Graph API Management Agent
• or a custom PowerShell Management Agent.

The choice between to two will be largely driven between the usage scenarios. If a customer wants to simply invite users into the tenant to share resources, the Microsoft Graph API Management Agent is a great options since it offer the ability to do delta operations. The Graph API MA however has specific limitations regarding the attribute set it can manipulated (e.g. proxyAddresses cannot be written since this is a Exchange Online attribute, even though it can be imported via the Graph API).

I recently worked on a project where one of the core business scenarios was to enable a cross-tenant Global Address List (GAL) based off the B2B guest users in each of the Azure AD tenant. In order to manage all the different proxyAddresses (and other attributes) the had to integrated into Azure AD and Microsoft Exchange Online. As such the choice was made to sacrifice delta functionality (since the PowerShell module does not support delta data) and to use a custom PowerShell management agent to integrate both the Azure AD and Microsoft Exchange Online endpoints.

Granfeldt PowerShell MA for Azure B2B

The complete PowerShell MA is available on Github as is based off the Granfeldt PowerShell MA framework (available here). The management agent supports various configuration options which are controlled by on included settings files. Options include:

Source Available on GitHub

There are still many things that I would like to update and tweak on the implementation, but this will come in due course hopefully. If in the meantime this is valuable to someone or anyone wishes to contribute something the project on GitHub is the place to visit.