blogs

The notes behind "If AI helps teams build faster, who helps them build better?"

2026-06-24T09:01:24Z

When I wrote my LeadDev reflection, I had far more notes than I could sensibly include.

Rather than squeeze everything into the public post, I thought it might be useful to share the notes behind the argument for paid subscribers.

These are the talks that helped me connect quality engineering, AI, leadership, principles, culture, incidents and how teams perform under pressure.

They are not transcripts but my notes, interpretations and follow-up reflections.

This is the work behind my main post, “If AI helps teams build faster, who helps them build better?”

I’ve made the first section, based on Scott Carey’s talk on engineering leadership in 2026, free to read so you can get a feel for the format, tone and level of detail before deciding whether to continue.

How to use these notes

This is a long post, so I do not expect most people to read it from start to finish.

Think of it more like an open notebook from LeadDev. You can read it straight through, skim the sections that interest you, or jump to the part that connects most closely to a problem you are facing in your team.

The thread running through these notes is this:

If AI helps teams produce more output, what helps them produce better outcomes?

Across the five talks, I kept coming back to the same idea.

Better engineering outcomes rarely come from pushing individuals harder. They come from improving the conditions around the work.

That includes:

how leaders support teams
how principles turn into behaviour
how culture responds under pressure
how organisations learn from incidents
how teams perform when leaders are not in the room

Here are a few ways in:

If you are thinking about AI, leadership pressure and the future of management

Start with:

Scott Carey: Engineering leadership in 2026

Scott’s talk helped me think about how engineering leadership roles are changing as AI adoption, economic pressure and productivity expectations increase.

The part that stood out to me was the growing idea of the player-manager: engineering managers who are expected to lead people while also spending a large amount of their time contributing to the codebase.

This section is useful if you are thinking about:

what happens when managers are expected to be more hands-on
where the people work goes when management layers are squeezed
how AI might increase output while also increasing leadership pressure
what role quality engineering might play if teams need more coaching, mentoring and support around quality

The main QE question is:

If managers have less space for the human side of leadership, who helps teams build the habits, feedback loops and thinking needed for quality?

If you are thinking about principles, behaviour and pressure

Start with:

Manogna Machiraju: The shadow culture: Why engineering principles fail under load

Manogna’s talk helped me think about why engineering principles often fail to influence behaviour.

Most organisations have principles like “Quality first”, “Sustainable pace” or “Radical ownership”.

The problem is not that people disagree with them. The problem is that when pressure increases, the system often rewards something else.

This section is useful if your team or organisation has principles that people agree with, but struggle to live by in practice.

It is especially useful if you are thinking about:

why “quality first” is easy to say but hard to follow
how pressure reveals the real culture
how to translate principles into practical behaviours
how to identify what makes better behaviour difficult
how to avoid blaming people when the system makes the shortcut easier

The main QE question is:

What makes quality-focused behaviour easy, realistic and safe to follow when the pressure increases?

If you are thinking about productivity, transformation and organisational culture

Start with:

Randy Shoup: We doubled engineering productivity at eBay, but couldn’t change culture

Randy’s talk helped me separate two things that are often treated as the same:

improving engineering productivity
changing organisational culture

The eBay velocity initiative improved deployment frequency, lead time, change failure rate and time to recover. It helped teams move faster and improved local engineering performance.

But the wider organisation did not really change.

This section is useful if you are thinking about:

why better delivery metrics do not automatically mean better business outcomes
why local engineering improvement can still be blocked by wider culture
how planning, strategy and incentives shape quality
why delivery improvement needs support from above, below and across
how organisations can use modern engineering practices while still operating through old cultural patterns

The main QE question is:

What kind of organisation are teams delivering inside, and does that organisation help or hinder quality?

If you are thinking about incidents, learning and psychological safety

Start with:

Lyndsay Prewer - Beyond timelines: Learning from incidents

Lyndsay’s talk introduced the Social Psychology of Risk, or SPoR, as a way to learn more deeply from incidents.

The part that stood out to me was the focus on more than the visible timeline.

SPoR looks at:

the workspace: what happened in the environment
the headspace: what people were thinking, feeling, noticing or worrying about
the group space: what the wider group understood, assumed or shared

This section is useful if you are thinking about:

how to make post-incident reviews more useful
why timelines alone rarely tell the full story
how to create safer conversations after incidents
how to understand the conditions that made failure more likely
how to learn from the human, social and cultural parts of incidents

The main QE question is:

What are we missing if we only look at what happened, and not how people experienced the work at the time?

If you are thinking about team performance, coaching and how teams behave under pressure

Start with:

Cathy Pank: Reading the game: Mental skills for high-performing engineering teams

Cathy’s talk used ideas from sport to think about engineering team performance.

The idea that stood out to me was the role of the football coach during a match.

Once the game starts, the coach can only influence so much. Most of the work is in the hands of the players. The real impact happens before the match, through training, preparation, shared understanding, habits and trust.

This section is useful if you are thinking about:

how leaders observe team dynamics, not just delivery
why teams need to prepare before high-pressure moments
how confidence, clarity and trust affect performance
how teams behave when leaders are not in the room
why good teams need to adapt to each other, not just perform individually

The main QE question is:

How do we help teams make better decisions when nobody is standing over their shoulder?

If you are thinking about Quality Engineering as a broader leadership practice

Read the whole thing, but pay particular attention to the reflection sections after each talk.

The thread running through all of this is that quality is not only created in code, tests, tools or delivery practices.

It is also influenced by:

the questions teams ask
the behaviours they practise
the risks they surface
the pressure they are under
the culture they work inside
the feedback loops they use
the decisions they make
the way they learn from failure
the way people interact when the work gets difficult

That is why I came away from LDX thinking about Quality Engineering as something broader than testing or delivery improvement.

For me, these talks pointed towards Quality Engineering as the work of helping teams create better conditions for better decisions.

Especially as AI increases the amount of software teams can produce.

This post is best viewed online or through the Substack app due to its length.

Scott Carey: Engineering leadership in 2026

Watch the talk here: https://leaddev.com/leadership/engineering-leadership-in-2026

What QE problem this helps with

This section is useful if you are thinking about how AI, economic pressure and changing expectations of managers might affect software quality.

The main QE question is:

If teams are expected to move faster, managers are squeezed, and AI increases output, who helps teams think about quality?

Why this talk influenced the post

Scott Carey’s talk helped me think about the future of engineering leadership.

One theme I noticed was the growing idea of the player-manager.

This is the idea that engineering managers do not only lead people, but also actively contribute to the codebase.

That sounds useful in some contexts. If a team is senior, stable and able to manage itself, maybe a hands-on manager can work well.

But it also creates a real tension.

Leadership is not just about delivery, planning and technical decision-making. It is also about supporting people, handling difficult conversations, creating clarity, managing tension and helping teams operate well.

If managers are expected to stay mostly hands-on, that people work has to go somewhere.

This is where the quality engineering connection became interesting for me.

As AI helps teams produce more output, QE may become even more important in helping teams slow down in the right places, ask better questions, reduce uncertainty earlier and build systems that make good quality more likely.

My notes

One theme I noticed at LeadDev was the growing idea of the player-manager.

This is the idea that engineering managers do not only lead people, but also actively contribute to the codebase.

One example shared was an organisation looking for a manager who would spend 75% of their time doing hands-on development and 25% of their time leading the team.

I can see how this might work if the team is senior, stable and able to manage itself. But the moment people issues appear, especially personal issues that affect someone’s work, that balance changes quickly.

Those situations can absorb a lot of a manager’s time. If the manager is expected to stay mostly hands-on, that support has to be offloaded somewhere.

It feels like the industry is slowly squeezing the middle manager role and shifting it towards a more hands-on leadership model. AI will probably accelerate that trend.

What the research suggested

LeadDev asked 600+ engineering leaders how their roles are changing in response to the current economic and technology landscape.

Some of the points shared included:

Money that previously funded headcount is now being redirected towards GPUs and AI infrastructure.
VP and EM roles are being squeezed closer to lead roles.

There is increasing interest in IC support agents and AI-enabled ways of scaling individual contributors.
Middle management appears to be one of the areas under the most pressure.

Some CEOs are publicly linking headcount reductions to AI.
Managers who remain in role often have more scope, more responsibility and more people to support.

The player-manager

The player-manager model expects engineering managers to both:

manage and support people
actively contribute to technical work

This creates a real tension.

Managers are being asked to do more, while also staying closer to the codebase. One point shared was that one in three engineering managers are looking to move back into IC roles.

There are also job descriptions asking people to “lead through code”, with examples where managers are expected to spend around 75% of their time hands-on.

That might sound efficient on paper. But it raises a bigger question:

What happens to the people work?

Because leadership is not just about delivery, planning and technical decision-making. It is also about supporting people, handling difficult conversations, creating clarity, managing tension and helping teams operate well.

If managers are expected to stay mostly hands-on, that support has to come from somewhere else.

What might this mean for quality engineering?

This raises an interesting question for quality engineering.

If managers are squeezed, teams are expected to move faster, and AI increases the amount of code being produced, then who helps teams think about quality?

One possibility is that a QE supports multiple teams more like a:

consultant
coach
mentor
teacher

Someone who helps teams think about:

how they build quality into their products
how they use AI agents safely
how they reduce uncertainty earlier in the work
how they create useful feedback loops
how they spot risks before they become expensive problems

In that model, the QE role becomes less about correcting quality issues after they appear, and more about helping teams build the mindset, practices and tools that make better quality more likely.

Another possibility is that we build more of the quality engineering mindset into the tools themselves, so teams are nudged towards better questions, better checks and better ways of working as they use them.

My takeaway

The player-manager model may help organisations reduce management layers and keep leaders closer to the work.

But there is a risk.

If we remove or squeeze the people support role too far, the work does not disappear. It gets pushed somewhere else, ignored, or picked up informally by people who may not have the time, skills or support to do it well.

For quality engineering, this could be an important shift.

As AI helps teams produce more output, QE may become even more important in helping teams slow down in the right places, ask better questions, and build systems that make good quality more likely.

Further reading: The Engineering Leadership Report 2026 Pre-register - LeadDev

Found this useful?

If this kind of reflection is useful, the rest of the post continues for paid subscribers.

In the paid section, I go deeper into:

Manogna Machiraju’s talk on why engineering principles fail under load
Randy Shoup’s story of improving engineering productivity at eBay without changing the wider culture
Lyndsay Prewer’s introduction to the Social Psychology of Risk and how it can improve incident learning
Cathy Pank’s lessons from football coaching and what they tell us about team performance under pressure

The thread running through all of it is the same:

If AI helps teams build faster, who helps them build better?

For me, these talks helped connect quality engineering, leadership pressure, principles, culture, incidents and the way teams behave when the work gets difficult.

And these are the notes, questions and connections that helped me get there.

Paid subscriptions help me keep making time for these longer, more reflective write-ups, alongside the free posts, Linky editions and shorter reflections I share publicly.

The Ice-Cream Seller

2026-06-24T08:53:37Z

A nostalgic story about childhood, family bonds, Sunday traditions, and the ice-cream seller who made every day special.

Continue reading on ILLUMINATION »

The Ice-Cream Seller

2026-06-24T08:53:37Z

A nostalgic story about childhood, family bonds, Sunday traditions, and the ice-cream seller who made every day special.

Continue reading on ILLUMINATION »

Too close to see the canvas – Part 5

2026-06-24T08:10:00Z

The Pattern beyond the pixels – Fit for purpose

So far, in what is quickly becoming a full blown small book, we have looked at the project and the problem, the danger of focusing too much on reactive work, changing the seemingly unchangeable and metrics and the dangers of measuring the wrong thing. This post is about a closely related problem that took me a long time to fully understand: what happens when you try to use one test suite to serve everyone — and why it almost inevitably serves nobody particularly well.

Everyone wanted something different

Once I started having proper conversations with the different people involved — developers, testers, product owners, managers — something became clear quite quickly. They all had completely legitimate needs from the automation. And those needs were not just different. In some cases they were actively in conflict with each other.

Developers wanted ultrafast feedback. The ability to make changes with confidence. For a developer applying a test-first approach, a red test is expected — even wanted. It means the feature isn’t done yet. A failing test is not a crisis; it’s the system working. For them, a report full of failures from ongoing work is mostly noise.

Testers wanted something completely different: feedback on the stability of a release candidate, and confidence that nothing previously working had broken. They needed to be able to test different release candidates in parallel, with different sets of data. For them, a failing test means a problem found that needs immediate attention. A bad report is not noise — it’s a blocker.

Product owners and scrum masters had a single question: is this safe to ship? A failing test means either we don’t release, or we have to make a difficult judgement call about severity. They needed a clear, trustworthy signal — not a noisy report they had to interpret.

Managers wanted strategic information. Are we spending money in the right places? What are the trends over time? Where are the root causes when things go wrong? They needed a different cadence and a different level of aggregation entirely.

All of these are completely reasonable things to want from a test setup. And we were trying to meet all of them with the same suite, running on the same schedule, producing the same report. Which meant it wasn’t really serving any of them particularly well — and sometimes the needs were pulling in directly opposite directions. Which of these people are you? Which ones are you trying to serve? (And yes, the answer can be more than one. Or all of them. Or maybe different ones.)

Tests (should) have layers

You don’t have to use one tool and one set of tests for everything. In fact, you shouldn’t.

Tests have layers — and different layers serve different purposes. Unit tests give developers fast, granular feedback at the component level. Integration tests check that pieces work together. End-to-end tests verify that whole user flows work as expected. On top of those you might have smoke tests for quick sanity checks, contract tests to catch breaking changes between services, performance tests, security tests, regulatory tests, chaos testing. The layers vary by organisation and product. (And anyone who tells you there is one right answer to how many layers you need is probably selling something.) The point is that each type answers different questions, runs at a different cadence, and costs different things to build and maintain.

What we had was primarily end-to-end, GUI-level functional tests run nightly. That tells you something specific and valuable — whether certain flows work from a user’s perspective at a given point in time. It does not tell you whether individual components are well-built, whether performance is acceptable, whether a change in one service is quietly breaking a contract with another, or whether a developer’s branch is in a shippable state right now.

Treating that one layer as the measure of overall quality is like trying to use a single instrument to play an entire orchestra. You get a sound. You don’t get the music.

The question to ask is: what do we actually need to know, and when? Then work backwards from that to figure out which layers to invest in. Layers of depth. Layers of cadence. Layers of focus.

Which bring me to the first glimpse of my last epiphany: This story was never about automation. It was about learning about human motivation, change management and having an impact. Because whatever it might look like – It’s always a people problem.

”The 2nd Law of Consulting: No matter how it looks at first, it’s always a people problem”
– Gerald Weinberg, The secrets of consulting, 1985

The sunk cost trap

One thing I found that I genuinely thought would be straightforward to fix turned out to be much harder than I anticipated. Not technically. Emotionally.

I’m talking about deleting tests. Retiring approaches that had stopped making sense. Letting go of things that had been built with real effort and real pride.

People hate getting rid of things. At the individual level: ”it might find something next time.” At the organisational level: ”we invested a lot into this.” The emotional barrier to removing something that people have built and are proud of — and that the organisation has spent real money on — is enormous. And completely understandable. But it causes real damage if it goes unchallenged.

Because every test run is a cost. And with automation – that cost is not as visible as when it is connected to a human pay slip. But in my opinion – A test should only be run if it has a value greater than its cost.

Cost can be visible, in the form of maintenance cost (the human resources needed), licenses, hardware, energy and other computing costs. Or it could be less obvious, in the form of complexity, execution time, slowing down time to market instead of speeding it up, context switching and mental energy needed, trust and confidence (in the software or your team),speed of feedback . That cost is ongoing, every single run, whether or not the test is still earning its place.

The question to ask is not ”has this test been valuable in the past.” That investment is gone regardless. The question is: does it make sense to keep running this from today forward?

The answer is sometimes yes. Sometimes no. Sometimes ”yes, but we should rebuild it differently.” The only way to know is to actually look at it honestly — which means tracking what things cost as well as what they return, and being willing to have the uncomfortable conversation when the maths stops mathing.

Of course there’s no simple threshold for when something starts costing too much — it depends entirely on your context. But measuring total run time and total maintenance time over time, and noticing when those start rising without a corresponding increase in value, is a simple place to start. The most valuable I’ve always found to be hard to measure – how confident and happy are your people? Do they feel joy working with it? Not a metric upper management will love, but one that builds trust and confidence in different ways.

In part six — the final post — we’ll look at the thing that underpins all of this and why none of the technical work above actually lands without understanding the human side of it. Change management and humans.

Faking plans is easier than fighting plans

2026-06-24T04:06:08Z

Ever entered a project as tester early on into a project? At that time when you still hold the illusion that plans are about to be set and you are on time? And then, only to find out that the only early enough is in the time the project manager gets trained on their ideas.

Well, it's a pretty deflating experience, and unfortunately all too common. It appears there is no early enough without a lot of friction to include testing.

The source of friction is in the mental model. Having Plan - Implement - Test - Release is such a simple illustration. But when you actually overlay testing, it's the other side of everything, and if you wanted real tasks and control over it rather than long-running categories in which you report hours into, you would actually need a more finegrained breakdown of the testing tasks.

Context for an inspiration

I worked on a fixed-price project, where the fixed-price was 50% of real price. These are typical in consulting world, and what they effectively mean is that the project environment is born passively hostile with a continuous fight on showing there is a change request lurking somewhere in the communication of two or more parties.

The fixed price estimation included no test planning, because why would anyone plan from testing perspective separate from project. And the whole plan of testing was described in usual abstract terms of unit, integration, end to end -testing and UAT.

I got to plan testing before the project plan, so I outlined the testing tasks:

Identifying testing not promised within 8 hours effort allocation, when promise was unit, integration, e2e, UAT and what of it was change to promised scope.
Deciding on quality practices: reviews, pipelines, timing of implement & test in mutually supportive ways
Reviews of requirements spec, solution spec
Testing while implementing, collaboratively, unit and integration testing
Testing after implementing while integrated with a 3rd party system, minimalized, e2e testing
Testing while integrated with a 3rd party system, necessary scope, but could be e2e or UAT and thus done from two different budgets
Staying updated on learnings in the project

While planning for testing and reviewing the documents, I found a bug worth 25 000 € / annually, and facilitated a confirming conversation with the client, and patted myself on the back for planning testing early on to avoid expensive problems down the road.

If it only was that simple. Weeks later, I take a look at the project plan and none of the things I planned for testing got integrated. While appearance of a plan is not the plan underneath the surface, I find it both sad and funny that this is how we still insist on doing project planning. Not with a twist: they would not plan for things with people, or read the things from people, they would have AI in between all of that.

There is a special sense of middle finger that comes with "I did not bother reading the idea of key milestones you were communicating with an image, I rather sent your image to AI and here's a wall of text it had to say about it that made me completely miss the point, with a twist of false claims I have no capability of detecting".

I took my wins of finding the bug, and decided appearance of plan and the plan that will be executed can coexist, because faking it is easier than fighting it.

Lessons from the inspiration

As always, doing the work teaches you a lot.

Changing other persons mental model was more possible at time before AI. AI allows for not making an effort in building own mental model.
Actual collaboration would win over co-creating and handing off artifacts. Two hours together on a plan wins two hours separately on the very same plan. That removes also the AI workslop in between that is worse than before.
Change management politics on fixed price projects remain an antipattern caused by competition tactics.
Feeling detached from appearance of plans is necessary for survival. You know, that sphere of influence, the cost of applying influence for change greatly outweighs the cost of faking it.
The things that matter like access to version control tool to collaborate on in implementation or pipeline so that tests are not in head of someone don't need to make it into plan to make it into action.
Use of AI splits us to two: developers and knowledge workers, and I find myself increasingly frustrated with the hurdles the latter crowd creates in the handoff.

Selenium to Playwright Migration Part 2: Setup and Configuration

2026-06-23T22:00:00Z

Convert pom.xml to package.json, TestNG XML to playwright.config.ts, Java properties to .env, and Maven CI to GitHub Actions with 4-shard parallelization.

The post Selenium to Playwright Migration Part 2: Setup and Configuration appeared first on Software Testing & Automation.

Transform Selenium into AI-First Automation

2026-06-23T21:14:15Z

Transform Selenium into AI-First Automation

Executive Summary

Software engineering is entering a period of unprecedented acceleration.

AI-assisted development tools are dramatically increasing the amount of software organizations can create. As code generation becomes faster, cheaper, and increasingly autonomous, the bottleneck in software delivery is shifting.

The challenge is no longer generating software.

The challenge is generating confidence.

Most enterprises have already invested heavily in testing infrastructure. Years of Selenium development, thousands of automated tests, custom frameworks, CI/CD integrations, reporting systems, and organizational expertise represent significant corporate assets.

Many engineering leaders now face a difficult question:

How can organizations become AI-first without discarding years of automation investment?

The answer is AI-transformation.

Rather than replacing existing automation, organizations can use AI to analyze existing Selenium suites, recover embedded business intent, identify hidden assumptions, generate AI-first test definitions, and execute them using modern Computer Use Agents (CUAs).

The result is not merely new automation.

The result is a testing organization that is better aligned with an AI-first future.

The New Testing Challenge

For more than two decades, software testing has focused on automating user interactions.The dominant model has been procedural automation:

Click this button
Wait for this selector
Enter this value
Verify this exact string

Frameworks such as Selenium, Playwright, Cypress, and Appium have been enormously successful because they allow organizations to automate repetitive activities at scale.

However, these frameworks share a common characteristic.

They encode implementation details.

When interfaces change, workflows evolve, selectors move, or systems behave differently than expected, automation requires maintenance.

Historically, this tradeoff was acceptable because application change rates were relatively manageable.

That assumption is changing.

AI-generated software is increasing both the volume and velocity of change.

Organizations are beginning to discover that the limiting factor is not code generation.

It is maintaining confidence in rapidly changing systems.

This creates an opportunity to rethink how automation itself is represented.

Why Existing Selenium Assets Matter

A common mistake during modernization initiatives is treating existing automation as technical debt.

In reality, most mature Selenium suites represent a significant corporate asset.

Large organizations often have hundreds, thousands, or even tens of thousands of automated tests. Those suites frequently represent years of engineering effort and, in many cases, millions of dollars of cumulative investment.

That investment is not just the code itself.

It includes:

Business workflows
Customer journeys
Domain expertise
Risk models
Compliance requirements
Historical defect knowledge
Integration logic
Validation rules
Operational experience

The scripts themselves are not the asset.

The knowledge trapped inside them is.

Viewed through this lens, Selenium becomes less of a testing framework and more of a repository of institutional knowledge.

Every automated test represents a decision someone made about what matters to the business.

Every validation captures a requirement.

Every workflow encodes understanding about how customers, employees, partners, or systems are expected to behave.

The challenge is not replacing that investment.

The challenge is extracting, preserving, and modernizing it.

For many enterprises, the automation portfolio represents millions of dollars of accumulated testing investment. The objective is not replacement. The objective is preservation and transformation.

Organizations that approach AI-first testing as a wholesale replacement effort often underestimate the value already embedded in their automation portfolio.

Organizations that approach it as a transformation effort can preserve years of accumulated knowledge while gaining the benefits of AI-native execution.

This is why the goal is not to throw away Selenium.

The goal is to transform it.

A Four-Step Transformation Framework

The transformation process consists of four major phases:

Analyze Code for AI Readiness
Q&A with AI
Generate AI-First Test Definitions
Run and Validate Loop

See content credentials

AI Transformation Process

Each phase incrementally converts procedural automation into AI-first automation.

Step 1: Analyze Code for AI Readiness

Before transformation can begin, AI must understand not only what tests do but also what they depend on.

AI reviews:

Selenium scripts
Page Objects
Helper libraries
Setup scripts
Teardown logic
Fixtures
Configuration files
Utility code
Supporting source code

The objective is not merely to understand what the tests do.

The objective is to understand everything they depend on.

Hidden Assumptions

Many automation suites rely on assumptions that are never explicitly documented.

Examples include:

Environment variables
Pre-seeded databases
Shared state
Generated accounts
Feature flags
Deployment configuration
Scheduled jobs

AI is remarkably effective at identifying assumptions that appear to exist but are not visible in the current code.

External Dependencies

Most automation suites interact with:

APIs
Databases
Queues
Authentication systems
Internal platforms
Third-party services

AI evaluates whether these dependencies can be externalized through APIs, MCP endpoints, services, or reusable execution utilities.

Dynamic Runtime Data

Many tests generate:

User accounts
Tokens
Temporary records
Unique identifiers
Transaction references

These flows must be identified and transformed into reusable runtime services.

Test Dependencies

Large Selenium portfolios frequently contain hidden sequencing requirements.

For example:

Test B assumes Test A executed.
Test C depends on Test B’s output.

AI identifies these relationships and recommends opportunities for decoupling.

Validation Logic

Assertions may exist in:

UI elements
APIs
Databases
Logs
Reporting systems

Understanding these validations is critical because they ultimately define what success means.

The output of this phase is an AI Readiness assessment and a roadmap for transformation.

The scripts themselves are not the asset.

The knowledge trapped inside them is.

Step 2: Q&A with AI

One of the biggest surprises during transformation projects is how much knowledge exists outside the code.

Even after analyzing every Selenium file, helper library, page object, setup script, and supporting framework, there are usually important assumptions that remain undocumented.

Questions such as:

Why is this workflow executed in this order?
What business rule is this validation actually checking?
Why is this value generated dynamically?
What assumptions are being made about acceptable outcomes?
What conditions would make this workflow fail?
What edge cases were the original test authors worried about?

Historically, recovering this information required tracking down the original automation engineer, developer, tester, or product owner.

AI changes that process.

After analyzing the automation suite, AI generates a focused list of questions designed to recover missing intent.

The AI is effectively asking:

“What would I need to know to confidently transform this test?”

This is an important shift.

Instead of humans manually reverse-engineering years of automation, AI performs the analysis and identifies the specific knowledge gaps.

In practice, most tests require surprisingly little clarification.

Many require no clarification at all.

Most require only one or two answers.

A small percentage contain complex business logic, hidden dependencies, or operational assumptions that require deeper discussion.

The answers are then fed back into the transformation process.

At this point, the automation is no longer just code.

It has become a combination of code, business intent, operational knowledge, and organizational expertise.

That knowledge becomes the foundation for AI-first execution.

Step 3: Generate AI-First Test Definitions

Once assumptions, dependencies, and intent have been captured, transformation can begin.

Large Selenium portfolios often contain hundreds or thousands of tests.

Rather than attempting to transform everything simultaneously, AI first ranks tests according to transformation complexity.

Simple tests are transformed first.

Complex tests are flagged for review.

This approach delivers value quickly while reducing transformation risk.

From Procedures to Intent

Traditional Selenium automation focuses on implementation details.

Examples include:

Click this button.
Wait for this selector.
Enter this value.
Verify this exact string.
Check this specific DOM element.

AI-first automation focuses on outcomes.

Examples include:

Create a new customer account.
Complete checkout successfully.
Verify the customer receives a confirmation.
Ensure the order total appears reasonable.
Confirm the workflow behaves as expected.

This distinction is critical.

The goal is not to describe every action.

The goal is to describe what success looks like.

This allows execution systems to adapt when applications evolve.

In many cases, providing less implementation detail produces more resilient automation.

Overly procedural instructions frequently create brittle tests.

Intent-driven definitions create flexibility.

The challenge is finding the right balance between precision and adaptability.

The New Test Artifact

The output of the transformation process is not traditional test code.

Instead, the system generates structured AI-first test definitions.

These definitions capture:

Business intent
Expected outcomes
Validation rules
Runtime parameters
Execution hints
Contextual information
Operational assumptions

OpenTest.AI formats are great for storing AI-First test definitions in a structured schema optimized for AI-driven execution

The schema becomes the modern, AI-first equivalent of the original Selenium script.

Instead of telling the browser exactly what to click, it tells the execution engine what the test is trying to prove.

Example AI-First Test Definition

See content credentials

OpenTest.AI Test Schema

The structure intentionally separates:

Title
Goal
Description
Thinking
Action
Target
Value
Expected Outcome

The inclusion of the thinking field is particularly important.

Traditional automation captures actions.

AI-first automation captures reasoning.

By preserving the reasoning behind a step, execution systems can make better decisions when interfaces change, workflows evolve, or unexpected conditions appear.

This structure is intentionally simple.

It is easy for humans to read.

It is easy for AI systems to generate.

It is easy for Computer Use Agents to execute.

The result is an artifact that serves both humans and machines.

Computer Use Agents (CUAs)

Once AI-first definitions have been created, they must be executed.

This is where Computer Use Agents enter the picture.

A Computer Use Agent (CUA) is an AI system capable of interacting with software applications similarly to a human tester.

A CUA can:

Observe interfaces
Interpret context
Make decisions
Interact with controls
Recover from disruptions
Continue pursuing an intended outcome

Unlike traditional automation frameworks, Computer Use Agents make many decisions at runtime.

Traditional Selenium determines how to interact with an application before execution.

A Computer Use Agent determines how to interact with the application during execution.

This distinction fundamentally changes how automation behaves.

Selenium decides the path ahead of time.

A Computer Use Agent discovers the path at runtime.

When selectors change, dialogs appear, layouts move, or workflows evolve, a Computer Use Agent can often adapt and continue without requiring maintenance.

This makes the execution model substantially more resilient than traditional procedural automation.

CUA Landscape Is Moving Fast

Computer Use Agents are evolving rapidly.

Organizations evaluating AI-first testing should understand the strengths and weaknesses of current platforms, and consider their internal systems and AI-decisions.

For an up-to-date comparison of:

OpenAI Operator
Anthropic Computer Use
Google Mariner
Browser Use
Manus
Open-source alternatives

ChatGPT CUA Comparison

The landscape is changing quickly enough that any static comparison will become outdated.

The important takeaway is not which CUA wins today.

The important takeaway is that the execution model itself is changing.

Step 4: Run and Validate Loop

Transformation is not complete when definitions are generated.

They must be executed, validated, and refined.

The execution loop performs the following activities:

Load AI-first test definitions.
Prepare runtime data.
Execute workflows through a Computer Use Agent.
Capture evidence.
Validate outcomes.
Review failures.
Refine definitions.
Execute again.

Runtime Data Preparation

Many workflows require information before execution begins.

Examples include:

Creating test accounts
Generating credentials
Requesting tokens
Calling APIs
Seeding databases
Retrieving environment information

Historically, these activities were embedded within helper code and setup scripts.

In AI-first architectures, they can often be externalized into reusable services, APIs, MCP endpoints, or utility functions.

The resulting data is then provided to the Computer Use Agent at runtime.

This keeps test definitions focused on intent while still supporting complex execution requirements.

Validation Becomes Richer

One of the most important benefits of AI-first execution is richer validation.

Traditional automation typically validates exact conditions.

Examples include:

Exact text matches
Exact values
Specific selectors
Exact DOM structures

AI-first automation can still perform precise validations when necessary.

However, it can also evaluate broader outcomes.

Examples include:

Does the workflow appear successful?
Does the page appear visually broken?
Does the content make sense?
Does the user experience appear confusing?
Does the page align with brand guidelines?
Does the result match business expectations?

This allows organizations to validate aspects of quality that were previously difficult or expensive to automate.

The Transformation Loop

The first execution pass is rarely perfect.

Some tests transform successfully immediately.

Others reveal hidden assumptions, missing data, environmental dependencies, or workflow ambiguities.

Instead of manually rewriting every failure, AI reviews the evidence and asks:

What information was missing?
What assumption was incorrect?
What additional context would help?
What should be clarified by a human?

The answers are incorporated into the definitions.

The tests are executed again.

Each cycle increases transformation quality.

The loop does not simply test the application.

It tests the transformation itself.

Why Transform Selenium into AI-First Automation?

Most organizations begin this journey because they want to reduce maintenance.

That is understandable.

Selector maintenance, brittle automation, false failures, and ongoing script updates consume significant engineering time.

However, maintenance reduction is only one benefit.

The larger opportunity is strategic.

Organizations are increasingly adopting AI-first engineering strategies.

Executives want AI-first development.

They want AI-first operations.

Increasingly, they want AI-first testing.

Transforming Selenium into AI-first automation provides a practical bridge between existing investments and future architectures.

Rather than discarding years of work, organizations can preserve existing knowledge while modernizing execution.

This allows testing to evolve alongside software development.

ROI Matters More Than Token Counts

One of the most common questions about AI-first testing involves token consumption.

In practice, token consumption is usually not the most important metric.

The more useful questions are:

How much engineering effort has been eliminated? How much faster is my team?

Every hour spent:

Updating selectors
Debugging automation failures
Investigating false negatives
Maintaining brittle scripts
Repairing workflows after UI changes

has a real cost.

Organizations should evaluate:

Reduced maintenance
Faster adaptation to change
Improved reliability
Increased engineering productivity
Faster releases
Future-proof architecture

Many teams discover that maintenance savings alone justify transformation.

The strategic value extends far beyond execution costs.

From Theory to Practice

IcebergQA’s Experts leverage Automated Transformation Workflows

Organizations can ad-hoc perform transformation with ChatBots and LLMs, and larger organizations can internally automate some of the transformation with integrations of traditional software and LLMs.

IcebergQA has implemented an automated implementation of the tranformation process and supported by IcebergQA transformation services. It is also possible to run this system on your own private cloud infrastructure.

The platform can import:

Selenium
Playwright
Cypress
Custom automation frameworks
Manual test cases
Requirements and user stories

It automates:

AI Readiness analysis
Question generation
Transformation
CUA execution
Validation
Iterative improvement

The goal is not merely to automate execution.

The goal is to accelerate organizational transformation, and IcebergQA provides 30-day AI QA sprints to help your team transform as quickly and confidencly as possible.

Conclusion

Organizations do not need to choose between preserving their existing automation investments and becoming AI-first. They can do both. The path forward is transformation: preserving the knowledge already embedded within Selenium suites, recovering the business intent behind those tests, modernizing the execution model, and building a testing architecture capable of scaling alongside AI-generated software.

As AI accelerates software development, the challenge is no longer generating code — it is generating confidence. Organizations that successfully transform their existing automation assets into AI-first testing systems will be better positioned to move faster, maintain quality, reduce maintenance costs, and adapt to the next generation of software engineering. Don’t throw away your Selenium investment. Transform it.

— Jason Arbon, CEO Testers.ai, Principal IcebergQA.com

Transform Selenium into AI-First Automation

2026-06-23T21:14:15Z

Transform Selenium into AI-First Automation

Executive Summary

Software engineering is entering a period of unprecedented acceleration.

The challenge is no longer generating software.

The challenge is generating confidence.

Many engineering leaders now face a difficult question:

How can organizations become AI-first without discarding years of automation investment?

The answer is AI-transformation.

The result is not merely new automation.

The result is a testing organization that is better aligned with an AI-first future.

The New Testing Challenge

For more than two decades, software testing has focused on automating user interactions.The dominant model has been procedural automation:

Click this button
Wait for this selector
Enter this value
Verify this exact string

Frameworks such as Selenium, Playwright, Cypress, and Appium have been enormously successful because they allow organizations to automate repetitive activities at scale.

However, these frameworks share a common characteristic.

They encode implementation details.

When interfaces change, workflows evolve, selectors move, or systems behave differently than expected, automation requires maintenance.

Historically, this tradeoff was acceptable because application change rates were relatively manageable.

That assumption is changing.

AI-generated software is increasing both the volume and velocity of change.

Organizations are beginning to discover that the limiting factor is not code generation.

It is maintaining confidence in rapidly changing systems.

This creates an opportunity to rethink how automation itself is represented.

Why Existing Selenium Assets Matter

A common mistake during modernization initiatives is treating existing automation as technical debt.

In reality, most mature Selenium suites represent a significant corporate asset.

That investment is not just the code itself.

It includes:

Business workflows
Customer journeys
Domain expertise
Risk models
Compliance requirements
Historical defect knowledge
Integration logic
Validation rules
Operational experience

The scripts themselves are not the asset.

The knowledge trapped inside them is.

Viewed through this lens, Selenium becomes less of a testing framework and more of a repository of institutional knowledge.

Every automated test represents a decision someone made about what matters to the business.

Every validation captures a requirement.

Every workflow encodes understanding about how customers, employees, partners, or systems are expected to behave.

The challenge is not replacing that investment.

The challenge is extracting, preserving, and modernizing it.

For many enterprises, the automation portfolio represents millions of dollars of accumulated testing investment. The objective is not replacement. The objective is preservation and transformation.

Organizations that approach AI-first testing as a wholesale replacement effort often underestimate the value already embedded in their automation portfolio.

Organizations that approach it as a transformation effort can preserve years of accumulated knowledge while gaining the benefits of AI-native execution.

This is why the goal is not to throw away Selenium.

The goal is to transform it.

A Four-Step Transformation Framework

The transformation process consists of four major phases:

Analyze Code for AI Readiness
Q&A with AI
Generate AI-First Test Definitions
Run and Validate Loop

See content credentials

AI Transformation Process

Each phase incrementally converts procedural automation into AI-first automation.

Step 1: Analyze Code for AI Readiness

Before transformation can begin, AI must understand not only what tests do but also what they depend on.

AI reviews:

Selenium scripts
Page Objects
Helper libraries
Setup scripts
Teardown logic
Fixtures
Configuration files
Utility code
Supporting source code

The objective is not merely to understand what the tests do.

The objective is to understand everything they depend on.

Hidden Assumptions

Many automation suites rely on assumptions that are never explicitly documented.

Examples include:

Environment variables
Pre-seeded databases
Shared state
Generated accounts
Feature flags
Deployment configuration
Scheduled jobs

AI is remarkably effective at identifying assumptions that appear to exist but are not visible in the current code.

External Dependencies

Most automation suites interact with:

APIs
Databases
Queues
Authentication systems
Internal platforms
Third-party services

AI evaluates whether these dependencies can be externalized through APIs, MCP endpoints, services, or reusable execution utilities.

Dynamic Runtime Data

Many tests generate:

User accounts
Tokens
Temporary records
Unique identifiers
Transaction references

These flows must be identified and transformed into reusable runtime services.

Test Dependencies

Large Selenium portfolios frequently contain hidden sequencing requirements.

For example:

Test B assumes Test A executed.
Test C depends on Test B’s output.

AI identifies these relationships and recommends opportunities for decoupling.

Validation Logic

Assertions may exist in:

UI elements
APIs
Databases
Logs
Reporting systems

Understanding these validations is critical because they ultimately define what success means.

The output of this phase is an AI Readiness assessment and a roadmap for transformation.

The scripts themselves are not the asset.

The knowledge trapped inside them is.

Step 2: Q&A with AI

One of the biggest surprises during transformation projects is how much knowledge exists outside the code.

Even after analyzing every Selenium file, helper library, page object, setup script, and supporting framework, there are usually important assumptions that remain undocumented.

Questions such as:

Why is this workflow executed in this order?
What business rule is this validation actually checking?
Why is this value generated dynamically?
What assumptions are being made about acceptable outcomes?
What conditions would make this workflow fail?
What edge cases were the original test authors worried about?

Historically, recovering this information required tracking down the original automation engineer, developer, tester, or product owner.

AI changes that process.

After analyzing the automation suite, AI generates a focused list of questions designed to recover missing intent.

The AI is effectively asking:

“What would I need to know to confidently transform this test?”

This is an important shift.

Instead of humans manually reverse-engineering years of automation, AI performs the analysis and identifies the specific knowledge gaps.

In practice, most tests require surprisingly little clarification.

Many require no clarification at all.

Most require only one or two answers.

A small percentage contain complex business logic, hidden dependencies, or operational assumptions that require deeper discussion.

The answers are then fed back into the transformation process.

At this point, the automation is no longer just code.

It has become a combination of code, business intent, operational knowledge, and organizational expertise.

That knowledge becomes the foundation for AI-first execution.

Step 3: Generate AI-First Test Definitions

Once assumptions, dependencies, and intent have been captured, transformation can begin.

Large Selenium portfolios often contain hundreds or thousands of tests.

Rather than attempting to transform everything simultaneously, AI first ranks tests according to transformation complexity.

Simple tests are transformed first.

Complex tests are flagged for review.

This approach delivers value quickly while reducing transformation risk.

From Procedures to Intent

Traditional Selenium automation focuses on implementation details.

Examples include:

Click this button.
Wait for this selector.
Enter this value.
Verify this exact string.
Check this specific DOM element.

AI-first automation focuses on outcomes.

Examples include:

Create a new customer account.
Complete checkout successfully.
Verify the customer receives a confirmation.
Ensure the order total appears reasonable.
Confirm the workflow behaves as expected.

This distinction is critical.

The goal is not to describe every action.

The goal is to describe what success looks like.

This allows execution systems to adapt when applications evolve.

In many cases, providing less implementation detail produces more resilient automation.

Overly procedural instructions frequently create brittle tests.

Intent-driven definitions create flexibility.

The challenge is finding the right balance between precision and adaptability.

The New Test Artifact

The output of the transformation process is not traditional test code.

Instead, the system generates structured AI-first test definitions.

These definitions capture:

Business intent
Expected outcomes
Validation rules
Runtime parameters
Execution hints
Contextual information
Operational assumptions

OpenTest.AI formats are great for storing AI-First test definitions in a structured schema optimized for AI-driven execution

The schema becomes the modern, AI-first equivalent of the original Selenium script.

Instead of telling the browser exactly what to click, it tells the execution engine what the test is trying to prove.

Example AI-First Test Definition

See content credentials

OpenTest.AI Test Schema

The structure intentionally separates:

Title
Goal
Description
Thinking
Action
Target
Value
Expected Outcome

The inclusion of the thinking field is particularly important.

Traditional automation captures actions.

AI-first automation captures reasoning.

By preserving the reasoning behind a step, execution systems can make better decisions when interfaces change, workflows evolve, or unexpected conditions appear.

This structure is intentionally simple.

It is easy for humans to read.

It is easy for AI systems to generate.

It is easy for Computer Use Agents to execute.

The result is an artifact that serves both humans and machines.

Computer Use Agents (CUAs)

Once AI-first definitions have been created, they must be executed.

This is where Computer Use Agents enter the picture.

A Computer Use Agent (CUA) is an AI system capable of interacting with software applications similarly to a human tester.

A CUA can:

Observe interfaces
Interpret context
Make decisions
Interact with controls
Recover from disruptions
Continue pursuing an intended outcome

Unlike traditional automation frameworks, Computer Use Agents make many decisions at runtime.

Traditional Selenium determines how to interact with an application before execution.

A Computer Use Agent determines how to interact with the application during execution.

This distinction fundamentally changes how automation behaves.

Selenium decides the path ahead of time.

A Computer Use Agent discovers the path at runtime.

When selectors change, dialogs appear, layouts move, or workflows evolve, a Computer Use Agent can often adapt and continue without requiring maintenance.

This makes the execution model substantially more resilient than traditional procedural automation.

CUA Landscape Is Moving Fast

Computer Use Agents are evolving rapidly.

Organizations evaluating AI-first testing should understand the strengths and weaknesses of current platforms, and consider their internal systems and AI-decisions.

For an up-to-date comparison of:

OpenAI Operator
Anthropic Computer Use
Google Mariner
Browser Use
Manus
Open-source alternatives

ChatGPT CUA Comparison

The landscape is changing quickly enough that any static comparison will become outdated.

The important takeaway is not which CUA wins today.

The important takeaway is that the execution model itself is changing.

Step 4: Run and Validate Loop

Transformation is not complete when definitions are generated.

They must be executed, validated, and refined.

The execution loop performs the following activities:

Load AI-first test definitions.
Prepare runtime data.
Execute workflows through a Computer Use Agent.
Capture evidence.
Validate outcomes.
Review failures.
Refine definitions.
Execute again.

Runtime Data Preparation

Many workflows require information before execution begins.

Examples include:

Creating test accounts
Generating credentials
Requesting tokens
Calling APIs
Seeding databases
Retrieving environment information

Historically, these activities were embedded within helper code and setup scripts.

In AI-first architectures, they can often be externalized into reusable services, APIs, MCP endpoints, or utility functions.

The resulting data is then provided to the Computer Use Agent at runtime.

This keeps test definitions focused on intent while still supporting complex execution requirements.

Validation Becomes Richer

One of the most important benefits of AI-first execution is richer validation.

Traditional automation typically validates exact conditions.

Examples include:

Exact text matches
Exact values
Specific selectors
Exact DOM structures

AI-first automation can still perform precise validations when necessary.

However, it can also evaluate broader outcomes.

Examples include:

Does the workflow appear successful?
Does the page appear visually broken?
Does the content make sense?
Does the user experience appear confusing?
Does the page align with brand guidelines?
Does the result match business expectations?

This allows organizations to validate aspects of quality that were previously difficult or expensive to automate.

The Transformation Loop

The first execution pass is rarely perfect.

Some tests transform successfully immediately.

Others reveal hidden assumptions, missing data, environmental dependencies, or workflow ambiguities.

Instead of manually rewriting every failure, AI reviews the evidence and asks:

What information was missing?
What assumption was incorrect?
What additional context would help?
What should be clarified by a human?

The answers are incorporated into the definitions.

The tests are executed again.

Each cycle increases transformation quality.

The loop does not simply test the application.

It tests the transformation itself.

Why Transform Selenium into AI-First Automation?

Most organizations begin this journey because they want to reduce maintenance.

That is understandable.

Selector maintenance, brittle automation, false failures, and ongoing script updates consume significant engineering time.

However, maintenance reduction is only one benefit.

The larger opportunity is strategic.

Organizations are increasingly adopting AI-first engineering strategies.

Executives want AI-first development.

They want AI-first operations.

Increasingly, they want AI-first testing.

Transforming Selenium into AI-first automation provides a practical bridge between existing investments and future architectures.

Rather than discarding years of work, organizations can preserve existing knowledge while modernizing execution.

This allows testing to evolve alongside software development.

ROI Matters More Than Token Counts

One of the most common questions about AI-first testing involves token consumption.

In practice, token consumption is usually not the most important metric.

The more useful questions are:

How much engineering effort has been eliminated? How much faster is my team?

Every hour spent:

Updating selectors
Debugging automation failures
Investigating false negatives
Maintaining brittle scripts
Repairing workflows after UI changes

has a real cost.

Organizations should evaluate:

Reduced maintenance
Faster adaptation to change
Improved reliability
Increased engineering productivity
Faster releases
Future-proof architecture

Many teams discover that maintenance savings alone justify transformation.

The strategic value extends far beyond execution costs.

From Theory to Practice

IcebergQA’s Experts leverage Automated Transformation Workflows

Organizations can ad-hoc perform transformation with ChatBots and LLMs, and larger organizations can internally automate some of the transformation with integrations of traditional software and LLMs.

The platform can import:

Selenium
Playwright
Cypress
Custom automation frameworks
Manual test cases
Requirements and user stories

It automates:

AI Readiness analysis
Question generation
Transformation
CUA execution
Validation
Iterative improvement

The goal is not merely to automate execution.

The goal is to accelerate organizational transformation, and IcebergQA provides 30-day AI QA sprints to help your team transform as quickly and confidencly as possible.

Conclusion

— Jason Arbon, CEO Testers.ai, Principal IcebergQA.com

Fixing Cypress alias warnings with template literals

2026-06-23T17:00:00Z

After upgrading to the latest Cypress (15.17.0), a pile of TypeScript warnings suddenly showed up across our test suite.

Argument of type 'string' is not assignable to parameter of type `@${string}`.

The warning you’ll start seeing

You probably know the following pattern. You create an alias with a unique uuid in it. We do that to keep aliases apart across parallel or repeated calls. Later in the code, your read it out with a string concatenation. Something like this:

cy.get<Cat>('@createCat' + uuid).then((cat: Cat) => {
 cy.wrap(cat).as(wrappedAs);
});

Until recently that compiled without complaint. Since the latest upgrade of Cypress, TypeScript flags it with something like:

Argument of type 'string' is not assignable to parameter of type `@${string}`.

A warning like this is a future problem in disguise. It doesn’t break the test today, but it signals that your code no longer matches what the library expects. If you leave it, it tends to pile up and hide real issues in the noise. A later upgrade may turn the warning into a hard error. So we fix them now, while the change is small and well understood.

What is a backtick string?

To understand the fix, you need to know about strings in JavaScript and TypeScript. Besides single quotes (’…’) and double quotes ("…"), there is also the template literal. A template literal is a string written with backticks (`...`) instead of quotes, and inside it you can embed any value with the ${ … } syntax. Here is the difference:

const name = 'Whiskers';

const a = 'Hello ' + name; // concatenation with +
const b = `Hello ${name}`; // template literal with ${...}
// a and b are both "Hello Whiskers"

Inside a backtick string you can drop in any expression with ${ … }, and its value gets inserted into the string. It’s the modern, readable way to build a string out of fixed text and variables, without gluing pieces together with the plus sign.

TypeScript takes this one step further with template literal types. A type can be written the same way to describe the shape of a string. For example:

type Alias = `@${string}`; // any string that starts with "@"

@${string} is not a value. It’s a type that says “a string beginning with the @-sign, followed by anything”. Keep that in mind, because it’s exactly the type Cypress now expects for an alias name.

Why Cypress changed this

This isn’t a mistake on our side, and it isn’t Cypress being picky for no reason. It’s a deliberate improvement. The Cypress 15.17.0 changelog describes it like this:

Improved TypeScript types for cy.get(’@alias’) and cy.wait(’@alias’) so that @-prefixed strings correctly resolve to Chainable<S> instead of Chainable<JQuery<HTMLElement>>.

Previously, cy.get('@createCat' + uuid) gave you back a generic Chainable<JQuery<HTMLElement>>, as if you were selecting a DOM element, even though the alias actually holds a Cat type. You then had to cast it yourself. Now that is done automatically.

To return the correct type, Cypress first has to recognise that an argument is an alias and not a CSS selector. It does that by tightening the expected argument type to the template literal type `@${string}`: a string that starts with the @-sign. When Cypress sees that, it knows to resolve the aliased type (Chainable<S>) instead of a DOM element.

And that’s where our warning comes from. The result of '@createCat' + uuid is, as far as TypeScript is concerned, just a plain string. The compiler cannot prove it starts with @-sign, so it does not fit `@${string}`. Hence the warning.

The fix: template literals

The fix is small and reads better than the original. Swap the concatenation for a template literal:

// Before
cy.get<Cat>('@createCat' + uuid).then((cat: Cat) => { ... });

// After
cy.get<Cat>(`@createCat${uuid}`).then((cat: Cat) => { ... });

Because the template literal literally starts with @, TypeScript can see that it satisfies `@${string}`. The warning disappears, and as a bonus you now get the correct type back out of the chain.

The same applies to cy.wait:

// Before
cy.wait('@loadDogs' + uuid);

// After
cy.wait(`@loadDogs${uuid}`);

And when several variables are involved, you just fold them into the same literal:

// Before
cy.get('@adoptDog' + index + uuid);

// After
cy.get(`@adoptDog${index}${uuid}`);

Don’t touch `.as()`

A common misconception: only reading an alias got stricter. Creating one with .as() still expects a plain string, without the @. You can change it, but you do not have to change it now.

cy.wrap(cat).as('createCat' + uuid); // fine: no @, no template needed

Sources

Cypress changelog (version 15.17.0)
Variables and Aliases | Cypress Documentation
Template literal types | TypeScript

DeepEval vs PromptFoo for SDET Teams

2026-06-23T04:36:05Z

DeepEval vs PromptFoo for SDET teams: compare eval workflows, code examples, CI strategy, and career value for QA engineers.

The post DeepEval vs PromptFoo for SDET Teams appeared first on Software Testing & Automation.

AI Made Engineers Faster. It Also Made Systems More Fragile.

2026-06-23T03:51:01Z

A few months ago, a developer told me something that perfectly captures the current state of software engineering:

Continue reading on Medium »

Playwright MCP for QA Engineers: Practical Guide

2026-06-23T03:35:08Z

Playwright MCP for QA engineers helps turn AI browser exploration into repeatable Playwright checks with evidence, evals, and safe workflows.

The post Playwright MCP for QA Engineers: Practical Guide appeared first on Software Testing & Automation.

Playwright Sharding: Day 15 Parallel CI Tutorial

2026-06-23T03:34:39Z

Playwright sharding splits slow TypeScript test suites across CI runners. Learn workers, GitHub Actions matrix jobs, report merging, and isolation.

The post Playwright Sharding: Day 15 Parallel CI Tutorial appeared first on Software Testing & Automation.

Book Review: Software Testing with Generative AI

2026-06-23T00:00:00Z

My insights from "Software Testing with Generative AI" book by Mark Winteringham

Selenium to Playwright Migration Part 1: Why Migrate and How to Plan It

2026-06-22T22:00:00Z

Selenium vs Playwright feature matrix, architecture comparison, TypeScript quick reference for Java developers, and the 4-step bottom-up migration strategy.

The post Selenium to Playwright Migration Part 1: Why Migrate and How to Plan It appeared first on Software Testing & Automation.

Karpathy’s LLM Wiki as Agent Memory

2026-06-22T21:29:25Z

A practical look at how Karpathy’s LLM Wiki pattern can serve as long term agent memory, turning raw sources, markdown pages, and schema instructions into a living knowledge system for semantic, entity, episodic, summary, and procedural memory.

The post Karpathy’s LLM Wiki as Agent Memory appeared first on Angie Jones.

One Second

2026-06-22T15:04:58Z

Many years ago I presented a project timeline to a room of about a dozen people. A junior engineer, maybe eighteen months into her career, raised her hand. “I don’t think we’ve accounted for the migration work. I think that’s another two weeks.”

She was right. I knew it about four words in.

I don’t remember deciding to get defensive. I remember feeling it in my shoulders, something close to embarrassment, and then hearing myself say the migration was “already factored into the buffer.” It wasn’t. She didn’t push back a second time.

Six months later, that same engineer had a much bigger concern about a launch. She mentioned it to a peer instead of to me, and I understood exactly why.

The Thorn

That feeling, right before I opened my mouth is what’s on my mind today.

I recently read Michael Singer’s The Untethered Soul, and there’s an idea in it that I I find easy to apply to leadership. Singer calls it the thorn. Someone says something, or something doesn’t go your way, and it pokes you. A little spike of discomfort. Most people spend their whole lives doing one of two things with that spike. They build a wall around it so it can’t get poked again, or they react outward and try to make the discomfort someone else’s problem.

Neither one deals with the thorn. Both just move it around.

I wrote about this from a different angle a while back in Being Wrong at the Right Level. That post was about what happens after the reaction, the slow erosion of trust when a leader can’t admit a bad call. This is about the half second before that. The moment where the reflex fires and you still, technically, have a choice.

The Wall and the Spike

Singer’s framing is that most of us run one of two patterns when we get poked.

The wall. You get criticized once, it stings, and some part of you quietly decides not to let that happen again. You get a little more guarded. A little less open to feedback from that direction. Do this enough times over a career and you end up running a leadership style built entirely out of old wounds you never actually dealt with. You’re not protecting the team. You’re protecting the thorn.

The spike. This is the pattern Larry was showing in the newsletter linked above. Someone questions you, the discomfort shows up, and instead of sitting with it you push it back out. You get defensive, or you double down, or you find a reason the other person is wrong so you don’t have to feel the poke. It’s faster than building a wall, and it does more damage, because everyone with you watches it happen in real time.

This took me a long time to figure out. The discomfort isn’t the problem. You’re allowed to feel discomfort. The problem is what you do in the half second after you feel it, on autopilot, without ever deciding to.

Watching

Singer’s suggestion is almost annoyingly simple. When the spike happens, don’t immediately become the spike. Notice it.

In a leadership context this looks like a specific and learnable pause. Someone challenges your decision. You feel the pressure in your shoulders. Instead of letting that pressure push the next sentence out of your mouth, you let it be there for one extra second. You ask yourself what’s actually true, not what would make the discomfort go away fastest.

That one second is the whole skill. It’s not suppressing it, and you’re not pretending you didn’t feel it. Just refuse to let the feeling drive for an extra beat.

Trust over Feelings

I write about trust a lot, probably because it’s the actual currency of leadership. You can read They Did What You Said for the other half of this, what happens when your team complies without trusting you. This is upstream of that. Every time you react out of the spike instead of watching it, your team logs the data point. They learn what topics are safe to raise and which ones aren’t. They learn it faster than you’ll ever notice you taught it.

The wall does the same damage slower. A leader who’s quietly armored against a certain kind of feedback doesn’t look defensive in the moment. They just stop hearing that feedback at all. Nobody around them clocks it as a problem until something important doesn’t reach them in time.

Both patterns come from the same place. An old, automatic refusal to just feel the thorn and let it pass. The spike doesn’t care whether you’re in a conference room or a conversation with a good friend. The wall doesn’t care either.

The Practice

You don’t fix this by deciding to be calmer. That’s the wall and the spike both, in disguise, a new layer of management instead of actually dealing with the thing underneath.

What works is smaller and weirder than that. Next time you feel that spike in a meeting, the pressure in the shoulders, the urge to defend or shut down, try just naming it to yourself. Not out loud. Just internally: there’s the spike. You don’t have to do anything else yet. Just notice it’s there before you let it drive.

Do that enough times and the pause stops feeling like work. It’s just the half second you take before you let the truth land.

If you want to talk through what this looks like in your specific team, I’d like to chat with you (schedule time here).

Workroom PlayTime 060: Sitegeist as an Exploratory Interface

2026-06-22T14:23:10Z

This week's Workroom PlayTime is Thursday 2 July at 4:00pm London time. Please note the change of time.

We'll play with (abandonware) tool Sitegeist as an Exploratory Interface. It needs setup – and I need you to come with it already set up. If you're not already set it up, you won't be able to play.

Setup is easy, and you can do it yourself, but I'm happy to help if that helps you come along. Pop something in my diary to come and say hello, or drop in for the last 10 minutes of the session on 25 June. You'll need a machine that has the right permissions to let you install browser extensions, and you'll need a chromium browser (Chrome, Brave, Edge, Opera, Vivaldi and others).

We'll gather on Zoom. These exercises are for everyone, for free. All subscribers get to play together most weeks, paid subscribers get a guaranteed slot or rerun if we're full. If you get this in an email or can see the joining info section below, then you're a subscriber and have access to the Zoom etc. for the workshop. If you'd like to bring a friend, you can do that. I'll say yes until I've got too many.

Use this Savvycal link to book your spot and get reminders. Note to regulars: I hope that I've got it working so that not only is there a link to the zoom in this email, but in the "usual zoom" text in the reminder emails that savvycal sends.

↓ signed-in subscribers will see joining info and links below ↓

Workroom PlayTime 059: Play with Tenfold

2026-06-22T14:10:43Z

This week's Workroom PlayTime is Thursday 25 June at 2:30pm London time. Please note the time, which is back to usual.

We'll Play with Tenfold – where we'll explore a toy made by Ink and Switch. This will be a play-focussed session to see what we can find: their toy is both wide and deep, but doesn't particularly lend itself to exploration by iteratikon nor to the judgement that is fundamental to testing work.

Next time, for Workroom PlayTime 060, we'll finally get to Sitegeist as an Exploratory Interface. It needs setup – to help you get set up, come to a short session immediately after Thursday's thing, or pop something in my diary to come and say hello. You'll need a machine that has the right permissions to let you install browser extensions, and you'll need a chromium browser (Chrome, Brave, Edge, Opera, Vivaldi and others). It's easy, and you can do it yourself, but I'm happy to help if that helps you come play.

↓ signed-in subscribers will see joining info and links below ↓

The Great Capitulation

2026-06-22T10:27:08Z

You know where this is heading, right?

We’re being told that AI systems are too complex, too sophisticated, and too difficult to test and that it’s easier to try to conjure up a confidence index than it is to properly test them. Respected people in the software testing world are saying things like “you need to let it go and just like, trust the AI” and describing AI native test tools as a “magic trick”.

These are not serious people.

They would have you believe that uncertainty, context, risk, system behavior, and human judgment should be repackaged from the foundations of good quality management to a reason to lower the bar. Not openly, of course. Nobody says, “We’ve discovered testing and decided it’s too hard, so we are going to replace it with a vibes chart.” That would sound ridiculous. So instead, we get another BS test management theory.

The broader business world is already preparing for the quality capitulation to our AI “friends”. Cal Newport’s piece sets out that AI is not breaking work because it was already broken. We were already drowning in shallow busy work with middle management consistently mistaking movement for action. Modern testing practices turned this into a high art.

This article in Fortune lays out the idea that AI will not transform your business unless you redesign the work itself. Fair enough, but you can see the next step in this Register story about why Amazon isn’t in love with “human in the loop” because of how messy we are. I guess when we finally give up the desire for “baseline human comfort” like drinking water, we can quicken the birth of our super-intelligence overlord to tells us how confident we should be in the future.

As all these converging factors start to come into focus, the software testing industrial complex does their usual complete and utter abdication of their role as the conscience in technology to just make sure everyone likes them.

Instead of saying, “We need better accountability, better evidence, better models of risk, and better use of human expertise,” the AI Confidence Bros take their 30 pieces of silver for trying to quietly downgrade everyone’s ambitions.

The great irony of it all is that the exact same people who created, sold, and grifted the buyers of testing services through vanity metrics for the last 20 years, are now the exact same people trying to wash their hands of it all. The lack of self-awareness in creating another manifesto full of shallow binary choices in the style of the one that got us into this mess in the first place is breathtaking.

Shilling confidence is the final form of the enshittification of software testing. First, tools help testers. Then tools reshape testing around what vendors can automate. Finally, testing itself degrades into whatever the platform can measure, summarize, and sell back as “confidence.” The product gets worse, the language gets better, and everyone gets to pretend test has transformed.

But hey, want to feel confident? Just lean back and let it do it’s thing…I mean, it’s got electrolytes!

The post The Great Capitulation first appeared on Quality Remarks.

brawndo.mp4 (450kB)