blogs

Linky #32 - The practice is not the principle

2026-05-25T17:18:54Z

This week’s links had me thinking about something that is foundational to a lot of quality engineering work:

Are our practices still serving the principles behind them?

Because it is easy for teams to inherit the practices, rituals, tools, boards, ceremonies, and the language, without always keeping hold of why those things mattered in the first place.

That becomes a problem when the conditions change.

A practice that helped one team learn faster can become a habit that slows another team down. A process that created clarity in one context can create friction in another. A metric that once helped expose a problem can become the thing people optimise around.

Quality is not created by process alone. It is shaped by how people understand the work, how they respond to uncertainty, how they make trade-offs, and how willing they are to adapt when the system around them changes.

This week’s links look at that from a few different angles: the Product Operating Model, first principles, flexible team interactions, change as a social process, AI increasing output, and why understanding the system still matters.

Latest post from the QE Newsletter

Over the last couple of months, I’ve been deep diving into the Product Operating Model (POM), which a few software organisations are moving towards.

What stood out to me was that quality engineering could be an incredible enabler for it.

One advantage I see with the POM is that it gets organisations more focused on outcomes rather than outputs. That feels increasingly valuable when AI is accelerating output, but not necessarily helping us understand whether we are creating the right outcomes.

For me, this is where quality engineering has a lot to offer.

Not in testing the thing at the end, but in helping teams ask better questions earlier:

What are we trying to learn?
What feedback do we need?
What risks are we carrying?
What assumptions are we making?
How will we know if this is working?

I have a feeling we’ll be hearing more and more about the POM as time goes on, and I think there is a real opportunity for quality engineering to help shape how organisations make that shift in practice.

Getting back to first principles

The Agile Manifesto remains the best place to start. Not because it explains everything, it doesn’t, and the world is more complicated than four values and twelve principles.
But it’s first principles. Everything else, the practices, the ceremonies, the tooling, only makes sense in light of it. Without that foundation you’re just copying other people’s answers.

The Agile Manifesto is the foundation you build your agile practices on top of.

But I think this is where a lot of teams can get stuck. They inherit the practices, rituals, boards, ceremonies, and language, but not always the principles underlying them.

So when the environment changes, they struggle to adapt.

The practice becomes the thing to protect, rather than the principle that helped make the practice useful in the first place.

That matters for quality engineering because quality practices are the same. Testing, automation, exploratory testing, pairing, CI/CD, retrospectives, risk conversations, incident reviews, definitions of done. None of these are good just because we do them.

They are useful when they help us learn, reduce uncertainty, surface risk, improve decisions, and build better software.

Continuous improvement is meant to be a core part of the work, but in practice, it is often the thing teams drop when delivery pressure rises.

And that is usually the moment when they need it most. Via Agile Foundation: Beyond Practices to Shared Understanding | Rob Bowley

Adapting to the conditions

In Team Topologies, we frequently discuss setting up teams for flow, clarity, and managing cognitive load. It’s not just about picking one structure and sticking to it. Teams need to shift how they interact, depending on what’s happening.
Sometimes you need someone calling the shots, other times, you want to hear from everyone and learn as a group.

The example cited in the post is from the US Navy SEALs.

In the field, they operate with a clear hierarchy because mistakes can lead to serious injury or death. In that context, clear commands and people following through are critical.

But back at base, during debriefs, the dynamic changes. The structure becomes flatter because the purpose has changed. It is no longer about immediate coordination. It is about learning.

That distinction feels really relevant for engineering teams.

During a live incident, the priority is minimising harm to users, the business, and the team. In that moment, a more command-and-control approach can make sense. You need coordination, clarity, and quick decisions.

But during a post-incident review, the purpose is different.

You need to understand what happened, how it happened, what the system made easier or harder, and what you could improve next time. That requires openness, curiosity, and the willingness to hear different perspectives.

The mistake is thinking one way of working should apply everywhere.

Quality is not created by blindly applying the same process to every situation. It is created by understanding what the situation needs and adapting how we work around that.

During an incident, you need coordination.

After an incident, you need learning.

Via Navy SEALs’ Flexible Leadership Approach to Team Dynamics | Matthew Skelton

Change happens one conversation at a time

One from me this time:

... change happens person by person. Conversation by conversation. Standard by standard. Not all at once. Not because one heroic person fixes everything. But because enough people keep caring, talking, acting, and nudging the system in a better direction.

We work in big, complex adaptive systems, and trying to change them so quality is built in can feel almost impossible.

But in some ways, maybe that is not what we are trying to do.

We are not trying to fix the whole system in one heroic move. We are trying to change it little by little.

Through conversations, decisions, team habits, better feedback loop and when someone chooses to ask a better question instead of accepting the default.

That might sound small, but I think this is how a lot of quality work actually happens.

Quality engineering is not only about tools, pipelines, automation, or test strategies. It is also about helping people notice what the system is doing and nudging it towards something better.

That kind of change is slow, but it compounds.

And over time, enough small changes can start to move the whole system. Via There are some days when I think, what’s the point? | Jitesh Gosai

Quality Engineering Newsletter is reader-supported. If these posts help you think more clearly about quality, teams, systems, and how better software gets built, you can subscribe for free or become a paid subscriber.

Free subscriptions help more people find the newsletter. Paid subscriptions help me keep making time for the reading, writing, reflection, audio versions, and practical resources that go into it.

Subscribe now

Quality is not proportional to quantity

AI is very focused on more output…

Producing a garbage product faster benefits nobody; never has. It’s good outcomes (e.g., happy customers, painless improvements) that we need, not more output.

Which is a good point.

Just producing more does not mean producing more quality.

But I also think there is a useful nuance here. More attempts can create more opportunities to learn, try things, test assumptions, and find what might work.

The problem is when organisations confuse that activity with value.

More ideas, more prototypes, more code, more documentation, and more tests are not automatically useful. They become useful when they help us learn something, make a better decision, or move closer to an outcome that matters.

Then, once you have found something valuable, the work changes.

You need to productionise it. Make it scalable, performant, maintainable, and suitable for your context. You need to understand the risks. You need to build the feedback loops that tell you whether it is still working.

So quantity can help, but only if the system around it is good at learning.

Otherwise, all you have done is create more stuff to manage. I wrote about this in When quantity leads to quality.

Quote via Value Is Not Proportional to Quantity | Allen Holub

AI is increasing inventory, not value

AI is reducing the cost of creating inventory. And inventory is only valuable when it helps us learn, validate, deliver, or change customer and business outcomes. Until then, it is potential value at best. At worst, it is risk.

As Jacob mentions in his post, inventory can be things like code, tests, documentation, designs, tickets, and ideas.

AI makes it easier to create more of all of that.

But more inventory does not mean more value. We can do things faster, but that does not mean we are creating better outcomes faster.

This is where the system matters.

If your organisation is optimised for output, then that is what you are going to get. More code, tickets, PRs, documents and more visible signs of activity.

But if your organisation has built strong practices for learning and validation, then AI could be used very differently. It could help teams explore, compare options, surface risks, improve understanding, and test assumptions more quickly.

The difference is not the tool but what the surrounding system rewards, notices, and learns from.

So maybe the question is not simply “how much more can AI help us produce?”

Maybe the better question is:

What kind of learning does all this extra output create?

Via AI has made output cheap, moving the constraint from production to validation. | Jacob Clark

It’s never been about speed, but building quality in

I don’t usually post two posts from the same person, but Rob’s been on fire these last few weeks.

He’s been looking back as far as the 1940s, showing that the bottleneck in software has never really been about speed...

Maurice Wilkes, one of the pioneers of stored program computing, later reflected in Memoirs of a Computer Pioneer his realisation in the late 1940s, that “a good part of the remainder of my life was going to be spent in finding errors in my own programs.” From the very beginning, debugging and verification, not writing code, dominated effort.

Rob cites another seven examples showing that speed has rarely been the real issue.

The harder work has always been making sure what we have built actually does what we thought it would do.

That is quality work and its about reducing uncertainty in how our software systems behave.

This is why building quality in matters. It can feel slower at first because you are investing in the system around the work: better tests, better feedback loops, clearer design, smaller changes, stronger collaboration, more testable systems, and earlier conversations about risk.

But over time, that investment changes what the team is capable of.

You spend less time being surprised by the system, on finding issues, on untangling problems you could have made visible earlier.

So building quality in is not really about slowing down to be careful.

It is about creating a system that can move with more confidence because it is better at learning. Via Coding has never been the bottleneck | Rob Bowley

Are we heading for an AI disaster?

...whole swath of software engineers have forgotten why we ever cared about good code in the first place. It’s fine if your prototype is slop, but good code compounds over time so slop only really becomes a problem over time in a compounded nature, when it’s far too late.

This feels like one of the bigger quality engineering challenges we might be walking into with AI.

Not because AI is bad, but because AI may make it much easier to create software faster than we can understand it.

Metrics like MTTR are useful. They help teams embrace failure, detect problems, minimise harm, and recover from production incidents.

Throughput metrics like cycle time and deployment frequency are useful too. They help teams understand whether work is flowing.

But there is a potential trap here.

A team can be shipping frequently, recovering quickly, and still slowly losing comprehension of the system.

They might understand less about:

how the system actually works
why certain decisions were made
where the dependencies are
how components interact under stress
what assumptions the system is relying on

The system keeps evolving, but fewer people understand it deeply.

That is dangerous because some problems do not show up straight away. They accumulate quietly. Complexity, coupling, weak assumptions, and messy boundaries can sit unnoticed for a long time because everything still appears to be working.

Until one day, the conditions line up in just the wrong way.

The CrowdStrike outage is a good example of how a relatively small issue can cascade into something much larger when systems are deeply interconnected.

I think we are going to need better ways of noticing whether our understanding of a system is increasing or decreasing over time.

That’s quality engineering work to me.

Not just checking whether the system works today, but helping teams stay close enough to the work that they can understand, question, adapt, and intervene when things go wrong.

Because if AI helps us produce more while quietly reducing our shared understanding, then we may be improving output while weakening the system’s ability to learn. Via Software Engineers Forget Good Code at Their Own Risk | Christopher Grounds

As always, I’d love to know what you think. You can reply directly to one of my emails, leave a comment, or message me on socials.

Past Linkys

b6044fbb15cdafd60393fa24999f7e39.mp3

The cool chain of data

2026-05-25T15:12:26Z

Perishable goods like milk and some medicine need to be kept cool otherwise they go bad. With something like milk, if it has gone bad enough you can tell from the smell. However, this will probably be after you’ve paid for it and brought it home from the shop, so you have wasted time and money. Medicine is worse – you often can’t tell if is ineffective or dangerous without specialist equipment.

Things are made more complicated for both because there are many steps involved in getting them to you, and these steps can involve many different companies, technologies, risks etc. All these steps need to link together in an unbroken chain, hence the name cool chain. It doesn’t matter if the milk is kept refrigerated at the dairy farm, in the tanker, in the shop and in your fridge if it’s kept in an unrefrigerated warehouse along the way.

Photo by Bob Embleton, under Creative Commons Attribution-Share Alike Generic Licence 2.0
20 gallons of unpasteurised sales data

A cool chain is also a useful metaphor for data pipelines (I’ve written about other metaphors elsewhere). Data usually goes through a series of steps that transform it into some desired state, similar to the many steps involved in getting milk from farm to fridge.

As soon as a step is defective – letting the milk or data go bad in some way – that’s often it. You can’t unspoil the milk or easily fix all data problems. If there are problems with the data pipeline, often the best you can hope for is spotting problems as soon as they happen, to minimise the amount of data that has gone off.

That leads to another issue: How confident are you that you can detect problems? Are you doing the equivalent of merely looking at the milk, or are you smelling it or even checking its chemical composition? You might be processing data that has came from elsewhere. Before you go to the trouble of doing lots of processing, have you checked its quality? How much can you trust the earlier stages of the cool chain?

So far I’ve been talking about the data engineering kinds of data processing, but cool chain type thinking also applies to the more statistics kind of data processing. There are ways to summarise data where the output is only as strong as the weakest link, similar to worrying about the least cool step of the cool chain. This is things like how many decimal places can be in the answer if there are several steps of multiplication, division etc.

This has implications for things like when you round numbers e.g. to the nearest 1 decimal place, when you throw away values if they represent groups of people that are small enough to hit data privacy issues etc. If possible, you should round values, apply data privacy filters etc. as late as possible. Stretching the analogy a bit, they correspond to a relatively warm step in the cool chain, in that they define the maximum quality of the output regardless of what steps follow and how cool they are.

For example, if I have numbers that I need to sum to 1 decimal place, I could round each number to 1 decimal place and then add them. This would guarantee that the answer is to 1 decimal place, but it would be less accurate than if I add them unrounded and then round the sum.

If we take 9 lots of the value 0.11:

0.11 + 0.11 + 0.11 + 0.11 + 0.11 + 0.11 + 0.11 + 0.11 + 0.11 = 0.99 -> round to 1.0
0.11 -> round to 0.1 -> 0.1 + … + 0.1 = 0.9

There is a 10% difference between the two answers, and I would argue that this is because the second answer is incorrect by 10%.

Thinking about your data processing as a cool chain, whether that’s data engineering or statistics, could help you design it better and assess the quality of the data it produces in a cost-effective way.

You Don’t Need More AI Tools. You Need a Pipeline That Actually Pays.

2026-05-25T14:33:05Z

Most creators have ten tools and zero systems. Here is how to fix that.

Continue reading on Medium »

Checking Isn’t Testing. Soon It Won’t Be Employment Either

2026-05-25T12:30:03Z

The debate in the software testing industry over “checking versus testing“ has always annoyed the right people. The ones who say it’s just semantics. The ones who say it is elitist. The ones who say normal people don’t care about a “distinction without a difference”.

I mean, I get it, nobody likes being told that the thing they’ve been calling testing is the equivalent of a green check, but that distinction is about to become an economic reality that most testers won’t survive.

And the sad part won’t be that AI is taking that work, it’ll be that so many testers were told by the “anti-checking” crew that what they were doing was testing.

RST: Elements of Excellent Testing

Let’s face it: AI can already do checking.

Not perfectly. Not without some risk. But well enough to replace a great deal of the current population in our business. It doesn’t even need to be excellent.

It only needs to become cheaper, faster, and plausible enough to kick the teeth in of a large part of the software testing job market.

Once that problem gets close to being solved (a problem being worked on RIGHT NOW by your friends in the test tool industry), those jobs are going away and not coming back.

And a big part of why they aren’t coming back is because that money is moving.

Research and Markets values the AI model evaluation market at $2.36 billion in 2026 and forecasts it to reach $6.24 billion by 2030, with a 27.5 percent compound annual growth rate.

The market described in that report includes model validation, monitoring, benchmarking, compliance and governance, trending towards continuous validation, bias and fairness auditing, and explainable AI assessments.

The irony of all this is that this should have been the testers time to shine. Who should be better placed to test complex systems that behave unpredictably, depend on data, interact with humans, produce variable outputs, degrade over time, and fail in ways that are subtle, contextual, and expensive?

Testers!

Testers are investigators. Testers are system thinkers. Testers are people who understand that risk emerges from relationships between the system, the user, the environment, the business, the data, and time.

That’s why the “checking versus testing” argument was never about vocabulary, it was asking permission to stop thinking. If you define testing as checking, then AI is your competitor. If you define testing as skilled investigation, AI is a tool. And it only gets worse for the checking business from here.

A VentureBeat article on AI agents and chaos engineering describes a category of incident enterprises are not tracking well yet: autonomous agents taking technically reasonable actions from incomplete context that cause cascading problems in their infrastructure.

The article gives examples of agents restarting services, rerouting traffic, scaling resources, or modifying configurations in response to anomalies. In isolation, the action may make sense.

In the actual system state, with dependencies under pressure and other teams making changes, the action becomes a chaos event you can’t “check”.

Which brings me back to my point: most testers will not be laid off because AI is too smart. They will be laid off because they accepted a stupid definition of their own craft by people who don’t even work in our business!

The “checking versus testing” debate is dead. Bury it next to “best practices” and whatever has become of “shift left”. AI gets checking and testers get the future.

Layoffs in testing have only been nibbling round the edges, so take a class in RST and buckle up, the next leg down is going to be bumpy!

The post Checking Isn’t Testing. Soon It Won’t Be Employment Either first appeared on Quality Remarks.

Who’s in your “team”?

2026-05-25T12:03:42Z

This based on my personal thoughts.

Life is great when things are going well. For me that’s I’m doing well at work, family is happy and healthy and I’m able to go to the gym and run. It’s all good.

The issue comes when things start to get challenging. Maybe injuries set you back, or family members need help. Or work could have gone better.

It’s at points like this it’s handy to have people in your “team” who you can turn too.
I’m lucky I have a supportive family (wife and daughter are amazing) I also have a physio in Lara and a life coach in Laura who help me get back in the game and work through my challenges. I also have trusted colleagues at work who help me on what I need to be doing.

Some have a mentor to help show them a path and encourage learning and development and reflect back on outcomes and things to try next. This could equally be a boss or a someone outside work that can do this.

Why does this matter?
For me sometimes you need experts to help you work through and be objective about the things you need to do. They’re not connected by being family so can look at things for what they are.

So if like me you think it’s useful having a “team” have a think who could be in it.

Book Review: Software Testing Strategies

2026-05-25T00:00:00Z

My insights from "Software Testing Strategies: A Testing Guide for the 2020s" book by Matthew Heusser and Michael Larsen

Test-Retest Reliability: Is Your Rubric Consistent Run-to-Run?

2026-05-24T20:31:47Z

QA engineers spend a lot of energy hunting down flaky tests in their code pipelines, but almost no one checks whether their LLM rubric gives the same verdict twice on the...

The video of my SeleniumConf Valencia talk is now available

2026-05-24T00:00:00Z

Here is the video recording of my SeleniumConf Valencia talk "Reinventing the wheel" about our open source reporting software "Cluecumber".

Modern professional software development is more akin to sport than anything else

2026-05-23T14:36:13Z

For too long we’ve accepted one failed metaphor or another to help define developing software professionally. All models are fallible but the one I believe breaks least often is: software development is a contact team sport. One shared clear goal, smaller milestones to progress to that goal, filled with opportunities for success and failure to learn from and adapt and when on the pitch, it doesn’t matter what your ‘position’ is, you have one responsibility: make your team win.

John Cutler recently asked “What is a team?” in his post TBM 423: Why Defining Teams Is So Hard. What I took from his piece aligns a lot with my experience: organisations are messy.

We have one very strong source of inspiration for what real teams are, from sport. There are a number of different ‘teams’ in sport. There are examples of individual sports where the ‘team’ exists but is less apparent in direct competition, such as tennis or golf. Formula 1 is an individual sport but which cannot reach the extreme heights of innovation and excellence unless there’s an acceptance that success is impossible without the team around them. Then there’s the sports where the team is in competition together, in real time e.g. football and rugby.

It is my belief that ‘teams’ which have a shared goal, made up of more than 2 people, operate closer to football and rugby, than F1, golf or tennis. If a team doesn’t see themselves in this way now, making this subtle change in perspective will be transformative.

As a Manchester City fan, the announcement that Pep Guardiola is leaving is a hard one to process but it does instigate a moment to reflect. Pep’s Barcelona team is generally accepted to be the best football team created in the modern era. His strong beliefs, derived from his inspiration Johan Cruyff’s Total Football, applied to a squad of exceptional players, with one standout legend in Lionel Messi. This team dominated football matches from start to finish. I still don’t believe I’ve seen a better single performance from a football team than their 3-0 victory over Manchester United in the Champions League final.

It wasn’t until he became Manchester City manager that I started to delve deeper into his methods. Over the 10 years of his tenure as manager, I’ve watched countless press conferences, documentaries and interviews, attended more games home and away than my bank balance would like to admit and watched even more on TV. One thing about Guardiola stood out more for me than his tactics or beliefs.

The Pep Guardiola Effect

It’s easy to see why Pep’s teams do so well. Without going into too much detail, Pep’s teams have a number of very specific quality attributes: the team moves up and down the pitch with the ball; when the ball is lost the team have to get it back immediately; the individual skill level of every player is incredible to enable them to maintain the ball under intense pressure; there’s always a free player to pass to. It’s simple when written down, but only Pep has really been able to implement it consistently, season after season, at 3 major clubs. I asked myself why and I think I found it: responsibility.

The Pep Guardiola effect is evident most in the interviews and press conferences he does when City have lost or even more so, when they’ve won but not played well. He doesn’t hold his players accountable, he takes responsibility and makes the players responsible.

Accountability is often displayed in questions focussed on the past: why did this happen, why didn’t you do this?

Responsibility is best displayed in questions focussed on the future: what can I do differently, how can we be better?

Pep helps them be responsible for themselves, to be their best but more importantly, be responsible to their team, the fans and the club. Fear can drive you in short bursts, but responsibility is the internal fire that powers us. Pep is able to build incredible levels of responsibility into the players in his teams. That responsibility blurs the once static understanding of positions: goalkeepers just stop the ball, defenders stop attacks, midfielders collect the ball, wingers get the ball to the strikers, strikers score. This is now and forever more no longer true. In a Pep team, the goalkeeper has to play football as well as midfielder, strikers have to be able to retrieve the ball in an attacking position as well as any defender in their own penalty area. This responsibility, a responsibility to work for the team with a single shared goal, to win every game. This is what creates the virtuous feedback loop for excellence. That feedback loop or continuous learning is a relentless pursuit of the perfection of implementation of the beliefs. Some of the most successful players at Manchester City joined the club and were pretty terrible in their first season. But they stayed. They believed in themselves. They improved. Until Erling Haaland it’s arguable that Pep’s City had never signed a complete superstar. Every other player who turned out to be world class, became world class at City. But how can Pep build that level of responsibility into people?

The Pep Guardiola Approach

Pep thinks in systems. The pitch is a constraint and there are many others. He doesn’t accept all of those constraints as immovable, but he accepts them in the moment. That acceptance gives Pep a canvas to push those constraints to their absolute limits. There is one constraint he embraces more than any other: that his team is made of human beings. Humans have feelings, they have lives, they have limitations. We are all flawed. Pep is flawed. Pep knows this and he does one thing I don’t think I’ve seen anyone else do close to as well as him, other than Jurgen Klopp: he cares about those players as humans. Not as an extended family, there are clear boundaries of responsibility in place. But he cares about all of them as individuals. He signs players who he believes have a desire to reach the very top of their potential. He gets them there not through fear but by giving a damn about the success of each of them. This can’t be faked. Humans have excellent instincts to detect fakeness. Maybe not initially, but when it really matters. It is impossible for Pep to turn huge potential into world class, season after season, unless he is authentic. And you can see it in his behaviour.

We’ve all been told how psychological safety is important for excellence. Excellence is an emergent property of the ability to adapt following failure. Pep is able to build such a high level of trust in him from his team, that despite intense pressure, individuals feel the responsibility to take the circumstances, not the moments, the circumstances and lift themselves and each other to improve, over and over and over again.

People have written about this. Countless white papers of research, conference talks and many a book. One in particular Radical Candour.

“Care personally for your colleagues while also challenging them directly.’”
Matt Rutherford

The feeling of being cared for can only come from someone else’s action. Saying you care for someone only survives if you behave like it. Not perfectly, that’s not possible, but proportionally more often than not. It’s in the moments of highest stress when that care has to come to the fore. No one messes up on purpose. Football is a complex game, with a huge surface area for mistakes. Software development is no different. But excellence can only come from learning from those mistakes and to learn from those mistakes, the people who could scorn you, who could hold you accountable, have opportunities to demonstrate their level of care by accepting their responsibility in the failure or offering words of encouragement and reassurance.

And this is why that sustained level of care is necessary. Pep has said himself on a number of occasions that he overstepped challenging directly, during or after a game. You can watch some of the changing room exchanges online. As an outsider they’re uncomfortable to watch. But in that group, the players know it’s not personal, it’s just frustration. And because that frustration doesn’t boil over into accountability, even in this most intense of environments, players can feel safe that tomorrow, when the frustration is gone, responsibility kicks in again.

Almost everyone I’ve heard talk about this book falls into the ‘fake care’ problem. They fail to understand that the success of challenging directly is directly proportional to the amount of care the person receiving the feedback feels they have from the person providing it. These people will only be involved in creating and achieving genuine excellence, through luck.

Excellence in the world of AI

As building software becomes even more commoditised, everyone gets the lift up. Pep changed football, now every club, from junior to senior, try to emulate what Pep achieved, not because they solved that problem themselves, but because they’re standing on the shoulders of a giant. If everyone tries to play the same way you have two options for success: do it better than everyone else or be better than everyone else.

The organisations and teams of any reasonable scale will succeed in the era of AI by playing Pep’s football. Blur roles, build individual excellence, one shared problem to solve but solve it together. But that’s not enough, you need to be better. Being better is a tomorrow problem. Tomorrow can only be better if people take responsibility for it.

This isn’t about reading a book and looking for a recipe or process to follow. You’ve already failed if you do. Now as humans more than ever we have to stand out. That means looking inside ourselves and finding the humanity. That’s going to be the differentiator. You don’t have to completely overhaul an org overnight, you don’t need to do an all-hands. You start tomorrow by changing you. Take the next situation you’re perceived to be a leader for and demonstrate care, demonstrate responsibility. If it’s an incident review, look forward and find a path together. If it’s slipping schedule, look forward and find a path together. If it’s a feature that’s not living up to expectations, look forward and find a path together. Not in a slide deck or slack message but around a desk or in an online call, with the problem in front of you. Start together, end together. Drag each other over the finish line.

Data Driven API Testing in Playwright TypeScript: Part 1

2026-05-23T11:51:55Z

Learn how to implement data-driven API testing in Playwright with TypeScript using static test data and reusable tests.

Continue reading on Level Up Coding »

Data Driven API Testing in Playwright TypeScript: Part 1

2026-05-23T11:51:55Z

Learn how to implement data-driven API testing in Playwright with TypeScript using static test data and reusable tests.

Continue reading on Level Up Coding »

The Agile Circus

2026-05-22T19:50:40Z

A humorous story exposing fake Agile culture, endless meetings, toxic leadership, and workplace chaos in a modern corporate team.

Continue reading on ILLUMINATION »

The Agile Circus

2026-05-22T19:50:40Z

A humorous story exposing fake Agile culture, endless meetings, toxic leadership, and workplace chaos in a modern corporate team.

Continue reading on ILLUMINATION »

Selenium vs Playwright 2026: The Benchmark Data Every Migration Team Needs

2026-05-22T04:38:34Z

Playwright now leads Selenium by 23x in npm downloads. See real benchmark data, migration traps, and an 8-week roadmap for teams considering the switch in 2026.

The post Selenium vs Playwright 2026: The Benchmark Data Every Migration Team Needs appeared first on Software Testing & Automation.

n8n for QA Automation: Build No-Code CI Pipelines That Run Playwright Tests

2026-05-22T04:34:15Z

Learn how QA teams can build no-code CI pipelines with n8n and Playwright. Step-by-step tutorial covering triggers, JUnit parsing, Slack alerts, and Jira bug creation without writing YAML.

The post n8n for QA Automation: Build No-Code CI Pipelines That Run Playwright Tests appeared first on Software Testing & Automation.

DOM State Clarity With cy.depends Command

2026-05-22T04:00:00Z

Imagine a simple web application scenario: you click on a button. App tries to load some data. The data might load, or the backend might return an error. You write a Cypress test:

1
2
3

cy.visit('cypress/fetch-fails-sometimes.html')
cy.contains('button', 'Fetch data').click()
cy.contains('#data', 'Fetched data: "List of items"')

The test is green, but there are a couple of downsides.

test is slow Loading the data might take a while. In our case it takes almost 8 seconds, so your test must wait.

mismatch in speeds Network call might fail and the test times out waiting for the #data element. The network error itself is fast: it returns 500 status code after only half a second. But since the successful data load is slow, we have to keep waiting and waiting for #data element.

it(
  'handles occasional backend error',
  // longer command timeout to allow for slow data load
  { defaultCommandTimeout: 15_000 },
  () => {
    cy.visit('cypress/fetch-fails-sometimes.html')
    cy.contains('button', 'Fetch data').click()
    cy.contains('#data', 'Fetched data: "List of items"')
  }
)

errors are not observed The test does not report the "error" element shown. Instead the test times out looking for the "#data" element. It would be nice to write a test like this:

click on the button
wait for the "#data" element to appear and check its text
while waiting, if the "#error" element appears, fail the test with a clear message

Of course, we could rewrite the test to observe the network call - but not every action has a clear observable "single point". Instead many tests have to consider several possible outcomes at once.

DOM states

In our application, the app is simple. From the starting state (before we click the button), the app goes into "loading data" state, followed by the "successful data load" OR "failed data load" states. We can ignore the "loading data" state, since our app does not really show anything observable on the page (and we ignore the network call for now). At the end of the loading state, the app might show an element with ID "data" or an element with ID "error". Nothing else (unless things go really wrong)

In this blog post, I will show my new query command cy.depends from the cypress-if plugin that lets you consider multiple possible DOM states at once. This command can wait for all given selectors and then perform actions depending on the first matched selector.

Let's write a test and wait simultaneously for success and error states. For now, let's just print a message for each outcome.

cy.contains('button', 'Fetch data').click()
// simply report success or failure
cy.depends({
  '#data': 'Data loaded',
  '#error': 'Failed to load data',
})

When the network load succeeds, we see 2 items logged into the Command Log: the query command waits for all selectors using all matching CSS selector. Once some elements are found, cy.depends matches the elements again to the find the first individual selector and performs the actions specified in the arguments object. For example, if the value is a string, it simply logs it.

Let's say the network fails. Notice the cy.depends matched the #error element and the test finished immediately after the network load. No need to wait for #data element for 15 seconds when the app is clearly in the "error loading" state.

Handle each state

We can print log messages for each state. Or we can fail the test for some states. When the test finds the "#error" element, we want to fail the test immediately, which we can by giving an Error instance.

cy.contains('button', 'Fetch data').click()
// report success, fail on error
cy.depends({
  '#data': 'Data loaded',
  '#error': new Error('Failed to load data'),
})

Again, the test is fast and clear. If we see an element matching #error CSS selector, we want to throw the Error object. We can go beyond simply throwing an error. We could perform custom assertions on the found elements. The cy.depends command yields the matched selector and found elements to the callback function

cy.contains('button', 'Fetch data').click()
// do something depending on the matched selector
cy.depends({
  '#data': 'data loaded',
  '#error': 'load error',
}).then(({ selector, elements }) => {
  if (selector === '#data') {
    expect(elements, 'data element')
      .to.have.length(1)
      .and.to.have.text('Fetched data: "List of items"')
  } else if (selector === '#error') {
    expect(elements, 'error element')
      .to.have.length(1)
      .and.to.have.text('Fetch failed: Network response was not ok')
  }
})

By the way, if none of the cy.depends selectors match, the query command times out and the test fails.

Conditional actions

Querying multiple states helps solve conditional testing problems without nesting multiple IF/ELSE paths. Think of cy.depends as a switch statement; yes, it could be equivalent to multiple if/else statements, but having one switch usually beats multiple branches.

For example, let's say we have a page with a dialog that might be open, and we always want it closed. If we don't have a good programmatic way of controlling the dialog (I know, I know, but we are in the real world), then:

it(
  'closes dialog if open',
  () => {
    cy.visit('cypress/close-dialog.html')
    cy.depends({
      'dialog[open]': ($dialog) => {
        cy.wrap($dialog).find('button#close').click()
      },
      'dialog:hidden': 'dialog is already closed',
    })

    // check if the dialog is open
    cy.depends({
      'dialog[open]': new Error('dialog should be closed'),
      'dialog:not([open])': 'closed dialog',
    })
  },
)

Of course, you can have more than two states, just list the CSS selectors ordered from the most discriminating to the least discriminating to properly determine the correct state.

// conditional state depending on the number of items in the list: 3, 2, or 1
cy.depends({
  '#fruits li:eq(2)': 'Found 3 items',
  '#fruits li:eq(1)': 'Found 2 items',
  '#fruits li:eq(0)': 'Found 1 item',
})

Embrace and extinguish the ambiguity by handling every possible page state in your end-to-end tests.

🎁 Find the cy.depends query command in my cypress-if plugin.

Docker Compose for Test Automation: One File to Run Playwright, Selenium Grid, and API Tests in 2026

2026-05-22T03:40:59Z

Run Playwright, Selenium Grid, and API tests from one docker-compose.yml. Production-ready Docker Compose test automation setup for 2026 with health checks and CI/CD.

The post Docker Compose for Test Automation: One File to Run Playwright, Selenium Grid, and API Tests in 2026 appeared first on Software Testing & Automation.

From Manual Tester to AI Engineer: The 90-Day Roadmap for Indian QA Professionals in 2026

2026-05-22T03:40:07Z

The exact 90-day roadmap for Indian QA professionals to transition from manual testing to AI engineering. Includes real salary data from PayScale, the tools to learn, and a portfolio project that gets you hired.

The post From Manual Tester to AI Engineer: The 90-Day Roadmap for Indian QA Professionals in 2026 appeared first on Software Testing & Automation.

The QA Career Pivot Guide 2026: From Testing Features to Owning Quality Intelligence

2026-05-21T22:00:00Z

QA is evolving from testing features to owning quality intelligence. The career pivot guide for 2026: new skills, new role titles, and how to make the transition.

The post The QA Career Pivot Guide 2026: From Testing Features to Owning Quality Intelligence appeared first on Software Testing & Automation.

Measuring for AI success and quality improvement

2026-05-21T15:10:40Z

I’ve worked primarily in a tester and quality engineering role for a few decades now. “Metrics” have always been the magic beans of improving quality. People ask me “What metrics should we use?”, as if they will reach their quality goals by measuring the right things. They want something easy, like code coverage and bug counts.

Small experiments

In my experience, it works best the other way. My teams worked to achieve one improvement goal at a time. Having agreed on the next goal, we tried experiments to move towards it step-by-step. Then we’d figure out some way to measure whether each experiment was working. For example, one team I was on wanted to shorten our cycle time, from when we started working on a user story to when we got it into production.

We decided to try example mapping to get better shared understanding of each story in the planning stage. This practice should help us cut down on re-work when acceptance testing found issues. What metrics would track the success of our experiment? Besides cycle time, we could measure the percentage of stories rejected in acceptance testing. When it was reduced by half in two iterations, we kept up the example mapping.

Measuring progress in your AI journey

Today, we have AI tools to help us achieve our quality goals. The principles of building quality in throughout the whole development life cycle remain the same. AI assistance gives us new software tools that help us work smarter. Research shows that AI amplifies the strengths of high-performing organizations, and the problems of dysfunctional ones.

In my view, approaching AI adoption step-by-step, iterating through small improvement experiments, is a good way to enjoy the benefits and avoid the pitfalls. Measuring appropriately to track each experiment is key.

Enter the DORA AI Capabilities Model. I encourage you to go download the DORA AI Capabilities Model Report. Open it in your PDF viewer of choice. Now, do a “find” on the word “measure”. Found on 23 pages! Wow! Go back to the start of the document. Check out the seven key capabilities on page 4.

The Seven Key Capabilities

Now read page 5, where you will learn that the report gives you details on how to get started on each of the seven capabilities. Keep reading. As you explore the detailed section for each capability, and why that capability matters for AI, you’ll learn how to improve that capability day-by-day. And, hooray, the report explains specific ways to measure your progress. You’ll also find the common obstacles to successfully adopting capability.

There’s a lot to digest here. You’ve got so much information here for your team to discuss. But where do you start? We’re only human, we can’t grow seven capabilities at once.

No worries, the report provides detailed sections on techniques to help your team prioritize which capability to work on first. This takes an investment of time, for sure. Then your team will be ready to start iterating on those small experiments to improve the highest-priority capability. The best part is you will now know some good ways to measure your progress towards improving that capability.

Making the investment

I know this sounds overwhelming. Most people do look pretty overwhelmed by the challenges that AI brings along with its benefits. We humans tend to not make decisions based on scientific research. Yet the potential rewards of successful AI adoption, and the potential disasters of unsuccessful AI adoption, should motivate this investment. So now, go download DORA’s ROI of AI-Assisted Software Development report. It shows the initial investment needed to reap the eventual exponential growth that AI assistance enables.

Be patient, take baby steps, do one or two small experiments at a time. Make them short enough so that you know in a week or two if the experiment is working – based on those appropriate metrics you chose. At some point, your team will get past “J-curve of AI value realization” and start saving time. Rather, making time for humans to do what humans do best, while the robots do our grunt work. As the ROI of AI-Assisted Software Development report concludes,

Invest in engineering excellence today to ensure AI acts as an amplifier of value rather than a catalyst for downstream chaos.

You’ve got the recipe books here, thanks to DORA. Invest the time now, and enjoy tracking your quality – and your enjoyment – improving faster and faster over time.

The post Measuring for AI success and quality improvement appeared first on Holistic Testing with Lisa Crispin.