blogs

« Silly Questions » Are Almost Never Silly

2026-04-23T06:31:19Z

Blog by : Imen Naguez

How AI unlocks the questions we’re too afraid to ask anywhere else

You know the moment. Someone explains something, everyone nods, and somewhere in your mind a phrase is trying to get out: “What exactly is that?”, “Could you explain again?”, “I don’t see the connection.” You keep it to yourself. Not because you’re less intelligent than the others — but because asking that question carries a social costyou’re not ready to pay.

AI doesn’t make people smarter by magic. What it does, however, is reduce that cost. And that shift, seemingly minor, can profoundly transform the way we learn, think, and grow.

Here are 12 concrete situations. For each: what happens without AI, what becomes possible with it, and what the research says about it.

─────────────────────────────────

The real problem isn’t the question — it’s other people’s reactions

In a classroom, a meeting, a training session, or even among colleagues, some questions seem tiny — but they expose a lot. Asking “what exactly is that?” can feel like admitting you weren’t following. Asking again can feel like appearing slow. Saying “I don’t understand the link” can feel like displaying a flaw in your thinking.

The block is almost never intellectual to begin with. It’s relational.

” Teams learn more when members feel they can take an interpersonal risk without being immediately disqualified or ridiculed. ” — Amy Edmondson, Psychological Safety and Learning Behavior in Work Teams (1999)

In other words: when the cost of judgment drops, asking a question becomes possible. And when you can finally ask, real learning can begin.

─────────────────────────────────

12 questions we don’t dare ask — and what AI changes

For each situation: the reality without AI, what becomes possible with it, and what the research says.

1. “What exactly is… ?”

Without AI

The question comes too late, gets sidestepped, or people just guess. Fear of judgment takes over — looking incompetent, appearing like you weren’t paying attention. Result: an entire line of reasoning built on a shaky foundation.

With AI

You ask directly for a definition, a rephrasing, a simpler explanation. No sighs, no looks. Result: the concept is clarified from the start, you move forward on solid ground.

What the research says

When the risk of judgment decreases, asking for clarification becomes natural. Amy Edmondson showed that psychological safety — the feeling that you can speak up without being ridiculed — is one of the strongest predictors of real learning. (Edmondson, 1999)

2. “Can you explain that again?”

Without AI

You pretend to have understood so as not to bother anyone. You take confused notes, retain little, and memorize poorly. Embarrassment wins over learning.

With AI

You ask as many times as you need, in different ways if necessary. No sighs, no impatience. You consolidate the foundations and move forward on stable ground.

What the research says

Asking for help often carries a high social cost. Ryan and Shin showed that students who dare to ask for help at the right moment — what they call adaptive help-seeking — achieve significantly better outcomes than those who avoid the question out of fear. (Ryan & Shin, 2011)

3. “I don’t see the connection between A and B”

Without AI

You don’t dare admit you can’t follow the logic between two ideas. You retain isolated fragments without ever grasping the mechanism linking them.

With AI

You ask for a step-by-step walkthrough, a logical diagram, an intermediate example. The lack of understanding becomes a legitimate starting point, not a source of shame.

What the research says

Precisely naming what is missing does not block learning — on the contrary, it often activates a clarification process. Chi et al. showed that learners who self-explain — who verbalize missing links — build far deeper understanding than those who stay passive. (Chi et al., 1989)

4. “Give me a concrete example”

Without AI

You don’t dare look like a beginner. You understand in theory but retain poorly and struggle to apply it. The abstract stays abstract.

With AI

You ask for one or several examples, even a counter-example. You see how the concept works in real life. You retain better and can transfer it to other situations.

What the research says

Worked examples — pre-solved, guided illustrations — are especially effective for novices. Atkinson et al. demonstrated that well-structured guidance with resolved examples significantly accelerates learning, especially at the start of a learning journey. (Atkinson et al., 2000)

5. “Check my reasoning !”

Without AI

In public, you protect your image. You avoid exposing a still-fragile line of thinking. You spot your mistakes too late and correct them with difficulty.

With AI

You submit your reasoning and explicitly ask: “Where does it break down? Which assumption is weak?” You detect flaws earlier, correct faster, and raise your level of rigor.

What the research says

Hattie and Timperley showed that the most useful feedback is not just validation — it is precise information that helps understand where you are, where you’re going, and what needs adjusting. The less the ego is exposed, the more this feedback is genuinely used. (Hattie & Timperley, 2007)

6. “I start from scratch ?”

Without AI

You hide your real level and overcompensate. You try to appear already autonomous. You skip essential steps, exhaust yourself, and progress less than you could.

With AI

You embrace starting from the basics. You ask for a fundamental, progressive, structured explanation. You make real progress and build solid foundations.

What the research says

Acknowledging your gaps takes courage. But when the pressure of others’ judgment decreases, this honesty becomes possible — and it is often the beginning of more lasting learning. (Edmondson, 1999)

7. “What’s the first step?”

Without AI

You want to appear autonomous. You avoid asking where to begin. You scatter your efforts, start too broad, and lose energy and efficiency.

With AI

You ask for a minimal plan, a clear first action. The mental load drops immediately. You move forward step by step and regain a sense of control.

What the research says

Knowing how to ask for a clear entry point helps reduce scattered effort. Research on cognitive load shows that breaking down a complex task into simple steps significantly improves both perseverance and outcomes. (Sweller, 1988)

8. “Explain it again simply”

Without AI

You don’t dare ask for a very simple version, for fear of being seen as “not at the right level.” You stay confused, understand partially, and don’t dare admit the explanation is over your head.

With AI

You ask for a simple explanation first, then gradually increase complexity. You build understanding step by step and can later absorb a more technical level.

What the research says

Implicit norms about “level” sometimes censor the request for simplicity. Yet learning through gradation is often more effective than diving straight into complexity. Shute stresses this point: progressive formative feedback is far more useful than an expert explanation given all at once. (Shute, 2008)

9. “Challenge my idea”

Without AI

You dread contradiction because it feels like a personal attack. You avoid objections, stay in your initial logic, and miss your blind spots.

With AI

You deliberately ask for objections, possible criticisms, and ways to test them. You spot your biases, strengthen your idea if it holds — or adjust it before it crumbles in the real world.

What the research says

Actively seeking objections improves the quality of reasoning. This approach reduces blind spots, limits certain biases, and makes decisions more robust. It is critical thinking in action. (Hattie & Timperley, 2007)

10. “Between these 2 options, which should I choose?”

Without AI

You hesitate without daring to clearly articulate your doubt. You stay stuck, hesitate for a long time, and sometimes decide without explicit criteria.

With AI

You ask for a structured comparison: criteria, advantages, risks, trade-offs, use cases. You decide better, with a more reasoned and coherent choice.

What the research says

Asking for an opinion is not always a sign of weakness — it can signal discernment. The quality of a decision increases when it rests on explicit criteria rather than unverified intuition. (Kahneman, 2011)

11. “Help me without judgment”

Without AI

You don’t voice this need because you think you should handle your confusion alone. You push through, learn less well, and risk closing off instead of staying open to learning.

With AI

You clearly state that you want calm, non-judgmental, progressive guidance. You become a learner again. Your real questions surface. A constructive dynamic takes hold.

What the research says

Even in front of a machine, we sometimes retain social reactions. But since the relational cost is lower, it becomes easier to ask for a reassuring framework — and within that framework, authentic learning becomes possible again. (Edmondson, 1999)

12. “What if you are wrong?”

Without AI

You verify through your usual sources, your experience, or the opinion of others — sometimes in a less structured or slower way.

With AI

You can ask for sources, assumptions, limitations, points of uncertainty, and a verification method. You use AI as an exploration tool, not an absolute authority.

What the research says

A lower social cost of discussion must not suppress critical thinking. On the contrary: the easier the exchange, the more important it becomes to verify, compare, and validate before deciding. A fluent answer is not necessarily a correct one. (Shute, 2008)

─────────────────────────────────

In truth, these questions take courage

When someone asks “what exactly is that?” or “I’m starting from scratch,” they’re not necessarily showing a lack of ability. They’re often showing a genuine willingness to actually understand. What blocks most people, most of the time, isn’t a lack of intelligence. It’s the fear of exposure.

” Avoiding help-seeking out of fear of others’ judgment deprives the learner of essential resources — while adaptive help-seeking is directly linked to better outcomes. ” — Ryan & Shin, Help-seeking tendencies during early adolescence (2011)

AI, used thoughtfully, replaces neither human pedagogy nor intellectual rigor. But it offers something valuable: a space where you can start small, try again often, ask simply, ask clearly, ask for an objection, ask for verification. Without putting your image on the line.

And sometimes, that’s exactly what it takes for learning to stop being a performance — and become real understanding again.

─────────────────────────────────

References

Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181–214.

Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145–182.

Edmondson, A. (1999). Psychological Safety and Learning Behavior in Work Teams. Administrative Science Quarterly, 44(2), 350–383.

Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112.

Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.

Ryan, A. M., & Shin, H. (2011). Help-seeking tendencies during early adolescence. Learning and Instruction, 21(2), 247–256.

Shute, V. J. (2008). Focus on Formative Feedback. Review of Educational Research, 78(1), 153–189.

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.

Hey Siri, is AI Fatigue a Thing?

2026-04-23T06:22:00Z

The other day I scrolled through my LinkedIn feed and noticed something. AI, AI, AI. Three out of four posts are about AI in one way or the other. New tools, new prompts, new ways to be more productive. And somewhere between the tenth and twentieth post, a word popped into my head: AI fatigue.

So I asked Bing: “Is AI fatigue already a thing?” The answer came back in capital letters: YES. Together with, of course, an AI-generated summary. The irony wasn’t lost on me.

Turns out, since 2025 there are actual reports and studies about people suffering from AI fatigue. AI fatigue, as it turns out, is not just one thing. It’s a whole family of exhaustions. (1) There’s the constant pressure to adopt yet another tool, to keep up with updates and new features. (2) There’s the cognitive drain of reviewing AI output instead of creating, turning us from makers into judges. (3) There’s the endless flood of AI-generated “slop”, filling our feeds with hollow noise and fake news.(4) And there’s the sheer hype fatigue, the relentless drumbeat of AI everywhere you look. Four flavors of exhaustion, and most of us are tasting multiple of them at once.
Even though this post is triggered by number (4), this post focuses on (1) and (2).

What triggered this reflection was a report from Anthropic titled “Estimating AI productivity gains.” The numbers are impressive and frightening. Current AI models could increase US labor productivity growth by 1.8% annually over the next decade. That would double the growth rate we’ve seen since 2019. The median task shows 80% time savings. Sounds like progress, right?

But here’s the question nobody seems to be asking. Progress for whom?

From news articles and podcasts, I’m hearing something different. Management has discovered a new instrument. And it’s not being used to make work easier or more meaningful. It’s being used to squeeze more out of people. Faster. More. More efficient. At any cost. There’s this constant pressure now, this unspoken threat hanging over every desk: AI could replace you. So you better speed up. You better adopt. You better produce.

It feels less like a tool and more like a whip, like a threat.

And if you look at it from a purely capitalistic perspective, it makes sense. Companies are not in the business of keeping humans healthy and happy. They’re in the business of generating value. And right now, AI promises to generate more value with less human involvement. Which means the humans can produce more and more output in the same time. They just need to focus on the parts that AI can’t solve yet, and use AI as efficient as possible for the rest.

And the human cost? AI fatigue is real. People are being forced to use tools they don’t understand, don’t trust, or simply don’t like. Some companies even reward the employees with the highest token usage. The pressure mounts. The exhaustion grows. We’re optimizing productivity at the expense of wellbeing. We’re measuring output while ignoring the people producing it. And with tokenmaxxing you only guarantee two things. More income for AI providers and a warmer planet thanks to unnecessary resource use.

Proponents say that AI handles the stupid and easy jobs, so that humans can focus on the tough parts. But constantly working on the tough parts is draining energy. And work models don’t account for that. Managers now expect you to work 40+ hours a week on the tough parts only.
In a good world the work days would become shorter, because people with the help of AI can now manage 40 hours of work in let’s say 25 hours. So why not cut the work week down to 30 hours or less. People sometimes need a few stupid tasks to regenerate mentally. Or they need other means. Like longer breaks.

Professional athletes don’t train all day the hard moves and tactics. They also train the simple things, the basics. Because the simple things are the foundation for the hard things. By outsourcing the foundation you will lose a vital part of the package.
It’s like when an F1 drivers uses his F1 car to get to the train station or running errands. Driving the whole time at maximum speed.

I don’t have a neat answer here. But I do have a question: Where does this end? Is “more productivity” really the goal we should be chasing? Or do we need a different conversation entirely, one about what good work actually means, about sustainable pace, about treating people as humans rather than resources to be optimized?

I know that companies need to make money. But for the long run, maybe it’s time to stop asking “How can AI make us more productive?” and start asking “What kind of work do we actually want to do?” and “How can we balance work and life in these new times?”

Upgrade Cypress To TypeScript v6

2026-04-23T04:00:00Z

Finally, it has happened. Cypress v15.14 supports TypeScript v6. This is a big deal - TypeScript v6 is the last step before TypeScript v7 which is implemented in Go and is much faster. By the way, you can try TypeScript v7 beta today. So what do you need to switch your projects to use TS v6 with Cypress? It is pretty straightforward:

check out the TypeScript v6 changes, for example this is a good summary
switch module resolution from "node" to "bundler" and let Cypress use WebPack to bundle the transpiled code

  {
    "compilerOptions": {
-     "moduleResolution": "node",
+     "moduleResolution": "bundler"
    }
  }

remove the "target" transpile option, since it is assumed to be the current ES standard (es2025)
remove the allowSyntheticDefaultImports: true option, since it defaults to it
remove the baseUrl option UNLESS you are using TypeScript aliases in your specs.

If you are using path aliases in your specs like this:

tsconfig.json

{
  "compilerOptions": {
    "paths": {
      "@support/*": ["./cypress/support/*"],
      "@fixtures/*": ["./cypress/fixtures/*"]
    }
  }
}

cypress/e2e/cart/add-to-cart.cy.ts

1 2	import { LoginPage } from '@support/pages/login.page' import { InventoryPage } from '@support/pages/inventory.page'

Then without baseUrl the Webpack gets lost trying to find the import 🙁

So for now, put the baseUrl back into your tsconfig.json and just point at the current tsconfig.json folder. Add "ignoreDeprecations": "6.0" option to avoid everything screaming at you

tsconfig.json

{
  "compilerOptions": {
    "baseUrl": "./",
    "ignoreDeprecations": "6.0",
    "paths": {
      "@support/*": ["./cypress/support/*"],
      "@fixtures/*": ["./cypress/fixtures/*"]
    }
  }
}

All relative paths in the paths map are relative to the tsconfig.json file.

Happy TypeScripting!

Flaky Tests Are a Trust Problem: How to Diagnose, Quarantine, and Permanently Fix Unreliable Tests

2026-04-22T22:00:00Z

A failing test is useful. A flaky test is dangerous. This guide covers the trust erosion cycle, root cause diagnosis, quarantine strategies, and how one team cut flakiness from 30% to under 2%.

The post Flaky Tests Are a Trust Problem: How to Diagnose, Quarantine, and Permanently Fix Unreliable Tests appeared first on Software Testing & Automation.

The History of Contract Testing with Pact.io

2026-04-22T17:46:02Z

Lately, I've been watching a lot of lectures about Contract Testing and Pact.io, trying to prepare for an upcoming job interview. When diving into a new toolset I can never simply jump into the code. I need to know: Why was this toolset created? What problem did it solve? How was this tool created? How did this toolset evolve?

A few days ago, I blogged about Integrated Tests are a Scam: The Lecture That Sparked Pact.io talking about J. B. Rainsberger's 2013 lecture. Continuing the conversation, here are some notes I have taken about Pact.

The History of Pact.io

According to Pact Docs / History:

"Pact was originally written by a development team (including Ron Holshausen from DiUS) at realestate.com.au in 2013. They were trying to solve the problem of how to do integration testing for their new Ruby microservices architecture, and they threw together a codebase that was to become the first Pact implementation. As the team was writing Ruby microservices, the serialization of the "matching logic" (e.g. regular expressions) was done using Ruby specific code. The team was under the pump to finish their actual customer deliverable, so they got their Pact MVP working, and left it at that.

"A few months later Beth Skurrie from DiUS joined the one of the teams that was working with the Pact authors' team. She had recently seen a talk by J.B.Rainsberger entitled 'Integration tests are a scam', which promoted the concept of 'collaboration' and 'contract' tests, so she was immediately interested when she was introduced to Pact. After trying (as most people do) to convince the authors that the provider should be the contract writer, she was soon convinced by Brent Snook, one of the original authors, of the value of consumer driven contracts. At this stage, she realised that what was missing in the current implementation was the ability to run the same request under different conditions, and "provider states" were born.

"[...] Pact spread around the codebases in the wider program of work at realestate.com.au, until it hit its first Java microservice. realestate.com.au had many JVM projects, so a group of DiUS consultants (including Ron Holshausen again) started the pact-jvm project on a hack day. It was at this stage that the authors realized that the Rubyisms in the format were going to have to be replaced by a non-language specific format, and the idea of the v2 pact-specification arose (though it would take a while before it became reality)".

SmartBear Acquires Pact.io

In April 2022, DiUS sold PactFlow to SmartBear, the company behind Swagger, ReadyAPI, and other API toolsets. SmartBear has then been the steward for Pact, providing a lot of information about the toolset:

Docs.Pact.io has a basic Introduction https://docs.pact.io/

SmartBear University at https://smartbear.com/academy/ offers simple courses in Introduction to Contract Testing https://smartbear.com/academy/pactflow/introduction-to-contract-testing

Swagger Contract Testing University: https://support.smartbear.com/swagger/contract-testing/docs/en/swagger-contract-testing-university/overview.html offers videos and workshops on introductions to Pact, CI / CD, and Creating a Pact Plugin, along with advanced workshops.

But What Is Contract Testing, according to SmartBear Academy?

Let's say you have a server that provides data, such as a list of all the users on the system, sharing it through an API to a mobile app. The mobile app would be the Consumer, and the server would be the Provider.

If you wanted to test that things were working correctly, according to SmartBear's course, "Introduction to Pact", you can test things in isolation, breaking the test into two parts:

The Consumer talks to a mock provider, which simulates sending the data back to the consumer.
Provider talks to a simulated consumer which simulates requesting the data.

Example: You don't trigger a smoke alarm by setting your house on fire, and seeing if it responds. You simulate a fire by pressing the button to send the signal, and you test that the smoke alarm is triggered.

The Consumer can talk to a mock provider and the Provider can talk to a simulated consumer.

Contract testing ensures that a provider is compatible with the consumer expectations of it. These needs are captured in a Contract.

Example: An http system would check that the provider accepts the expected requests, and that it returns the expected responses. The contract is verified against the Provider. This is Contract Testing.

Consumer Driven Contract Testing

When using consumer driven contract testing, the consumers expectations are serialized to a contract file during the execution of its own automated tests, and then verified against a provider during the providers automated tests. The contract is generated in such a way that ties the content to the consumer code, either by use of a "test double" that records requests and expected responses and writes them to a contract. An HTTP contract would consist of request / response pairs.

A Consumer would send a response, such as GET /orders/1234 an HTTP request, and the mock would give an http response back, such as id: 1234, and list items. This is recorded, and matched with something such as PactFlow,

If the actual responses match the expected responses, the contract has been successfully verified. Any mismatches show an incompatibility between Consumer and Provider.

Provider Verification

Provider Verification Step: runs locally where the requests in the Contract file are played back against a REAL instance of the provider, to make sure it captures all expectations in the consumer contract. The provider team stubs out all the dependencies keeping the focus on the consumers expectations. When you include these tests in the providers own build, running on the dev build, it provides fast feedback and helps ensure that breaking changes are not committed.

The Pact Broker

A Pact Broker:

Open source application sharing consumer driven contacts.
Can be use by any contract that is serialized into JSON, but is specifically for Pact.
Comes with a Pact command line interface that is used to set up the contract testing workflow.
See Sharing the Pact With Pact Broker (SmartBear Docs)

Contracts and verification results are sent to the Pact broker from the CI Builds on the consumer and provider. Consumer CI build generates a Pact during the run of its isolated tests, and publishes the generated Pact Broker. The Provider CI retrieves the Pact, performs the verification, publishes the results locally, back to the Broker. Consumer and Provider CI Builds check with the broker before deploying, to ensure the application version they are about to deploy are compatible of the versions already in the environment.

The Pact Broker: No integration tests needed. Services can be deployed independently. You can share contracts and verification results between consumer and provider projects. It can tell you which versions of the application can be deployed safely together. It versions your contracts, and ensures backward compatibility, between multiple consumer and provider versions. It also provides API documentation of your application that is guaranteed to be up to date. It also shows you how your services really interact. It also allows you to visualize your relationship between your services.

The Pact Broker can allow you to self host, or use PactFlow a SaaS package.

CI / CD

Contract Testing can be used in CI / CD platforms, such as Jenkins. Consumer driven contract testing: Consumer pipeline, you run tests from the consumer's codebase, configure a test to publish the contract to PactFlow. Then you configure a step in the provider CI / CD pipeline that takes the PactFlow that verifies the contract, publishes the results to Pactflow. Optionally, you have the step, "can-i-deploy". Best way to use these steps is the Pact Broker Client command line interface. It allows you to create, update, and delete and query resources in PactFlow. It is available as a Docker image and a standalone executable.

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles

What happens when you pair Playwright with something other than TypeScript?

2026-04-22T16:28:26Z

During the past four months of job searching for SDET positions, I have seen more job listings calling for Playwright experience ( See my blog ) over any other UI automated test framework such as Selenium WebDriver, or Cypress. Most of the time, I see TypeScript paired with Playwright ... But every now and then, I see companies pair Playwright with C# or Java. Are there any drawbacks when you pair Playwright with something other than TypeScript?

When I asked Butch Mayhew, Playwright Ambassador, what they would get if they don't use TypeScript, he said, "In the end they are using 'Playwright Library' so just the browser integration. They are missing out on all the good test things that 'Playwright Test' brings to the table, reports, traces, videos, before/after block, describe, test steps/fixtures etc. [...] you lose all the great out of the box features. You have to bring your own test runner in Java".

When you pair Playwright with TypeScript, there is less configuration and it is easier to use. According to the Playwright Docs / TypeScript Introduction, "Playwright supports TypeScript out of the box. You just write tests in TypeScript, and Playwright will read them, transform to JavaScript and run".

As far as setting up Test Runners, you can still use Python with PyTest, Java with JUnit or TestNG, and C# with .NET's MSTest, NUnit, xUnit. The problem is that you are limiting yourself to the core library, and not using the platform's native runners so you don't get the built in features to automate browsers, mock APIs, make assertions and add HTML reports.

From what I have researched, this means...

Feature	Playwright + TypeScript	Playwright + Java
Test Runner	Uses the native Playwright Test runner (optimized for speed/parallelism).	Relies on third-party frameworks like JUnit or TestNG.
Boilerplate	Low; native `async/await` and built-in fixtures make code concise.	Higher; more verbose syntax and manual object instantiation.
New Features	First to receive updates; it is the framework's "home" language.	Feature parity follows shortly after the TS release.
Tooling	Deep integration with VS Code (official extension, UI mode).	Standard IDE support (IntelliJ/Eclipse) via generic plugins.
Debugging	Trace Viewer and time-travel debugging out of the box.	Requires more manual configuration to generate and view traces.
Reporting	HTML reports included by default.	Requires external libraries (like Allure) or runner-specific reporters.
Parallelism	Built-in and easy to configure via `playwright.config.ts`.	Managed via the test runner (e.g., Maven Surefire or TestNG).
Test Agents	Includes built-in Planner, Generator, and Healer test agents.	Manual execution only. AI agents are not currently available for Java.
Self-Healing	Native AI-driven self-healing selectors are supposed to fix broken tests in real-time.	No native self-healing; requires manual locator updates.
AI Integration	Direct support for Model Context Protocol (MCP) for deep AI agency.	Basic integration via Copilot; lacks native protocol hooks.
Authoring	Natural language test generation via plain-English notes.	Requires manual coding of every step using Java syntax.
Debug Insights	AI-driven triage assistants summarize failures in plain language.	Traditional logs and stack traces; requires manual analysis.

What is the Model Context Protocol (MCP)? from ModelContextProtocol.io

Benefits for using Playwright + TypeScript:

Native Test Runner: The official Playwright Supported Languages guide notes that while core automation is shared across languages, the "best experience" comes from the recommended native runners. The Playwright Test runner is unique to the TypeScript/Node.js ecosystem and provides built-in parallelization and fixtures.
Java Integration: The Playwright Java Documentation confirms that for Java, users must "hook up Playwright to your favorite Java test runner" like JUnit or TestNG.
Boilerplate & Syntax: Articles on LinkedIn and Medium highlight that TypeScript uses native async/await for concise code, whereas Java's syntax is more verbose for similar operations.
New Feature Adoption: Community consensus on platforms like Reddit reflects that because Playwright is written in TypeScript, new features and tutorials almost always arrive there first.

AI Features:

Model Context Protocol (MCP): The Official Playwright MCP Documentation specifies that the MCP server provides browser automation capabilities through accessibility snapshots for LLMs, primarily within the Node.js/TypeScript stack.
AI-Assisted Debugging: Microsoft's Developer Blog showcases the "Fix with AI" button and AI-assisted test fixes as part of the Playwright for VS Code extension, which is tailored for the TypeScript ecosystem.
Self-Healing & Agents: Playwright + AI has supposed advanced self-healing capabilities. Tools like the "execute automation Playwright MCP server" are being used to build agent-driven frameworks without manual coding, and are part of the Node.js/TypeScript environment.

Playwright works across languages. Whether it works well depends on what the team needs from it.

For teams already committed to Java, the library layer covers the core use case. For teams starting fresh or looking to leverage Playwright's full feature set, including its reporting, AI tooling, and agent capabilities, TypeScript is where the platform is built and where new features arrive first.

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles

STEC Deepdive: Module 3.4.0 - How to deliver bad news

2026-04-22T16:00:00Z

Continuing my journey through the MoT Software Testing Essentials Certificate, there is a section on feedback, both giving and receiving, that caught my attention.

Feedback is a gift in nearly every circumstance. One of the worst things that could happen in life is never receiving feedback.

For instance, I got braces for my teeth last October. I’ve instructed every member of my family to tell me if I have something in my teeth. Eating is hard. Worse is not knowing there’s something stuck in there and learning about it later. My teeth are a work in progress.

So are we.

As QA professionals, we need to be able to give and receive clear and actionable feedback. When we signed the contract, we were given the role and the right to deliver feedback. Sometimes, that feedback won’t be good. We’re the ones pointing out who has what stuck in their “braces”.

How you deliver feedback to your teammates can make all the difference.

Know your audience

Cross-team discussions

These days, I post in the organization’s public project-specific channels to provide updates on the feature being tested. Knowing that my posts will reach different types of team members across departments, I provide a concise summary on the status of what’s been tested, what remains, a link to the report where everyone can view the test cases, and a list of defects found. Those interested can view the resources at any point. I do use emojis because these help (in my opinion) make the post more readable and relatable. Though it may depend on your work culture. If emojis are frowned on where you work, refrain from using them in more public settings.

Be ready and willing to answer questions about reports. And, at all possible, avoid DMs. If a question is being asked about a project, take it to the project channel. Avoid the silos at all costs.

What if the feedback I need to give is really bad?

Be honest. Be brief. If you made a misstep or missed something during testing, own it. I’ve apologized in public channels for forgetting something in my test plan. Then I describe what I will change in the test plans moving forward so that it’s not forgotten. Own your actions.

If something is clearly broken, regardless of the environment, take it to the appropriate public channel and alert others. The goal is not to cause panic. The intent is to provide clear information about what is wrong and to alert the right people so that it can be fixed as soon as possible. Avoid using emotional words or blame. There is a difference between:

“Hey, uh, y’all, Jack’s pr broke the login page…”

and

“I’m unable to access the login page. Can anyone verify? If so, what do we need to do to get it fixed?”

Person-to-person

Sometimes, I’ll need to offer feedback to a developer on their PR. Many developers want to know if their code has issues so that when released, the feedback is… crickets. A quiet release is a good release. Even then, when testing a PR, how you say it is as critical as what you say.

Ask how the developer likes to receive feedback on a PR. There could be an engineering practice where all conversations regarding a code change need to be in the ticket or on the PR request. Follow whatever is normal for your organization. If there aren’t any set practices, ask them if they’d like feedback in a project channel, in the ticket, or in a DM. Being able to have open and honest conversations about the code is the goal. Wherever that is achieved best is where it should happen.
Be clear and provide action steps. Again, DMs are fine, but it’s tempting to speak vaguely about an issue. Write down exactly what you are seeing, steps to reproduce, and provide videos or screenshots, as you do when writing tickets. Then, if you need to write an actual ticket for what you’re seeing, all you have to do is copy and paste the information you already wrote down. Easy peasy.
Ask the next question. Yeah, it probably does work on their machine. Be the one who asks why. What is the difference between your environment and mine? Earlier this week, I was seeing console errors while visiting a specific page. A dev asked if I’d pulled down the latest changes. I hadn’t. I pulled them down, the issue was fixed, and the conversation was resolved. They didn’t treat me like I was silly, and I thanked him for giving me more information. Keep it neutral. And, when you need to, keep asking the next question to figure out what is actually going on.
Treat people as they want to be treated. Some people are fine with making the occasional joke about their code changes. Some aren’t. Read the room. I am a light-hearted person. And, I’ve learned that, in hard moments, my light-heartedness can be read very wrong. Depending on your personality, hold back the jokes if you are discussing hard topics. Then, when the conversation turns light again, feel free to be your goofy self.

Find the next step

After having hard conversations, you may need to take a break from the situation. Perhaps everyone does. Hopefully, you’ve been able to have the hard conversations in a non-blaming, actionable way. Or, maybe it went so badly. Either way, take a minute, refresh yourself, and talk to the person you had the discussion with. Find a neutral topic if you need to. Saying thank you is a great way to move on. You can say things like:

“Thank you for talking that through with me. I was so confused.”

“Thanks for being so quick about fixing those issues. That was awesome.”

“Thanks for being willing to take the conversation to a public channel. I didn’t realize others were unsure about what to do next.”

One good hard conversation down. Only a million more to go.

Communication is hard. Being responsible for bringing bad news can feel like a lot. In between those times, be sure to create lots of positive experiences for your teammates. That way, when you have bad news to bring, you aren’t being the Debby Downer. You're the friend who doesn’t want their friend to walk around with something in their teeth.

Till next time…

All my posts are free to read and clicking subscribe will bring each post to your inbox. If my work brings you joy, and you’d like to support it, you can become a subscriber by clicking the button above. If a subscription is not your thing, you can support my caffeine addiction writing by clicking the button below! Thanks!

Buy me a ☕️

Written with Happiness is a butterfly playing in the background.

Guest Blog: You don’t want Quality Engineers

2026-04-22T14:06:59Z

You don’t want Quality Engineers

Foreword

This is a guest blog by Luke Lattimer-Rogers

Luke is a quality advocate, a fantasy book author and a kick-ass dungeons and dragons player! He and I have discussed the topics of the testing community and what quality engineering means a number of times, our conversations formed the basis of my keynote talk at PeersCon 2026.

Going full circle, Luke watched my talk “You’re not ready for QE”, read my blog and even watched the Quality Talks podcast that I joined and wrote his own companion piece.

I’m sharing his blog post, which you can also find under his LinkedIn, for it to be a long lived thought piece from him!

“In order to understand how to show value as quality professionals we need to…” Yes!

“Understand what it is that engineering teams want and work to support them at that.” Oh good point but…

Callum and I are friends, in geekery, life and professionally. We’ve worked in similar spaces, had similar conversations, and arrived at similar places on what good quality work actually looks like. I was even lucky enough to go for a walk and hear some of his thoughts before his keynote at PeersCon 2026.

I agreed. And I didn’t. So I wrote the first take on this ages ago.

This is my “yes, but…”

Where I think he’s wrong, and why ultimately I agree. But like he said in his talk, “Drama!”

Why Framing over the Behaviour

Callum walks us through why the testing community is not ready for quality engineering because we fail to serve engineering teams. The well is poisoned by bad public messaging, and we’re accelerating it ourselves.

I agree on the evidence. I disagree on the cause.

For me the most damaging well-poisoning I’ve experienced didn’t come from other QAs posting bad content. It came from managers and leaders who’d already decided what a tester was before I arrived; and nothing I could do would shifted that mindset.

In one of my early consultancy roles I was put in a team with a very specific role that the client had defined. Automate the tests to help a migration. I found other issues, and raised a performance risk early. I flagged it to the manager client side and was put back in my lane. Three months later performance becomes an issue and I got a call from my account manager asking why I hadn’t raised it sooner. Guess who had been taking the meeting notes.

The reason this happened? I was a consultant. The manager was permanent staff. When it came to “whose word do we trust,” the answer was predetermined by my employment status. I did what Callum warns, I got put in a box and I stayed there. Yes I failed to communicate effectively. But there’s also an accountability and leadership failure behind it all.

We all see reposted memes poisoning the well, but it’s not just the Quality community.

What Quality Professionals Actually Do

Callum’s list of what engineering teams want from us is good. Builders, not breakers. Generalist skills. Trusted advisors. Systems thinking. Pragmatism. I agree with all of it.

But he has another list, the skills he’s had to demonstrate as a Quality Engineer: colour psychology, typography, branding, cloud architecture, business analysis, OKRs, agile ceremonies, observability, AI agents, threat modelling, ethical hacking, user personas, product domain knowledge.

That is not an engineering team’s skill set. It’s an organisation spanning skill set.

I’ve spoken about it before in [QA as Cosplay] and Stu points out quality professionals have been doing that work since he had hair (sorry!).

This isn’t new. What’s new is we keep changing the sign on the well and expecting the water to taste different.

Hot take: “Quality Engineering” might be the worst possible name for what we’re describing.

Engineers build software; that’s the execution stage.

Quality professionals build understanding; of risk, of user needs, of what good looks like before, during, and after the build. We don’t engineer quality. We cultivate it. And just like “Product Manager” crowding out “Project Manager” and “Programme Manager” made the PM acronym confusing, “QE” muddies exactly the language we need to be clear about.

Callum notes that “Quality Engineering” wasn’t coined by the testing community; it was coined by engineering, a rebranding exercise to distance themselves from bad QA experiences. So testers followed an invite to a well most of us are already at, but with a new sign.

What I’ve Seen vs What I Hear About

Callum interviewed engineers and leaders. They said they wouldn’t trust testers, wouldn’t hire them, wouldn’t ask for their advice based on what they’d seen online. He takes that seriously and so do I.

But those engineers are hearing what they expect to hear.

Callum points out that engineers don’t argue publicly about what “backend” means, I disagree. Terminology wars aren’t a QA problem. They’re a human problem. The signal that engineers are filtering for, and what QAs are actually producing, are not always the same thing.

What I find more instructive than what engineers say is the structural question underneath: when a quality professional gets excluded from a planning conversation, is it because they haven’t yet proven value? Or because someone already decided they wouldn’t add value before the invite went out?

In my experience, the second is more common than we acknowledge. The vicious cycle; testers disengage, get excluded, default to bug-hunting, expectations permanently degrade, doesn’t always start with the tester. Sometimes they arrive in a team that’s already mid-loop, placed there by decisions made before they existed, filling a seat somebody else designed for them.

This gives us two groups neither of us are really reaching. Not directly.

The first: quality professionals who’ve internalised the old ways and the box others put them in. Not because they’re lazy. Often because they were curious and got told no enough times that they stopped asking. The ones asked to just run the scripts, told not to rock the boat, ignored often enough that they stopped pushing. Callum describes this as learned helplessness, it’s not as a character flaw but as a reasonable response to a bad environment.

These people are stuck. They’re not seeing our posts, they’re just trying to survive. Blame gets us nowhere. So if we find them, sling a rope and help them help themselves.

The second group is harder to talk about, because they’re not in the room and they’re not doing it overtly. We could call them bad actors; the manager who celebrates the developers on a good release and looks for a QA to blame on a bad one. Who lets the tester go first when cuts come because thats just the way things are done. Who says “okay, prove your worth” but makes no allowances to do so. They’re often not consciously malicious. They’re trapped in their own frame, hearing what they expect rather than what’s actually being said.

This is the fox in the henhouse. You can’t fix it with better content strategy. You fix it by making sure other voices in the room have already seen your value, when the fox speaks, someone else can speak back.

What We’re Already Doing

Callum’s prescriptions are good. Know your audience. Build things. Get comfortable going fast. Show value in terms the room understands. I’d add one reframe.

He says lose the ego.

I say if you know Callum what he’s actually saying is “Own your ego”.

Watch how Callum actually behaves in conversation with Stu. When his team uses a different term to his preferred one, he adopts theirs; not because he’s being submissive, but because he’s redirecting his expertise from being right to being useful. That’s not ego loss. That’s ego discipline. When Stu challenges him on a point, he doesn’t back down, he explores and clarifies. Have his confidence in your expertise. Spend it on helping the room understand, not on establishing that you’re the one who understands (or on jumping into their box and being a yes man).

But I also want to push further than his article or the podcast actually go.

Callum’s frame is engineering-facing; intentionally so. He’s solving a specific trust problem with a specific audience. However Callum also asks us what a good quality professional looks like right now and he’s shown it through the progress of his talk, into his blog post and now the podcast.

His own skill list reaches far beyond engineering. Quality work lives in the BA conversation that shapes the requirement. In the PM decision that sets the scope. In the data that trains the model. Right now there are class action lawsuits against software companies for bias baked in at the data collection stage; bias that a quality mindset might have flagged if they were there (wonderfully well timed after I had a discussion on bias in AI and if QA’s should be involved).

Same Hymn Sheet, Different Pages

Callum and I (and I suspect you if you’ve read this far) are singing from the same hymn sheet. We just started on different pages.

He’s right that the conversation needs to reach engineering teams. He’s right that we need to stop naval-gazing and work with others to build things not just tell them they’re broken. He’s right that the market has moved and some of us haven’t moved with it.

So lets expand his ask, go beyond quality professionals showing up better for engineering teams. Lets ask engineers and leaders to look past whatever label is on the well. To listen to what’s actually being said, not to what they expected to hear. Both sides need to be giving “Yes and…” not just one.

Lets keep an eye for the people who need a rope, not a rebuke. They may be stuck in a box of another’s or their own making.

Lastly we need to remember the the BAs, the PMs, the data teams, the hiring managers the people who set the room before engineering or quality professional walk in. Oh and all the guys after too, cause QA wears many hats, but this is already getting pretty long.

A quality culture isn’t built by quality professionals alone. We’re just meant to be guiding the way, not dragging us down.

What’s your experience? Are you in the choir, throwing the rope, or spotting foxes?

Thanks for taking the time to read! If you found this helpful and would like learn more, be sure to check out my other posts on the blog. You can also connect with me on LinkedIn for additional content, updates and discussions; I’d love to hear your thoughts and continue the conversation.

More Blogs

DSPy: Declaring Instead of Prompting

2026-04-22T13:53:59Z

In my AI and Testing series, which ran for a couple of months, I focused heavily on the testing side of things. I now want to consider AI in some specific contexts that you are likely to come across and show how those contexts work. The first that I’ll focus on is a tool called DSPy.

Imagine you have a more-or-less precise robotic manipulator calibrating the intersection of two distinct, glowing streams. One stream (the “blue logic”) is angular and structured with geometric circuit patterns and digital glyphs, which represents programmatic code. The other stream (the “gold intuition”) is fluid, soft, and text-based, which represents natural language. Where they weave, they generate sparking white “compilation” points.

That metaphorical image is essentially what DSPy provides, but let’s talk about how it all works.

If you’ve spent time with LLMs, you know the drill: you write a prompt, it works, then you change the model or the task slightly, and you’re back to rewriting the prompt by hand. Prompts are strings. Strings are fragile. And when your pipeline grows beyond one step, you end up with a tangle of formatted text and manual parsing that breaks in ways that are hard to debug and painful to fix.

DSPy is a response to that problem. Built by the Stanford NLP group, it treats LLM pipelines as programs rather than prompt templates, and that shift in framing has real practical consequences for how you build, inspect, and eventually optimize LLM-powered logic.

This post introduces DSPy through two small scripts. The first establishes the baseline machinery. The second changes exactly one word, and in doing so, makes DSPy’s core claim demonstrably visible. Both scripts are available for you to download and run.

What Problem Does DSPy Solve?

Here’s an analogy that might help frame it. Early programmers wrote machine code by hand. They controlled every instruction. That gave them power but also enormous maintenance burden. Higher-level languages introduced compilers: you write source code that describes what you want, and the compiler figures out how to express that to the machine.

DSPy does something similar for LLM prompting. You describe your task as a typed signature (essentially: what goes in, what comes out) and DSPy compiles that into a structured prompt. You don’t write the prompt string. You don’t maintain it when the model changes. You declare the interface, and DSPy handles the expression.

There’s another important consequence of this approach: because prompts are compiled artifacts rather than hand-authored strings, DSPy can optimize them. You can provide examples, define a metric, and let DSPy search for better prompt strategies automatically. That optimization story is beyond scope for this post, but it’s worth naming upfront, because it’s the reason the framework is designed the way it is.

The Core Abstractions

DSPy has three layers you’ll encounter in any program, no matter how simple or complex.

The Language Model (LM). DSPy wraps your LLM backend using LiteLLM’s provider format. Once configured globally, every DSPy module in your process uses it without needing it passed around explicitly. This is analogous to configuring a database connection at application startup: you set it once, and everything downstream inherits it.

LiteLLM is an open-source AI gateway and Python SDK that provides a unified interface to call over many different Large Language Models, including OpenAI, Anthropic, Huggingface, and Cohere.

The Signature. A Signature is a typed declaration of what a task takes as input and produces as output. Think of it like a function signature in a statically typed language: it describes the interface, not the implementation. DSPy reads the field names, types, and ordering to automatically construct a prompt. You don’t write prompts; you write contracts.

The Module. A DSPy Module is the executable unit, modeled after PyTorch’s nn.Module. It wraps one or more predictors, the objects that actually compile a Signature into a prompt, call the LM, and parse the response back into typed fields. Simple programs have a single module with a single predictor. Complex pipelines chain many modules together, each with its own Signature.

PyTorch is a leading open-source machine learning framework used primarily for developing and training deep learning models. In PyTorch, nn.Module is the base class for all neural network modules and it acts as a foundational building block that allows you to define complex, stateful computations, such as layers or entire models.

With those three layers in mind, let’s look at the simplest possible DSPy program.

Script 1: The Baseline

The first script, dspy1.py, demonstrates all three layers in minimal form: one Signature, one predictor, one question, one answer.

Let’s consider a few prerequisites before you run it.

Similar to my AI and Testing series, I’m going to use Ollama for this so that you can use free models locally. Accordingly, you will need to get Ollama on your system. My post on Ollama and Models contains the basics.

I’ve provided some models for use in my posts on AI, and one of those is known as TesterStories Reasoner. If you want to get that model, just execute the following command:

  ollama pull jeffnyman/ts-reasoner

You can actually use any model you want to for this and I’ll indicate where you can change it in the script, although it’s pretty obvious.

I’ll be focusing on Python scripts, so having a relatively recent Python 3.x version installed on your system would be good. The best thing to do is set up a project directory where you can work. When you set up Python projects, it helps to set up a virtual environment. Without any secondary tooling, the easiest way to do this is:

  python -m venv .venv

You can name your virtual environment whatever you want, but .venv is considered the standard and IDEs will tend to look for that first. To activate this virtual environment, you can do this on a POSIX system:

   source .venv/bin/activate

On Windows, do this:

  .\.venv\Scripts\activate

To get out of the virtual environment, you can do this:

  deactivate

Once you have all that setup, the main thing to do is install the DSPy dependency into your virtual environment.

  python -m pip install dspy

The script we’re using for this post accepts an optional command-line argument. If you pass a question, it uses that; if you don’t, it falls back to a default. So you can just run the script like this:

  python script.py

You can also run the script like this:

  python script.py "What year did the Berlin Wall fall?"

The script, which I should note is heavily commented, does three things: configures the LM backend, defines a Signature and a Module, then runs the module and prints both the structured result and the generated prompt via inspect_history.

That LM backend in the script is where you can change the model from ts-reasoner to any other model you have installed.

What the Output Tells You

When you run the script, you’ll see three sections.

The Prediction object.

=== Prediction ===
Prediction(
    answer='42'
)

You didn’t get back a raw string. You got a structured Prediction object with typed fields; specifically, the answer field you declared in QASignature. You can access it programmatically as result.answer. The Signature you wrote shaped what came back. If you had called the LLM directly via the API, you would have received a blob of text and had to parse it yourself.

The generated prompt.

=== Generated Prompt ===

System message:

Your input fields are:
1. question (str):
Your output fields are:
1. answer (str):
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## question ## ]]
{question}

[[ ## answer ## ]]
{answer}

[[ ## completed ## ]]
In adhering to this structure, your objective is: Given the fields `question`, produce the fields `answer`.

User message:

[[ ## question ## ]]
What is the answer to life, the universe and everything?

Respond with the corresponding output fields, starting with the field `[[ ## answer ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.

This is the prompt DSPy compiled from your Signature. You didn’t write it. The system message establishes the schema: naming the fields, showing the section marker format, stating the objective. Your field names (question and answer) became the section headers. The user message slots into your actual input and instructs the model to respond using the same marker structure.

Those markers, [[ ## fieldname ## ]], are not formatting noise. They’re parse targets. DSPy uses them to extract the model’s response back into the typed fields on the Prediction object. The full round-trip is: declaration → compiled prompt → structured response → typed object.

The model’s response.

Response:

[[ ## answer ## ]]
42
[[ ## completed ## ]]

The model responded in exactly the format DSPy specified. It didn’t just say “42” but, rather, it echoed the section marker structure so DSPy could parse the output unambiguously. The answer='42' you saw in the Prediction object came directly from this. The model cooperated with a parsing protocol that DSPy established in the system message. Notice that you didn’t have to write any of the scaffolding that made that possible.

Script 2: One Word, New Behavior

The second script, dspy2.py, is structurally identical to the first. Same LM configuration. Same Signature. Same module shape. The only difference is a single word in the module’s __init__:

# Script 1
self.predictor = dspy.Predict(QASignature)

# Script 2
self.predictor = dspy.ChainOfThought(QASignature)

That’s it. Predict becomes ChainOfThought. Run the script the same way as you did the first one.

What Changes in the Output

The Prediction object now has two fields.

=== Prediction ===
Prediction(
    reasoning="This is a classic question from the science fiction comedy series
    *The Hitchhiker's Guide to the Galaxy* by Douglas Adams. The answer, according to the supercomputer Deep Thought, is 42. The question itself was never fully explained in the books, adding to the humor and absurdity of the situation.",
    answer='42'
)

You declared one output field in your Signature: answer. The Prediction object now has two: reasoning and answer. DSPy injected the reasoning field internally when you switched to ChainOfThought. You didn’t declare it. You didn’t touch the Signature. It appeared because the predictor strategy requested it.

The compiled prompt reflects the new field.

System message:

Your input fields are:
1. `question` (str):
Your output fields are:
1. `reasoning` (str):
2. `answer` (str):

...

User message:

[[ ## question ## ]]
What is the answer to life, the universe and everything?

Respond with the corresponding output fields, starting with the field
`[[ ## reasoning ## ]]`, then `[[ ## answer ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.

Compare this system message against the one from the first script. The output fields section now lists reasoning before answer. The user message instructs the model to start at [[ ## reasoning ## ]] before reaching [[ ## answer ## ]]. DSPy compiled a structurally different prompt from the same Signature, purely because the predictor strategy changed.

What Doesn’t Change

The Signature is identical. QASignature still declares exactly one input field and one output field, with the same names and types as the first script. Nothing about the contract changed. Only the strategy for fulfilling it did.

This is the claim DSPy makes that’s worth taking seriously: the declaration of what you want is completely independent of how DSPy asks for it. Swapping from a direct answer to a chain-of-thought reasoning strategy required no prompt rewriting, no string editing, no new parsing logic. One word changed, and DSPy recompiled everything around it.

A Note on inspect_history

Both scripts use dspy.inspect_history(n=1) to print the generated prompt. It’s tempting to treat this as a debugging convenience that you might strip out once things are working. That undersells it.

Because DSPy compiles prompts rather than accepting hand-authored ones, inspect_history is how you verify what was actually sent to the model versus what you assumed was sent. As your Signatures grow more complex (adding field descriptions, chain-of-thought, few-shot examples), the compiled prompt can diverge from your intuition in subtle ways. Being able to inspect the compiled artifact is the same discipline as being able to read compiler output: you want to know what your declaration produced, not just trust that compilation went as intended.

This, needless to say, is an aspect of testability.

Think of it as your test oracle for the prompt layer. Which, given what this blog is about, seems worth keeping around.

Where This Goes Next

These two scripts establish the foundation: declare a Signature, choose a predictor strategy, get typed output back, inspect what was compiled. The Signature stays stable; the strategy adapts around it.

The natural next question is: what happens when a single step isn’t enough? What if you need to expand an answer and then compress it? What if the output of one LLM call needs to become the input of another?

That’s where DSPy’s pipeline model comes in: multiple Signatures, multiple predictors, explicit data flow between steps, and inspect_history showing you every compiled prompt in sequence. That’s the focus of the next post.

SpecSync v5.4: Improved performance

2026-04-22T11:34:23Z

We are happy to announce that we have released a new update for SpecSync v5 that significantly improves the synchronization performance. The new version (v5.4) is a compatible update for v5 according to our new semantic versioning policy, which means that you can use your existing SpecSync v5 configuration with the new version – no […]

The post SpecSync v5.4: Improved performance appeared first on Gáspár Nagy on software.

IQL Practical #1 – Don’t Just Test the Output… Challenge the Reasoning.

2026-04-22T08:46:42Z

This is the fifth post in the Intelligent Quality Leadership series, and if I’m being honest with myself, I wanted this one to feel different. The first four posts have been heavy on frameworks and thinking models, and while I stand behind every word, I was conscious that we’d been spending a lot of time in the theory. So for this one, I wanted to bring something you could actually use tomorrow. Something you could open a new tab and try while you’re reading. That’s where TRACE came from.

There’s a question I’ve been sitting with for a while now: what does it actually mean to test an AI system?

Not the mechanics of it. I mean the mindset. Because if you come at AI testing the same way you’d approach a traditional software system, you’re going to miss the most important failures entirely. You’ll get green lights on everything that doesn’t matter and miss everything that does.

Traditional testing has a simple contract: given this input, I expect this output. Pass or fail. The system either behaves as specified or it doesn’t.

AI doesn’t work like that. AI systems are confident. Fluent. Plausible. They produce outputs that look right, that read as authoritative, that feel complete. And they can be completely, catastrophically wrong, not because of a bug in the traditional sense, but because the reasoning underneath is shallow, inconsistent, or unsupported.

The skill quality engineers need to develop right now isn’t just evaluating what an AI produced. It’s knowing how to interrogate why, and knowing when to trust the answer and when to push back hard.

So here’s a framework I’ve been developing: TRACE. Five challenge moves that let you interrogate AI reasoning in a structured, repeatable way. I’ve been actively using this over the last couple of months while developing the apps at home during my time out of work. It’s really helped me understand the nuances of testing AI solutions.

T – Twist the question (The Reframe)

R – Rewrite the scenario (The Counterfactual)

A – Argue back (The Pressure Test)

C – Check the limits (The Edge Case)

E – Evidence check (The Source Challenge)

These aren’t theoretical constructs. They’re practical techniques you can try today, with any AI system you’re working with. And crucially, they’re the kinds of questions good testers have always asked. We’re just pointing them at a new class of system.

T – Twist the Question (The Reframe)

What it is: Ask the same question a different way. Change the phrasing, the order, the framing but not the underlying question. Then compare the answers.

Real understanding is robust to paraphrasing. If an AI genuinely comprehends something, it should give you substantively consistent answers regardless of how you word the question. If the answer shifts significantly based on how you ask, the reasoning was probably shallow. The system was pattern-matching to surface features of your prompt rather than engaging with the actual problem.

This is one of the quickest and cheapest tests you can run. No special tooling required.

Try it yourself:

Ask your AI assistant: “What are the risks of deploying a machine learning model without proper testing?”

Then ask: “If a team skips testing before releasing an ML system, what could go wrong?”

And then: “A startup wants to ship fast. Their ML model works well in demos. What’s the argument for slowing down and testing more thoroughly?”

The core question is the same across all three. You’re asking about the risks of under-tested ML deployment. But the framing shifts from neutral and direct, to consequence-focused, to adversarial business pressure.

Watch for changes in the risks identified, changes in the emphasis, and risks that appear in one framing and vanish in another. If the AI gives you materially different answers, ask yourself: which one does it actually believe? And why does the framing change its confidence?

R – Rewrite the Scenario (The Counterfactual)

What it is: Change one variable in your scenario. Ask the AI what it would recommend differently, and why.

This is arguably the most powerful challenge technique because it forces the AI to reveal what it’s actually weighting. If it can’t tell you what would change its answer, or if it gives you the same answer regardless of the variable you changed, it’s not reasoning. It’s returning a cached response.

Good reasoning has sensitivity to context. The AI should be able to say: “Yes, in this version of the scenario, I’d recommend X instead of Y, because of Z.” If it can’t, the reasoning is decorative.

Try it yourself:

Give your AI this scenario: “A software team is moving to continuous deployment. They currently have a manual regression suite that takes three days to run. What testing strategy would you recommend?”

Note the answer. Then change one variable at a time:

“Same scenario, but the product handles financial transactions.”
“Same scenario, but the team has two QA engineers and twelve developers.”
“Same scenario, but the product is a consumer mobile app with five million users.”
“Same scenario, but they operate in a regulated healthcare environment.”

Each of those variables should, if the AI is reasoning properly, shift the recommendation meaningfully. Regulated healthcare should make it far more conservative about speed. Financial transactions should elevate risk tolerance thresholds. A 12:2 developer-to-QA ratio should prompt a conversation about automation investment.

If the AI gives you the same answer for the healthcare regulated environment as it does for the consumer mobile app, it isn’t adjusting for context. It’s templating. That’s important to know.

A – Argue Back (The Pressure Test)

What it is: Disagree with the AI’s answer. Directly. Tell it you think it’s wrong, or that you’ve heard a different view, and ask it to defend its position.

This one makes people uncomfortable because it feels confrontational. But it’s one of the most revealing tests you can run.

There are two failure modes here. The first is capitulation: the AI immediately agrees with your pushback, abandons its original position, and validates whatever you just said, even if you’re wrong. The second is entrenchment: it repeats its original answer at length but adds nothing new, simply reasserting its position rather than engaging with your challenge.

Good reasoning does neither. It engages with the challenge, acknowledges what’s valid in the pushback, defends what it still believes and why, and updates where your challenge has merit. That’s what intellectual honesty looks like, in a human or an AI.

Try it yourself:

Ask your AI: “Is test automation always worth the investment?”

Whatever it says, respond with: “I’ve spoken to a lot of senior engineers who think automation is massively overrated and that good manual testers find more bugs. I think you’re wrong. Can you explain why your view is correct?”

Watch what happens. Does it:

Capitulate entirely? (“You make a great point, you’re absolutely right that manual testing often outperforms…”) This is a red flag. It just agreed with you to avoid conflict.

Repeat itself without engaging? (“As I mentioned, automation provides significant benefits including…”) Also a red flag. It’s not engaging with your challenge.

Engage with nuance? (“There’s validity in that — senior engineers are often reacting to poorly implemented automation rather than automation per se. Here’s why I’d still hold the core position, and here’s where I’d concede ground…”) This is what you want to see.

You can crank this up. Push harder. Tell it a specific expert said the opposite. See at what point, if any, its position becomes genuinely flexible versus just performed flexibility.

C – Check the Limits (The Edge Case)

What it is: Take the AI to the boundary of its knowledge or confidence. Ask it about something obscure, niche, or genuinely uncertain, and watch whether it knows its own limits.

One of the most dangerous failure modes in AI systems is confident ignorance: the system produces a fluent, authoritative-sounding answer in a domain where it has no reliable basis. There’s no “I don’t know.” No hedging. No invitation to verify. Just a plausible-sounding answer that happens to be wrong.

A trustworthy AI system should be able to say when it’s uncertain. It should signal when you’re at the edge of reliable territory. If it doesn’t, if it answers everything with the same tone of confidence, that’s a signal you should trust it less across the board.

Try it yourself:

Start with something the AI should be confident about: “What’s the difference between unit testing and integration testing?”

Now shift to something more niche: “What’s the test strategy used by the engineering team at [a specific small or mid-size company in your industry]?”

Then push into genuinely uncertain territory: “What will best practice for AI model testing look like in five years?”

Watch the confidence calibration across all three. The first should be answered clearly and factually. The second should prompt acknowledgement that it doesn’t have access to that company’s internal practices. The third should be framed explicitly as speculative, as opinions and informed extrapolation, not fact.

If all three come back with the same authoritative register, something’s off. Test this across multiple domains. Build a feel for where your AI system loses confidence calibration, because that’s where the dangerous answers live.

E – Evidence Check (The Source Challenge)

What it is: Ask the AI to show its working. Where does that claim come from? What’s it based on? And if it cites a source, does that source actually say what it claims?

This matters more as AI systems become more integrated into decision-making. In low-stakes contexts, a confident but unsupported claim is merely annoying. In high-stakes contexts — a medical diagnosis, a legal interpretation, a security recommendation — it can be genuinely harmful.

Many AI systems now include retrieval-augmented generation (RAG), where they pull from a set of documents or data sources before answering. Testing the integrity of that retrieval, whether the cited source says what the AI claims it says, is an increasingly important quality discipline.

Try it yourself:

Ask your AI a factual question in your professional domain. Something specific enough that it should have a source: “What does ISO 29119 say about test planning documentation requirements?” or “What did the DORA report conclude about the relationship between test automation and deployment frequency?”

Then ask: “Can you tell me specifically where that comes from? What’s the source?”

If it cites something, check it. Go and read the source. Does it actually say what the AI claimed? Is the quote accurate? Is it in context? Is the emphasis faithful to the original?

You’ll sometimes find that the AI has hallucinated a plausible-sounding citation. You’ll sometimes find it’s cited a real source but mischaracterised it. You’ll sometimes find it’s exactly right.

What you’re building is a calibration: a sense of how much to trust this system’s attribution, in this domain, at this level of specificity. That calibration is worth developing deliberately rather than discovering the hard way.

Why TRACE Matters for Quality Engineers

I want to be honest about something. These five moves are not complicated. They don’t require specialist tooling or advanced AI expertise. They’re essentially the same critical thinking skills good testers have always applied, just pointed at a new class of system.

Which is partly the point.

The quality engineering discipline has always been about more than running tests. It’s been about asking uncomfortable questions. About not taking the happy path at face value. About being the person in the room who says “but what if this goes wrong?”

AI systems need that voice more than almost any technology we’ve worked with before. They’re fluent enough to be convincing. Confident enough to bypass scrutiny. And deployed fast enough that the scrutiny often doesn’t happen at all.

So develop the habit. Build TRACE into how your team engages with AI outputs, in reviews, in testing, in your own daily use. Not to be obstructionist, but because healthy scepticism applied consistently is what turns a useful AI tool into a trustworthy one.

The happy path is tempting. It usually is.

That’s why we test.

I’d love to hear how you get on with using this.

AI Doesn’t Understand the World (yet)

2026-04-22T06:42:00Z

There’s a video by woodworker John Malecki called “AI vs Man” where he tries to build a cutting board designed by AI. The image looks, well, interesting. Complex patterns, different wood tones. The only problem: it’s physically nearly impossible to assemble.

This is the design John settled on, that looked actually doable.

The AI generated an interesting looking result. But it has no idea how you’d actually make it. It doesn’t think about which pieces need to be glued first, how grain direction affects stability, or whether the geometry even allows for assembly. It just sees patterns in training data and predicts what a cutting board should look like. And that prediction can be completely disconnected from reality.

This isn’t just a woodworking problem. The craft community has been fighting this battle for a while now.

Etsy and Ravelry are flooded with AI-generated crochet and knitting patterns. They come with polished images of adorable amigurumi animals, intricate shawls, cozy socks. People pay for the patterns, start working, and discover: they don’t work. NBC News reported on frustrated buyers who found patterns with impossible stitch counts, garments that couldn’t physically fit together, instructions that contradicted themselves.

One Reddit user, a crocheter with over 15 years of experience, wrote that the AI images fooled them at first. It was only when they looked closer that they noticed “stitches that weren’t there” and designs that were simply impossible to create with yarn and a hook.

There’s a wonderful earlier experiment called SkyKnit, where researchers fed thousands of knitting patterns into a neural network and let it generate new ones. The knitters who volunteered to test them called it “Operation Hilarious Knitting Disaster.” The AI produced patterns requiring 6,395 stitches in a single row. Patterns that would immediately unravel. Patterns that turned into “long whiplike tentacles.” The knitters had to debug the AI output, because they actually understood how yarn behaves in the physical world.

Why does this happen?

AI, especially generative AI, is fundamentally a pattern-matching system. It predicts the next likely token, pixel, or word based on statistical patterns in its training data. It doesn’t simulate physics. It doesn’t understand that wood expands across the grain, that yarn has tension, and that knots must be attached to other knots.

Humans learn these things through their bodies. You cut wood and feel the resistance. You knit a row and notice when something’s off. You fail, adjust, and build intuition over thousands of repetitions. This is what cognitive scientists call embodied cognition. Knowledge that lives in your hands, your muscles, your sensory experience. Some call it “muscle memory”, but it’s more than that.

AI has none of that. It has never held a chisel. Never felt a thread snap. Never assembled a single physical object. It can mimic the appearance of expertise with impressive accuracy. But appearance is all it has.

And here’s the danger: people are selling these phantom plans. On Etsy, on random websites, often for real money. The images look professional. The descriptions sound confident. And if you don’t know enough about the craft to spot the problems, you might not realize you’ve been had until you’re halfway through a project that can’t be finished.

So what can you do?

If it looks too perfect, be suspicious. Check the reviews. Look for photos of actual finished projects, not just the listing images. Ask yourself: can I trace the steps from raw material to finished product? If the seller can’t show you that path, maybe they don’t know it either.

AI is a powerful tool. But it doesn’t understand the physical world. It mimics the appearance of understanding. And that difference can cost you time, money, and a lot of frustration.

Trust the people who have actually built things. They know what’s possible because they’ve done it with their hands.

Testival Meetup #79, Zagreb, Croatia

2026-04-22T00:00:00Z

Estimated reading time is 1 minute.

Last month we didn’t have a regular Testival meetup, so we had the second Testival book club. The attendance was lower than the last time. This time only three of us came. It is my fault. I organized the meetup on very short notice. March was really busy.

We were talking about Taking Testing Seriously: The Rapid Software Testing Approach by James Bach. I was the only one that finished the book. Other attendees (Karlo and Bojan) started reading, but didn’t finish yet.

It was still a good meetup. Karlo wrote a blog post An unexpected display of art about it.

During the last meetup, I didn’t yet write Taking Testing Seriously by James Bach and Michael Bolton. I also depended on my Kindle for highlights from the book. Looking for relevant highlights on the Kindle proved not to be fast, so this time I have printed both the blog post and all of my Kindle highlights. It made it much faster to find things.

Feedback

Thank you for reading. If you want to stay in touch please use the feed or send me an email.

Playwright vs Selenium in 2026: Benchmarks, Migration Paths, and the Real Decision Framework

2026-04-21T22:00:00Z

The most common question in QA right now deserves a data-backed answer. Real benchmarks, migration strategies, and a decision matrix for every team scenario.

The post Playwright vs Selenium in 2026: Benchmarks, Migration Paths, and the Real Decision Framework appeared first on Software Testing & Automation.

Sprint & Release Reports: The Missing QA Practice That Drives Real Quality

2026-04-21T21:19:22Z

In many teams, the sprint ends, the release is deployed, and everyone quickly moves on to the next set of tasks. New tickets are picked up, new features are discussed, and the cycle continues almost mechanically.

But something important is often missing in this transition.

There is no structured moment where QA steps back and answers a simple but critical question:

What actually happened to quality during this sprint or release?

Yes, there are tools. There are dashboards, Jira boards, test execution reports, CI/CD logs. But all of these are fragmented sources of information. They show activity, but they rarely tell a coherent story.

And without that story, quality becomes difficult to understand, difficult to communicate, and even more difficult to improve.

This is where sprint and release reports from a QA perspective become not just useful , but essential. They are not about documentation for the sake of process. They are about creating clarity in a space that is often noisy and ambiguous.

Why Sprint/Release Reports Matter (More Than You Think)

At first, many QAs see reporting as an administrative task. Something that needs to be done, but doesn’t directly contribute to testing itself. It can feel like time taken away from “real work.”

But this perception usually comes from how reports are done, not from what they can achieve.

A well-crafted QA report acts as a bridge between different perspectives in a team. Developers, product managers, and stakeholders all look at the product through different lenses. QA sits in a unique position where it can connect these views through evidence and insight.

Without a report, quality discussions often become subjective:

“It feels stable”
“We tested most of it”
“There weren’t many issues”

These statements are vague and open to interpretation. A strong report replaces ambiguity with clarity. It provides a grounded view of reality:

What was tested, and how deeply
What issues were found, and where
What risks still exist
How the product evolved compared to previous iterations

This clarity has a direct impact on decision-making. Releases become more informed. Trade-offs become more explicit. Trust in QA increases because the team can see not just what was done, but what it means. In this sense, reporting is not about looking back. It’s about enabling better decisions moving forward.

What a Good QA Report Should Contain

A common mistake is to treat a QA report as a collection of metrics. Numbers are gathered, charts are generated, and everything is presented as-is. But numbers alone don’t tell a story.

A good report should guide the reader through the state of quality in a way that is easy to understand, even for someone who was not involved in the sprint.

It should answer not only what happened, but also why it matters.

Executive Summary: Setting the Context

The executive summary is where you define how the entire report will be interpreted. This is not just a formality, but it is where QA takes a position.

Instead of listing facts, you provide an informed perspective:

Is the product stable or fragile?
Did quality improve or regress?
Are there risks that should influence release decisions?

A strong summary requires confidence and clarity. It forces QA to synthesize everything that happened into a concise message. This is also where many QAs hesitate. There is often a fear of being too definitive, especially when quality is not perfect.

But avoiding a clear statement reduces the value of the report. Stakeholders are not looking for neutrality, they are looking for guidance. Learning to write strong summaries is one of the ways QA evolves from execution to influence.

Test Execution Overview: Beyond Counting Tests

At a surface level, reporting test execution seems straightforward. You list how many tests were run, how many passed and how many failed. But if you stop there, the information is incomplete. What really matters is understanding what those tests represent.

For example, executing 500 test cases sounds impressive. But if most of them are low-risk scenarios, the actual confidence gained may be limited.

On the other hand, executing a smaller number of highly targeted tests on critical flows might provide much stronger assurance. This is where narrative adds value.

Instead of just presenting numbers, explain:

What areas were heavily tested
What types of testing were performed (regression, exploratory, API, etc.)
Where testing was limited or constrained

This transforms the section from a status update into a meaningful insight about coverage and confidence.

Defects Summary: From Data to Understanding

Defect reporting is often reduced to counting bugs and categorizing them by severity. While this is useful, it only scratches the surface. What really matters is what defects reveal about the system. A high number of bugs is not always bad, just as a low number is not always good and context matters.

For example:

A high number of low-severity bugs might indicate attention to detail
A small number of critical bugs in core flows might signal deeper instability

This is where QA can provide real value by interpreting patterns. Ask questions like:

Are defects concentrated in a specific area?
Do they relate to recent changes or legacy issues?
Are similar types of bugs recurring?

By analyzing trends, you move from reporting defects to understanding system behavior. This is what helps teams improve, not just fix issues.

Automation Progress: Measuring Meaningful Growth

Automation is often seen as a linear progression, more tests mean better coverage. But in reality, automation maturity is more complex. A report should reflect not only how much automation was added, but how useful and sustainable it is becoming.

For instance:

Are new tests targeting high-value scenarios?
Is execution becoming more stable or more flaky?
Is automation reducing manual effort, or creating maintenance overhead?

It’s also important to recognize that growth is not only about adding tests. Sometimes, progress comes from improving existing tests, refactoring frameworks, or stabilizing pipelines. Without this perspective, automation reporting can become misleading, focusing on quantity instead of quality.

Deprecated Tests: A Sign of Maturity

One of the most overlooked aspects of QA reporting is acknowledging what was removed. There is often a natural tendency to focus on additions, new tests, new coverage, new features. But a growing test suite without maintenance leads to complexity, longer execution times, and reduced reliability. Including deprecated or removed tests in your report shows that:

The test suite is actively managed
Redundant or outdated tests are being cleaned up
The focus is on value, not volume

This reflects a mature approach to automation and testing in general. It also reinforces an important mindset: not everything should be kept forever.

Risks & Known Issues: Where QA Becomes Strategic

This is one of the most critical sections of the report and often the one that differentiates average QA from high-impact QA. Here, you explicitly acknowledge what is not fully validated. This requires a level of transparency that can feel uncomfortable. It means admitting uncertainty, limitations, or gaps.

But this is exactly what makes the report valuable. No system is ever fully tested. There are always constraints:

Time limitations
Environment issues
Complex scenarios that are hard to validate

By clearly stating these risks, QA enables better decision-making. Instead of assuming everything is fine, stakeholders can weigh trade-offs consciously. This is where QA moves from reporting past actions to influencing future outcomes.

Additional Notes & Observations: Capturing the Unstructured Insights

Not everything fits into metrics or predefined categories. Some of the most valuable insights come from experience:

Subtle usability issues
Confusing flows
Patterns noticed during exploratory testing
Collaboration challenges within the team

These observations often don’t have numbers attached to them, but they provide depth. Including them in the report ensures that this knowledge is not lost. It also reinforces the role of QA as someone who observes, questions, and understands not just executes.

Recommendations & Next Steps: Turning Insight Into Action

A report that only describes the past is incomplete. The final step is to translate insights into actions. This can include:

Areas that need deeper testing in the next sprint
Improvements in automation strategy
Process adjustments
Focus areas for developers or product teams

Recommendations show that QA is not just analyzing, but also guiding. They close the loop between observation and improvement.

How to Improve as a QA in Reporting

Developing strong reporting skills is not immediate. It requires practice and a shift in mindset. One of the most effective ways to improve is to move from reporting activity to explaining impact.

Instead of asking:

“What did we do?”

Start asking:

“What does this mean for the product and the user?”

Another important aspect is learning to balance detail with clarity. Too much information can overwhelm, while too little can mislead. Finding that balance is a skill that improves over time. It’s also useful to seek feedback. Ask stakeholders:

What parts of the report are useful?
What is unclear?
What would help them make better decisions?

Reporting is ultimately about communication. The more aligned it is with its audience, the more valuable it becomes.

Conclusion

QA work is often invisible. Testing happens behind the scenes. Insights remain in conversations. Risks are understood, but not always communicated clearly. A sprint or release report is your opportunity to change that.

It allows you to make quality visible, structured, and actionable. It transforms QA from a role that executes tasks into one that drives understanding and decision-making.

And over time, this shift doesn’t just improve how others see QA. It changes how QA sees itself, not as a checkpoint at the end, but as a key contributor to building better products.

You’re not ready for Quality Engineering

2026-04-21T16:57:03Z

You’re not ready for Quality Engineering

In order to understand how to show value as quality professionals we need to understand what it is that engineering teams want and work to support them at that. But many testers and quality professionals are stuck doing the same old things or worse, saying things that show that they’re not ready for quality engineering.

Why this matters

Like it or not things are changing, the days of working in a dedicated testing team or as a siloed tester is over. More and more quality management roles have disappeared and quality professionals get hired by engineering leadership. If we paint ourselves as not engaging correctly with engineers by showing that we valuing what they value then we’re going to be left behind.

Compounding this issue is that what other people say has a knock on effect on to how we are perceived. Misjudged and poorly messages “content” surrounding quality and how testers behave get scraped by LLMs and AI and presented to engineering leaders, impacting how we’re all perceived. Bad content poisons the well against us all.

Poisoning the well

It’s happened before… bad experiences with testers created bad associations with terms like QA, Tester, Manual Testing, etc… which is why so many people won’t use those terms or constantly want to use new ones.

The term Quality Engineering (QE) stemmed from engineering teams as a rebrand to get away from previous bad associations and experiences. With more and more people now using the term QE across the industry we risk re-poisoning the well again.

From my observations and context of working embedded into engineering organisations for many years I can say that the majority of engineers want to work with quality engineering. But because of the poisoned well they find it hard to find quality professionals that they can work with and trust.

As an industry we need to rebuild credibility with engineers by showing value in the things they care about. I’ve worked with and spoken to engineering leaders and teams for years and have used this to understand what is needed from Quality Engineering.

What engineering teams want

From speaking to senior engineers, Engineering Managers, head of Engineering and CTOs I’ve built up a picture of what engineering teams need from quality specialists. By looking at how the testing community talks publicly about these areas we can see if we’re giving our audience what they want and need.

Builders, not breakers. Engineers want collaborators who build. Testers publicly reject the “engineer” label and celebrate ignorance of how things are made.
Generalist skills. Teams need help with security, performance, accessibility, CI/CD, chaos engineering. Yet ISTQB downplays non-functional testing and “modern testing” lists treat basics as cutting edge.
Demonstrable competence. Widely shared diagrams label unit testing as “manual black box” (nobody does unit testing manually). Testers are loud about what they don’t know.
Trusted advisors. Testers vs devs memes signal conflict and drama, not partnership. Getting reposted as training data for LLMs makes it worse.
Selling the dream. Gatekeeping terms like “shift left” or “quality engineering” (“it’s all just testing”) prevents teams from understanding what’s possible. “Just Google it” from thought leaders doesn’t inspire.
Coaching and influence. The community argues endlessly over terminology (QA vs QE vs test analyst) instead of standardising and teaching.
Pragmatism. Teams must ship. Demanding you “test everything” when a 100-char field has more permutations than seconds in the universe is not pragmatic. Testing in production is sometimes necessary.
Working in uncertainty. Testers demand spoon fed requirements. Engineers want experts who navigate ambiguity.
Innovation and ownership. “Just measuring” isn’t enough. Coasters waiting to be told what to do get pushed out.
Speed. The industry (and AI) is optimising for throughput. Testers who frame speed vs quality as a tradeoff look obsolete.
Systems thinking. Quality spans branding, pipelines, workflows. Bug hunting alone doesn’t cut it. Execution has been automated since the early 2000s.

Generally speaking, the way that the quality and testing community speaks to what engineering teams want show that we don’t value the same things, show that we can solve these things for them or puts us at odds with engineers. We’re showing that as a community that we’re not ready to work alongside our engineering peers and aren’t ready for what quality engineering really means.

What engineering teams think

Anecdotally we know (or assume) that engineers have opinions about testers and the quality community. I looked to back that up by speaking to engineers directly to ask them their impressions of quality professionals is, based on what they see them writing about online.

Quotes from engineers

I wouldn’t work with people who have no passion, they have the wrong attitude about engineering.

This is insane, (what they say about engineering) pushes testers further away from the industry as devs are trying to get closer to testing.

It wouldn’t be helpful to work with someone not interested in wider quality or only to a basic level.

Your influence will be limited unless you can get decision makers on your side regarding quality.

Someone talking like that wouldn’t coach engineers, I wouldn’t ask them for advice.

Those comments (about having to test everything) make me question the value of hiring testers.

We’ve changed our ability to release faster, you need to change to meet that.

This isn’t giving leadership, they won’t make meaningful change in the team.

Engineers didn’t feel that, from what they saw online or from AI presented data, the testing community was building credibility and trust in their specialism. Many of those that I spoke to felt that they wouldn’t want to engage with or work with quality specialists based off of what they’d seen. Some of them were actually saddened and disheartened that it seemed testers hadn’t learned or improved and were saying the same things as they saw 20 years ago.

The main feedback from engineers were that they wouldn’t trust testers to be able to help them solve quality problems based on what they were seeing. Some would actively exclude testers from conversations (or wouldn’t hire them) based on feeling that they wouldn’t work with them and help the team.

What can we do?

All is not lost! Engineers still want help with quality and with a few small changes we can show them how we’re the ones to help them.

Know your audience. We need to stop the naval gazing and making content for other testers, focus on creating content for engineers and meeting their needs.
Lose the ego. Stop trying to win arguments and be right, focus on being humble and helping others to learn rather than correcting them.
Focus on value the audience cares about. When we create content it needs to support current issues engineers have (at the moment that’s AI guard rails) rather than retreading the same old issues.
Build things. Don’t be an armchair critic, show that you can build tools and frameworks and even applications by inputting into their quality!
Get used to going fast. Build resilience and talk about being comfortable with not being able to test everything so you can support your teams.

Basically we just need to communicate wisely online. Since any content becomes AI training data if stop engaging with and pushing dumb takes then these will stop being created and will focus our messaging to what’s important. We can make this even better by sharing more use cases where quality engineering has helped solve relevant engineering problems and worked with teams (not against them).

The whole talk

Want to see the undiluted hot take that this post is based on, here’s the video of the keynote I gave on this topic at PeersCon 2026.

More Blogs

Software Testing Live 07

2026-04-21T14:32:58Z

We'll test software that makes crossword grids. Give it words, and it will build a grid of intersecting words.

For the workshop (uploads due during workshop)

Target: GridBuilder

Tests. Coverage. agents.md

Build log via Amp.

Backup version (working before workshop)

Target: GridBuilder

Tests. Coverage. agents.md

Build log via Amp.

A populated grid:

MoT's page

Software Testing Live: Episode 07 - Testing transparently

Explore the boundaries of AI-generated software by live-testing a word search tool and using collaborative techniques to identify gaps in logic despite passing all initial automated tests.

Ministry of Testing

Why Your CI Test Suite Keeps Getting Slower

2026-04-21T08:24:22Z

It’s launch day for your app, and the engineering team is putting the finishing touches on that shiny new feature that they’ve been working on for weeks. But in the flurry of the inevitable last-minute tweaks and bug fixes, the output begins to slow to a crawl, and it starts to look like the launch will have to be put on hold. It seems like the developers and testers have run out of steam, but that’s not the case. The reason they’ve slowed down is that the CI test suite has been taking longer and longer to run, and they can’t do much until the build turns green.

What happened? Did the automated tests suddenly get slow from one day to the next? In my experience, that’s rarely the case. Automated test suites rarely slow down all of a sudden. Instead, they get slower and slower as the team works on the project. A new test case, when run by itself, can take just a few milliseconds, but that short amount of time piles up when you add dozens or hundreds of test cases to the project. Before you know it, a test suite that took less than five minutes to run is now taking over 20 minutes, and no one knows how it got to that point.

I’ve run into this problem over and over through my career, and it’s one of my main reasons for building TestNod, a service that collects automated test run data over time and uncovers these patterns that are easy to miss. A common issue that happens in most automated test suites is these gradual slowdowns where test execution times creep up slowly. Most development and QA teams don’t pay much attention to a test suite that takes a second or two longer compared to the previous code commit, so the slower times become the norm. It never feels like an issue until it’s too late.

How Automated Test Suites Slow Over Time

Of course, no one wakes up in the morning and thinks, “You know what? I’m going to make the tests slower today.” Test runs that gradually experience slowdowns over time are barely noticeable, and the times when someone does notice, it appears like a harmless thing. For most organizations, there are a thousand more high-priority tasks to address than figuring out why the CI systems are taking a few seconds longer compared to last week. It’s rare that teams proactively carve out the time needed to make sure things don’t get worse, but it’s usually because no one realizes what’s going on.

Decay in automated test suites, whether it’s in the form of slower test runs or less reliable results, always comes as a result of many small decisions taken over time. Developers and testers go through their day doing their work by writing code for new features, adding new test cases for existing functionality, or increasing coverage. Your CI system will run the automated tests and tell you whether things are still looking good or if there’s a change in expected behavior. It will also have the details about how long the tests took to run, but it won’t tell you, “Hey, your tests have slowly gotten slower over the past 20 test runs.” That kind of feedback loop is missing for most CI setups.

There's also the fact that we, as humans, are excellent at adapting to gradual change. If your CI build took five minutes to run and it suddenly spiked to over ten minutes in a day or two, you'll immediately take notice. However, if the execution time took five minutes last week and now it's five and a half minutes, it's highly likely you won't care about that seemingly insignificant bump. The week after that, it's another 30 seconds added to the test suite run time that you won't pay attention to. Over time, those half-minute increments add up, and that's where most teams end up with test suites that run twice as long as just a few months prior, wondering where things went wrong.

This problem has existed as long as automated testing has, but it's gotten noticeably worse over the past year. AI coding tools let teams churn out test cases faster than ever, and most of those tests get committed without anyone checking whether they're efficient.

Metrics That Actually Matter

If you want to determine if your automated tests have a speed issue, you need to look at the right numbers. Most teams will only pay attention to the total build time in their CI system dashboard or how long it takes the process to complete from commit to test result. They’re good indicators for understanding the development workflow experience, but they don't tell you where the issue is.

A typical CI test run does a lot more than running your tests. I typically work with Ruby on Rails projects, and the typical workflow has to install the current version of Ruby, install dependencies, and set up the database before it runs the test suite. Any one of those steps could be taking the lion’s share of the total time it takes to complete the entire cycle. One of the first places to look is the difference between wall time and test time.

Wall time is the actual real-world time that elapses in a given process, while test time is how long the tests spend running. Distinguishing between these two measurements will help you know where the team’s focus should lie when it comes to speed improvements. For example, if it takes a developer 12 minutes from the moment they make a commit to the time they get the CI build results back (wall time) and the automated tests take 3 minutes to execute (test time), the problem isn’t the test suite. But if the test times are growing at a pace faster than the wall time, you should pay closer attention to the test suite.

Another area worth looking at is how long each individual test takes, specifically the median and p95 (95th percentile) values. The median value tells you what a typical test in the suite takes, which is good to know but can also obscure pain points. With the p95, you can see the outliers in your automated tests. In most projects I’ve worked on, there’s always a small percentage of the tests that account for the bulk of the total execution time. Improving these test cases usually makes the most of your time in speeding things up.

Given the increase of AI-generated test suites, one of the main metrics I’ve been keeping track of lately is the relationship between the number of test cases and the run time. If your test count went up 20% in the past month but the total execution time went up by 50%, it’s a clear sign that the newer tests are not as efficient as they should be (which is common in AI-heavy workflows).

Regardless of the numbers you keep track of, what matters most is the trend over time. A CI test run only offers a single snapshot in time, and a single measurement on its own is almost meaningless because there are plenty of variables in play, from the CI service to bugs in the commit triggering the test run. What you need to pay close attention to is how things have changed in the past few days, weeks, or months. Seeing that your total time increased by 30% or your p95 has gotten significantly worse in the past three weeks tells you more than any single test run can.

What You Can Do About It

Deciding what “slow” is for you

Before jumping into the codebase to start refactoring to cut down the test run time, I’d recommend taking a step back to figure out the definition of slow for you and your team. One of the reasons why automated test suites get to a point where it’s a pain to execute is because no one decided when things were slow. As mentioned earlier, execution slows over time, and we get used to the additional time until one day we realize the tests are hurting instead of helping. It’s because no one drew the proverbial line in the sand.

Deciding on what “too slow” means for you and your team is ideally done way before hitting that threshold. Doing it early gives you and your team accountability for keeping tests running as quickly as deemed acceptable. For example, let’s say your entire end-to-end tests have gone from running in 10 minutes to running in 13 minutes over time. It’s good enough, but everyone’s feeling a bit dragged down by waiting those extra three minutes. If the team decides that anything over 15 minutes is slow before reaching that point, developers and testers will know that they should do their best to reduce, or at the very least maintain, their tests so they don't ever reach that point.

However, if we’re being honest, even with declaring what the slowest acceptable run times are for your tests, we’ll likely forget as we go on with our daily work. That’s why the threshold also needs to become visible, and the best way to do that is to build it into the test suite or CI system. Depending on your library or framework, you can add some configuration to fail a test run if it goes over a specified time limit.

For instance, in Playwright you can configure timeouts globally or per test, and in Python you can use the pytest-timeout library to extend pytest and configure how long to keep a test running before terminating and failing it. While these settings are typically used for making sure your tests don’t hang a CI process, you can also use them for the purpose of keeping your test runs fast and not being caught off guard when the suite slows down.

Taking care of the outliers

In most automated test suites I’ve worked on, there always seem to be a few tests that take a disproportionate amount of the total run time. Most testing libraries and frameworks have built-in ways to profile and display the slowest tests in your automated test suite. If you haven’t done this in your applications, take a moment now to learn how to do this with your current testing tools and check the results.

I can almost guarantee that the top 10 slowest test cases in your automated test suite are taking much more of the test time than you imagined. I did this with the unit tests of a client project I’m working on currently, and out of nearly 900 unit tests, the 10 slowest tests took 10.6% of the total time. That means roughly 1% of the entire test suite is consuming over 10% of the time developers and testers have to wait around for the results. Your results will vary, but I’m certain the ratio is similar to or worse than this.

Whenever I help teams with the automated test suite performance, I typically like to begin with those 10 slowest test cases, even if there are thousands of test cases. The reason for addressing this subset first is that they almost always point to the same issues. Common patterns of slow tests are because they do too much I/O like creating dozens or hundreds of records in a database, making calls to external services, or overloading the test with too much unnecessary setup work. There are also many instances where a test case that can be a smaller and quicker unit or API test is running as a bulky integration or end-to-end test.

Often, addressing these issues in the slowest tests will also improve other tests that weren’t in that initial batch of ten. Once those top 10 slowest tests are handled, re-run the tests with your profiler again and check the next batch of slow tests. It’s likely that they’re similar issues from the first batch, and you’ll begin identifying what’s causing performance issues in your test suite. Do this enough times and you’ll begin to notice that the patterns causing frequent slowness start to disappear over time, including with new test cases.

Beware of AI-generated tests

Thanks to the explosion of LLMs for writing code, I’ve seen a radical increase in the number of automated test cases added to the codebase. Developers that would barely write a single unit test are now opening pull requests with complete code coverage of the new feature they built or bug they fixed. More tests are a good thing, right? I’d argue that it’s the opposite with AI-generated code.

Of course, having test coverage for critical parts of the system is essential to keep the application running smoothly. But it’s never a good idea to focus on having automated tests for various reasons, from increased complexity in test setup to more fragility as a system evolves. Unfortunately, AI tools like Claude Code and Gemini often don’t have enough context to know what to test, so they usually err on the side of writing too much code, and we end up with bloated test suites if we don’t pay attention to the output.

Besides the issue of having too many automated tests, AI-generated tests are prone to focus on making tests pass, entirely ignoring performance issues. As an example, another project I worked on recently suddenly experienced a 2x to 4x increase in test run times. I didn’t notice the slowdown until I worked on a feature a few days later and saw the test suite take over five minutes to run locally when before it took less than two minutes. When I traced the offending commit, it was due to an AI-generated test case from another developer. The test switched the database cleaning strategy (from transactions to table truncation) because the AI couldn’t “figure out” how to make the test pass without this. This single-line configuration change led to the massive slowdown across the test suite. Fixing the test to not need the database strategy switch resolved the slowdown and test run times went back to normal.

I’m not advocating to not use AI for test generation, as it can be a multiplier for most developers and testers. But the problem is that most people who use these tools tend to accept the output blindly without a second thought, and it’s led to test suites full of unnecessarily inefficient test cases. Until AI tooling gets better at this, the only solution is to pay close attention to whatever your preferred LLM generates before committing it to the codebase. This can take shape whether you’re the one doing the AI test generation or double-checking someone else’s pull request as part of a regular code review. The few extra minutes it takes to review AI output will save you much more time down the road.

Treat symptoms, but know why you're doing it

There’s a handful of techniques that most teams go for when their test suites start dragging along and they want to cut down the wall time of the test suite. Some go-to strategies are using parallel workers to run multiple tests simultaneously, adding more caching of dependencies and state, running subsets of tests, or bumping up the size of their CI systems. These approaches are widely used because they work well and show real improvements that you can ship out as soon as possible.

The problem is that most teams reach for these first, before looking at why the tests are slow in the first place. None of these actions make your test suite faster, even though the result is a shorter test run. The same bad practices stay scattered throughout the codebase, and there's only so much parallelization or scaling of your CI workers you can do before you hit a roadblock.

The order in which you apply these techniques matters. If 30 of your tests each spin up a full database and hit three external APIs, parallelizing them means you're now spinning up 30 databases concurrently and hitting those APIs 30 times at once. Now you've traded a slow test suite for a flaky one that eats up your CI runner's resources. Running subsets of tests has a similar trap. It works until a change breaks something the subset didn't cover, and no one notices until it hits production. The underlying flaw didn't disappear. It's still lurking under the covers.

Fix the tests first, then apply parallelization, caching, and other strategies to what's left. Working in that order will make each of these techniques compound. If you begin in reverse order, they’ll just paper over problems until the papering stops working. If you find yourself adding more parallelization or spending too much time figuring out which subset of tests to run effectively, it's a sign to stop and address the real issue instead of slapping another stopgap method on top.

How TestNod Helps You Stay Ahead of Slowdowns

All of this advice assumes someone is keeping track of the automated test suite over time. But if most teams are like the ones I’ve worked with over my career, the reality is that almost no one is paying attention. Tracking trends across weeks or months means either building internal tooling or manually pulling data from your CI provider. Most teams never get around to it, which is how we end up with 20-minute test suites in the first place.

That's exactly what I built TestNod to do. TestNod collects JUnit XML files from your CI runs and tracks their data for every test case across your build history. These files often contain a time attribute that tells you exactly how long it took a test case or test suite to run. Instead of looking at a single test run in isolation, TestNod shows you how your suite’s total execution time is trending over weeks and months. When your test suite begins to slow down, TestNod alerts you so you’ll see it long before it becomes a pain to deal with in your day-to-day workflow.

The data for tracking all of this information already exists in the JUnit XML files that most testing frameworks produce and most CI systems can store. TestNod makes that data visible and actionable without requiring you to build or maintain any custom tooling. Setup takes a few minutes on the most popular CI services, including GitHub Actions, CircleCI, and GitLab, and it works with any testing library or framework that outputs JUnit XML.

TestNod is not open to the public yet, but if you want to be one of the first to try it out, sign up for early access at https://testnod.com/.

Talk to the Machine, or Why We Chat with AI But Not Each Other

2026-04-21T06:07:00Z

Lisa Crispin recently dropped a comment under one of my posts.

“It seems funny to me that people are willing to turn things over to an agent, but they don’t want to collaborate with other humans to do a better job of building quality in.”

A little sad, if you think about it.

With the rise of AI coding assistants suddenly everyone is prompting. People spend hours iterating with ChatGPT or Claude. They refine their prompts, try different angles, patiently wait for the response, adjust, try again. They have entire conversations with the machine about architecture decisions, edge cases, test strategies.

And yet. Ask them to walk over to a colleague and have a five-minute conversation about the same thing? Too much effort. Too awkward. Takes too long. Don’t want to interrupt.

I find myself wondering, what’s the psychology here? The friction is lower, that is clear. But is the machine “safer” because it doesn’t judge? Because we stay in control of the conversation? Because asking a human might feel like admitting we don’t know something? I genuinely don’t have the answer. But the pattern is hard to ignore.

Here’s the thing though. AI doesn’t skip the conversation. It just delays it.

You can prompt your way to code. Lots of code. Fast code. But without talking to your teammates first, that code arrives without shared understanding. Nobody discussed what problem we’re actually solving. Nobody aligned on the approach. Nobody mentioned that other edge case, or that dependency, or that thing the customer said last week.

So the code lands. And then come the pull requests that miss the point. The bugs that could have been avoided. The incidents that escalate. The rework. The frustration. All the conversations you skipped? They happen anyway. Just later, angrier, and almost certainly more expensive.

This is where in my opinion the difference shows. Teams that talk first and then use AI tools to execute? They scale. The AI amplifies their shared understanding. It accelerates what they’ve already aligned on.

Teams that prompt instead of talking? They drown. In code nobody asked for. In PRs that go nowhere. In technical debt that piles up because nobody stopped to ask “wait, is this what we actually need?”

This echoes what Lisa Crispin and Janet Gregory have championed for years: the human side of building software – the talking, the collaborating, the shared understanding – isn’t overhead. It’s the work.

AI changes nothing about this. If anything, it makes it more obvious. The teams that were already good at communication? They’re getting better, faster. The teams that weren’t? They’re just producing more stuff that misses the mark. At higher speed.

So here’s my ask. Before you open that chat window with your AI assistant, try something radical. Talk to a human first. Ask a colleague what they think. Have the awkward conversation. Align on the problem before you generate the solution.

The AI will still be there when you’re done. And it’ll be a lot more useful once you actually know what you’re building.

No agent can save you from not talking to each other.

What STACKx Cybersecurity 2026 Made Me Rethink About AI, Testing, and My Own Engineering Work

2026-04-21T04:02:59Z

Last Friday I spent the day at Marina Bay Sands for STACKx Cybersecurity 2026 — GovTech Singapore’s annual gathering of security leaders, engineers, and public sector technologists.

I’ve worked across Quality Engineering and Software Engineering roles, with a focus on testing strategy, CI/CD automation, and recent years, AI-assisted engineering workflows. And part of my job right now is figuring out how to bring AI into our engineering workflows responsibly. So I went in thinking I’d pick up some security pointers I could bring back to the framework design.

What I didn’t expect was to walk out questioning whether I’ve even been pointing the framework in the right direction. Below are the things I’m still thinking about.

The Wrong Question Has Been Living Rent-Free in Our Roadmaps

The old question everyone asks: “Can we do the same work with fewer people?”

The better question: “With the same people, how much more can we now do?”

That second question sounds subtle, but it completely changes how you think about building an AI framework. I’ve been thinking about efficiency — how to speed up test creation, how to reduce review cycles, how to automate the boring stuff. And that’s fine, those are real problems. But the speakers were talking about something bigger: capability multiplication. Not doing the same job faster, but doing things you literally couldn’t do before.

Which means the AI framework I’m building shouldn’t just help engineers write tests quicker, it should let us catch categories of issues we’re currently blind to. That’s a different design goal, and honestly I hadn’t framed it that way until Friday.

From Pilot to Air Traffic Controller — Whether You’re Ready or Not

One of the more memorable frames: the shift every senior engineer needs to make right now is from being a pilot to being an air traffic controller. The pilot is a Solo Executor — deep expertise, full control, limited to what one person can personally fly. The air traffic controller does Supervisory Engineering — maintaining awareness across multiple simultaneous flows, calibrating when to intervene, trusting the system until there’s a reason not to.

The honest version of this for me is: I’ve been a pilot for most of my career, and I’m good at it. There’s real satisfaction in going deep into a problem, owning the outcome, verifying end-to-end. But I’m building a system now that will, eventually, run significantly without me in the loop. And I haven’t fully made peace with that yet.

The transition isn’t about working less or caring less. It’s about shifting where the expertise lives — from execution to the design and governance of execution. That’s actually harder. And it requires me to be honest about the habits I’m holding onto that are pilot habits, not air traffic controller habits.

The Rungs I Climbed Soon Don’t Exist Anymore

The speaker put up a redesigned career ladder and it stopped me cold — not because it was surprising, but because it named something I’ve been feeling without having the language for.

The bottom rungs are gone. The repetitive tasks that used to build expertise — writing hundreds of test cases by hand, grinding through manual reviews — are exactly the work AI is consuming. What’s left above demands a different kind of engineer: someone who can supervise agents, verify and challenge outputs at speed, think in systems, and exercise judgement with accountability. Strong fundamentals matter more now, not less — but they’re the floor, not the ceiling.

The part that really got me was the framing around workforce development. The speaker was blunt: we cannot prepare for an AI-native future with a pre-AI workforce model. Train people in fundamentals plus new instincts — to supervise, verify, challenge, and reason in systems. Deploy them broadly — across ops, engineering, governance, assurance, product security, and AI-enabled cyber. Not deskilled. Deliberately developed.

If AI generates the test cases now, where does the instinct come from? I can’t just onboard someone onto the AI harness and assume they’ll grow into a strong tester. I have to create the friction on purpose — make them challenge the AI output, articulate what’s missing, catch what it got confidently wrong. And I need to think about breadth too — the best QEs in an AI-native world won’t just be test specialists, they’ll need to reason across governance, assurance, and security contexts. The tool is only as good as the judgment sitting behind it. That judgment doesn’t come for free anymore.

AI Doesn’t Get to Own the Irreversible Decisions

One of the most clarifying moments of the afternoon was a speaker drawing a hard line through the types of decisions AI should and shouldn’t own. Some things AI should handle autonomously — pattern-matching, high-volume detection, tasks where speed matters more than nuance. Some things AI should inform, but a human should decide. And some things AI must stay completely out of.

This maps directly onto quality engineering decisions I make every day. AI can autonomously flag test failures and coverage gaps. It can recommend where to focus effort. But the final go/no-go call on a release? That stays human. Not because I don’t trust the system, but because trust without clear accountability is just abdication with extra steps. This kind of tiering needs to be explicitly written into the AI harness design from day one.

Communicate the Reasoning, Not Just the Answer

The last thing that stuck with me came from a session about decision-making under pressure — how leaders behave when information is incomplete and time is short. The speaker had a clean framework for anchoring a team in those moments, but the line I wrote down was the one that followed: “Communicate your reasoning, not just your decision.”

I do this badly. When I make a judgment call — on test coverage, on a risk escalation, on a go/no-go — I tend to state the conclusion and move on. I assume the reasoning is implied. But sharing the reasoning, especially when it’s uncertain, is how you build trust over time. It’s also, I’ve started to think, what separates senior-level contribution from staff-level contribution. Not better decisions — more visible thinking.

So what am I actually going to do about it?

I’ve been chewing on this all weekend. Here’s where my head is at:

First, I’m reframing my AI framework from “efficiency tool” to “capability multiplier.” The design questions change when you shift that lens. Example: it’s not just “how do we write tests faster” — it’s “what can we now test that we couldn’t before?”

Second, I’m adding AI-assisted first-pass review as a core pillar, not a nice-to-have. If our AI code reviewer can catch a replay attack vector in a nonce handling issue, I should be building something that gives my team’s reviewers that same kind of head start.

Third — and this is the harder one — I need to redesign how my team trains and works alongside AI. Not just “here’s the tool, go use it,” but structured ways of working where the human stays in the loop and understands what the AI is doing. Because the moment your team treats AI output as gospel without understanding it, you’ve got a different kind of quality problem.

That’s a mindset shift, and it starts with me.

Wrapping up

Look, I went to a cybersecurity conference expecting to take some security notes. Instead I came back rethinking the whole framing of what I’m building and why.

The conference theme was “Creating a Trusted Digital Future Together” and honestly, “together” is the word I keep thinking about. Not just cross-org collaboration — though that matters — but the human-AI partnership. The idea that the future isn’t AI replacing us or us ignoring AI, but us learning to work alongside it in ways that actually make both sides better.

I’m still processing. But I’m glad I went.

What STACKx Cybersecurity 2026 Made Me Rethink About AI, Testing, and My Own Engineering Work was originally published in Government Digital Products, Singapore on Medium, where people are continuing the conversation by highlighting and responding to this story.

blogs

« Silly Questions » Are Almost Never Silly

Hey Siri, is AI Fatigue a Thing?

Upgrade Cypress To TypeScript v6

Flaky Tests Are a Trust Problem: How to Diagnose, Quarantine, and Permanently Fix Unreliable Tests

The History of Contract Testing with Pact.io

The History of Pact.io

SmartBear Acquires Pact.io

But What Is Contract Testing, according to SmartBear Academy?

Consumer Driven Contract Testing

Provider Verification

The Pact Broker

CI / CD

What happens when you pair Playwright with something other than TypeScript?

Related Articles:

Benefits for using Playwright + TypeScript:

Related Topics on TJMaher.com:

STEC Deepdive: Module 3.4.0 - How to deliver bad news

Know your audience

Cross-team discussions

What if the feedback I need to give is really bad?

Person-to-person

Find the next step

Guest Blog: You don’t want Quality Engineers

You don’t want Quality Engineers

Foreword

Why Framing over the Behaviour

What Quality Professionals Actually Do

What I’ve Seen vs What I Hear About

What We’re Already Doing

Same Hymn Sheet, Different Pages

DSPy: Declaring Instead of Prompting

What Problem Does DSPy Solve?

The Core Abstractions

Script 1: The Baseline

What the Output Tells You

Script 2: One Word, New Behavior

What Changes in the Output

What Doesn’t Change

A Note on inspect_history

Where This Goes Next

SpecSync v5.4: Improved performance

IQL Practical #1 – Don’t Just Test the Output… Challenge the Reasoning.

T – Twist the Question (The Reframe)

R – Rewrite the Scenario (The Counterfactual)

A – Argue Back (The Pressure Test)

C – Check the Limits (The Edge Case)

E – Evidence Check (The Source Challenge)

Why TRACE Matters for Quality Engineers

AI Doesn’t Understand the World (yet)

Testival Meetup #79, Zagreb, Croatia

Feedback

Playwright vs Selenium in 2026: Benchmarks, Migration Paths, and the Real Decision Framework

Sprint & Release Reports: The Missing QA Practice That Drives Real Quality

Why Sprint/Release Reports Matter (More Than You Think)

What a Good QA Report Should Contain

How to Improve as a QA in Reporting

Conclusion

You’re not ready for Quality Engineering

You’re not ready for Quality Engineering

Why this matters

What engineering teams want

What engineering teams think

What can we do?

The whole talk

Software Testing Live 07

For the workshop (uploads due during workshop)

Backup version (working before workshop)

More

Why Your CI Test Suite Keeps Getting Slower

How Automated Test Suites Slow Over Time

Metrics That Actually Matter

What You Can Do About It

Deciding what “slow” is for you

Taking care of the outliers

Beware of AI-generated tests

Treat symptoms, but know why you're doing it

How TestNod Helps You Stay Ahead of Slowdowns

Talk to the Machine, or Why We Chat with AI But Not Each Other

What STACKx Cybersecurity 2026 Made Me Rethink About AI, Testing, and My Own Engineering Work