In the previous post we explored how to use the MATLAB build tool to run only impacted tests during local, iterative development cycles. In this post, we'll take that concept further and apply incremental testing in a continuous integration (CI) environment to speed up feedback and improve overall throughput.

Contents

Example Project: arithmetic

For this walkthrough, we'll use a simple example project, arithmetic, that contains:

Project Structure:

The toolbox folder holds the source code, while the tests folder contains tests for those functions.

Context

Whether (and when) you choose to run only impacted tests versus the full test suite depends on many factors, including:

This post does not prescribe a software qualification strategy. Instead, it shows an example of how to achieve incremental testing in CI using the build tool's test impact analysis.

For this example, we'll configure the build to run incremental tests on feature branches whenever a developer pushes changes. The same approach, however, can be adapted to other events such as pull requests, merges to main , or nightly builds.

I'll demonstrate this example using GitHub Actions, but this pattern is easily transferable to other CI systems like GitLab CI, Azure DevOps or Jenkins.

The Build File

The buildfile.m in the project defines a build plan with a test task that runs the project's tests and produces results.

function plan = buildfile
import matlab.buildtool.tasks.*

plan = buildplan(localfunctions);

plan("clean") = CleanTask;
plan("test") = TestTask(SourceFiles="toolbox",TestResults="results/test-results.xml");

plan.DefaultTasks="test";

end

Incremental testing is enabled by setting RunOnlyImpactedTests property or task argument to true on a TestTask instance. In this example, we will use the task argument at runtime, rather than baking it directly into the test task configuration.

As discussed in the previous post, the build tool stores task traces in the .buildtool cache folder. A task trace is a MATLAB release-specific record of a task's inputs, outputs, actions and arguments from its last successful run.

By caching and restoring these task traces (and outputs such as test results) appropriately in CI, we can effectively enable the build tool to detect changes and run only impacted tests in CI workflows as well.

CI Pipeline Configuration

The GitHub Actions workflow configuration is defined in .github/workflows/ci.yml:

name: MATLAB Build

on:
  push:
    branches:
      - 'feature/*'    # Only run on feature branches

  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      # Check out the repository
      - uses: actions/checkout@v4
      
      # Sets up MATLAB on a GitHub-hosted runner
      - name: Set up MATLAB
        uses: matlab-actions/setup-matlab@v2
        with:
          release: R2025a
          products: >
            MATLAB
            MATLAB_Test

      # Cache .buildtool and test results
      - name: Cache buildtool artifacts
        id: buildtool-cache
        uses: actions/cache@v4
        with:
            path: |
                .buildtool
                results
            key: ${{ runner.os }}-buildtool-cache-${{ github.ref_name }}

      # Run MATLAB build
      - name: Run build
        uses: matlab-actions/run-build@v2
        with:
          tasks: test(RunOnlyImpactedTests=1)
          build-options: -verbosity 3

The configuration has the following key steps:

First Run (No Cache)

Let's create a feature branch feature/cache under this blog's branch 2025-Test-Impact-Analysis-CI.

On the first pipeline run, there is no cache hit for key Linux-buildtool-cache-feature/cache.

All tests run because no prior traces exist:

The pipeline then creates and saves a cache for the feature/cache branch for future runs:

Second Run: Modify h_my_add.m

Let's make a change.

Next, checkout and switch to the feature/cache branch, then modify h_my_add.m, say, by adding an arguments block:

function out = h_my_add(in1, in2)
% This is a helper function to add 2 values

arguments 
     in1 {mustBeNumeric}
     in2 {mustBeNumeric}
end

out = in1 + in2;
end

Verify locally with:

>> buildtool test(RunOnlyImpactedTests=1) -verbosity 3

Output (excerpt) :

Only 8 out of 21 tests were selected to run—those impacted by the modified file. That's promising!

Now let's push the changes and inspect the CI pipeline.

We see a cache hit and it confirms the .buildtool and results folder were restored.

The build tool leverages the restored cache to run only impacted tests.

Third Run: Modify my_subtract.m

Make another change, commit, and push. The cache is restored again, and now only the tests impacted by both outstanding changes on the feature branch are executed:

Wrapping Up

There you have it! This workflow demonstrates how to combine:

... to achieve incremental testing in CI.

You can refine this pattern by adding things like:

Would you run incremental testing in your CI pipeline? At what point in your release cycle do you feel most comfortable running only impacted tests?

We envision continuing to invest in making incremental testing better across both local development and CI workflows. Your feedback is essential in helping us focus on the improvements that matter most to you. So, leave us your thoughts in the comments section below.

That’s a wrap (for now) on our series about Test Impact Analysis. Stay tuned — we have more software development topics in the pipeline, and we look forward to sharing them with you!

Get the MATLAB code

Published with MATLAB® R2025a

Alright! I'm back after a mid-series break to continue our talk about incremental testing.

In parts 1 and 2, we looked at interactive development workflows to help run the right set of tests at the right time. In this post, we'll look at how that same idea extends to automated builds using the MATLAB build tool, and specifically, how incremental testing enables faster local prequalification. The motivation remains the same – to enable frequent, high-quality integrations verified by automated builds, without incurring the overhead of running the full regression suite after every incremental change.

Running all the tests might be manageable at small scale but it quickly becomes prohibitively expensive for larger projects.

Contents

What Is Incremental Testing?

The MATLAB build tool supports incremental builds, which skips unnecessary work by tracking each build task's inputs, outputs, actions and arguments to determine if the task is up-to-date. If nothing has changed since its last successful run, the task is skipped.

Starting in R2025a, the build tool enhances this capability with incremental testing, available with MATLAB Test. While incremental build determines whether a task should rerun, incremental testing goes further – it uses test impact analysis to decide which tests need to run. This can significantly optimize test execution and speed up your builds.

With incremental testing enabled, the build tool analyzes the source and test code changes and runs only tests impacted by those changes.

Let's unpack how this works. We'll start with some basics.

Background

We'll use the same goal as in Part 2 – enhancing the AND perceptron to introduce a trainable bias term. The exact goal isn't critical; it's just a useful backdrop.

Defining the Build

A buildfile.m at the project root defines your MATLAB build.

This build file creates a plan that describes how to build your project – it defines a set of build tasks and their dependencies to establish a task graph to execute with a single buildtool command.

function plan = buildfile 
import matlab.buildtool.tasks.*

plan = buildplan(localfunctions);

plan("clean") = CleanTask;
plan("test") = TestTask("tests", SourceFiles="toolbox");

plan.DefaultTasks="test";

end

In this example, the build file includes:

• A test task to run tests.
• A clean task to delete task outputs and traces.

These tasks are created using the built-in task classes. Built-in task classes simplify defining common tasks like identifying code issues, building MEX binaries and testing. These classes are designed for reuse and generally support incremental build out of the box.

Our test task uses the matlab.buildtool.tasks.TestTask class, which runs tests using the MATLAB Unit Testing Framework.

Optimizing Performance with Incremental Build

Before diving into incremental testing, let's briefly look at how incremental build works.

When we first clone our repository, the build tool has no task traces.

So, when we run the test task, it runs because there are no traces yet, and runs all the tests defined by the task.

...

This is nice because it gives us confidence that the repository is in a known-good state before making any code changes.

The build tool creates a .buildtool cache folder in the project root folder to store the task traces.

On subsequent builds, if nothing has changed, the task is skipped.

Now, let's say we modify a source file like initializeWeights.m, and re-run the build.

Since this file is tracked via the SourceFiles property, one of the task inputs of the test task that the build tool tracks as part of up-to-date check, the tool recognizes the changes and re-runs the task.

Nice! But do we really need to run all the tests for that single source file change? Wouldn't it be better if the test task ran just the impacted tests. That's exactly what incremental testing in R2025a delivers.

Enabling Incremental Testing

So, how do we get the incremental testing behavior?

Incremental testing is powered by test impact analysis. To enable it, you can simply run the test task with the RunOnlyImpactedTests task argument.

>> buildtool test(RunOnlyImpactedTests=1)

Let's look at an example. Say we change initializeWeights.m and calculateWeightedSum.m as part of our perceptron enhancement, like we did in Part 2:

Before:

function weights = initializeWeights()
% Initialize weights randomly for two inputs and one bias
weights = rand(1,3) * 0.5;
end

After:

function [weights, bias] = initializeWeights(numInputs)
% Initialize weights and bias randomly
weights = rand(1, numInputs); % Random weights for each input
bias = rand() - 0.5; % Bias initialized to a small random value between -0.5 and 0.5
end

calculateWeightedSum.m

Before:

function output = calculateWeightedSum(weights, inputs)
% Compute the weighted sum
total_input = sum(weights .* [inputs, 1]); 

% Activation logic
if total_input > 0 
% step function threshold; activate if weighted sum is positive
   output = 1;
else
   output = 0;
end
end

After:

function output = calculateWeightedSum(weights, bias, inputs)
% Compute the weighted sum
total_input = sum(weights .* inputs) + bias;

% Activation logic
if total_input > 0
% step function threshold; activate if weighted sum is positive
   output = 1;
else
   output = 0;
end
end

When you set RunOnlyImpactedTests to true, the test task analyzes those changes and runs only the impacted tests. Look at that!

Behind the scenes, the build tool tracks changes to the source and test files of the test task created using the the built-in TestTask class (more on this later in the post). When RunOnlyImpactedTests is true, the test task doesn't just detect that the files changed – it determines which tests are impacted by those changes and runs only those.

Increasing the build verbosity to level 3 (Detailed) or higher reveals which changes are causing the task to rerun. In our example, we see that the SourceFiles property has changed because initializeWeights.m and calculateWeightedSum.m were modified.

As seen in Part 2, changes to just those two files do not meet the goal. The diagnostics will reflect this.

Let's fix the rest of the code base like we did in Part 2 to ensure we have all the necessary changes and rerun the build with incremental testing.

After some coding and a cup of coffee...

Success! The build passes and we ran just the related tests.

Our toy example had just 5 tests, but real-world projects can have hundreds or even thousands – so the potential for performance gain is substantial.

To avoid the need to specify the task argument every time, we can configure the test task at plan-creation time in your buildfile.m to always run only impacted tests by default.

Then at run time, we can specify just "test".

>> buildtool test

You can toggle the RunOnlyImpactedTests option at run time or create separate test tasks with and without incremental testing to match your workflow preferences.

A Closer Look at Impact-Based Testing

Impact-based testing involves two key concepts:

Change Detection

When you enable test impact analysis, the TestTask detects changes to source files, supporting files and tests as defined by the task's SourceFiles, SupportingFiles, and Tests properties. It also tracks test class folders and test super classes.

But what is the change window? … It's the changes since the last successful run.

A picture may help… it always goes a long way for me.

For example, let's suppose you have a successful build, b¹. After you make a change C1, and run the build, the build tool checks for changes in source, supporting, and test files since b¹. If C1 causes a regression and b² fails, the next build, b³ still uses b¹ as the reference point – because it's the last successful run. This ensures the integrity of your commits and code integration.

Only files that are added or modified are considered for impact analysis; removed files are not. If you're using MATLAB projects, removing a file triggers dependency analysis to help you identify which files are affected so you action appropriately and clean up cruft and dead code.

In summary, change detection:

• Detects files defined by the SourceFiles, SupportingFiles and Tests properties of the test task.
• Detects files that are either added or modified.
• Detects changes since the last successful task run.

Impact Analysis

Like the Find Tests (Part 1) feature in the Editor and Impacted Tests Since Last Commit option (Part 2) in the MATLAB test manager app, the TestTask also leverages dependency analysis to find impacted tests. It leverages it via the DependsOn test selector, which, in R2025a, enhanced the accuracy of test selection with the ability to select individual tests within a test file.

Interaction Between Incremental Build and Incremental Testing

Earlier, I mentioned that if a task's inputs, outputs, actions and arguments haven't changed since the last successful run, the build tool skips the task. The RunOnlyImpactedTests option is itself considered an input to the task, so incremental build applies to this property as well.

When you rerun the test task with the same RunOnlyImpactedTests value and the task has no other changes either, the build tool skips the task. Why? 'cuz of Incremental Build!

Quiz Time!

Suppose you change the TestResults task output property:

From:

plan("test") = TestTask("tests", SourceFiles = "toolbox", TestResults = "results/test-results.xml");

To:

plan("test") = TestTask("tests", SourceFiles = "toolbox", TestResults = "results/test-results.html");

Question: What tests will run if RunOnlyImpactedTests is set to true, assuming the above buildfile.m is the only change made in your project since the last successful run and no other changes?

Let us know what you think in the comments!

Extra Credit:

In buildfile.m, disable incremental build:

plan("test") = TestTask("tests", SourceFiles = "toolbox", TestResults = "results/test-results.xml");
plan("test").DisableIncremental = 1;

Then run:

>> buildtool test(RunOnlyImpactedTests=1)

Question: What tests do you expect to run? And why? Share your thoughts in the comments.

I will share the answers in the comments next week!

Summary

There you have it! Incremental testing with the MATLAB Build Tool and MATLAB Test can greatly improve the efficiency of automated builds, locally and in CI, by running only the tests impacted by recent changes. It complements interactive development and helps reduce test times without sacrificing coverage.

While static dependency analysis has limitations, incremental testing can deliver substantial value with some user awareness and discipline.

This post focused on local builds, but the same benefits apply to CI environment as well – we'll dive into that in the next part!

Would you consider using incremental testing in your workflows? What challenges do you foresee? Are there gaps in change detection or impact analysis that MathWorks could address to help you trust incremental testing more? We'd love to hear your thoughts – share them in the comments section below!

Get the MATLAB code

Published with MATLAB® R2025a

In Part 1, we looked at how the Find Tests button in the MATLAB Toolstrip can streamline your workflow by surfacing the relevant tests for the MATLAB file you are currently editing. While that approach works great for making changes to a single file, as development progresses and you prepare to commit a batch of changes, your focus shifts from a single file towards ensuring the integrity of those whole commits. And, a quick feedback loop is key to enabling frequent and high-quality commits before pushing to your remote branch.

Starting in R2025a, the MATLAB Test Manager app helps you do just that. It lets you manage a test suite that automatically detects which tests are impacted by changes since your last source control commit. This means you can avoid running the full (potentially time-consuming) test suite, while still catching any regressions early.

Let's look at how that works in practice.

We'll take the same example from Part 1: A Simple AND Perceptron

But before we dive into the example, let me take a quick moment to highlight something that is also new in MATLAB R2025a - the Source Control panel.

This new panel, easily accessible off the sidebar, automatically detects source control folders you are actively working on. That means you can quickly get oriented without needing to dig through menus. As you make changes, it gives you a clear view of modified and untracked files, and let's you make source control actions right from the panel. You can also jump into the Branch Manager from there to manage your branches with ease. Wanna know more? Check out Source Control Integration in MATLAB documentation.

Note: Source Control integration in MATLAB supports both Git and SVN systems, but for the purposes of this post, I'll be using Git.

Alright, back to our example...

Contents

Goal: Enhancing the Perceptron to Use Bias as a Parameter

In the previous post, we explored how the perceptron model classifies data by calculating a weighted sum of inputs and applying a threshold function. This approach utilizes a fixed bias term to adjust the decision boundary.

In that implementation, the bias term is set to a constant value of 1, as shown below:

function output = calculateWeightedSum(weights, inputs)
% Compute the weighted sum
total_input = sum(weights .* [inputs, 1]);
% Activation logic
if total_input > 0 % step function threshold; activate if weighted sum is positive
    output = 1; 
else
    output = 0;
end
end

This fixed bias term simplifies the model but limits its adaptability.

Solution: Trainable Bias Term

To improve the model's flexibility and potentially enhance its performance, we propose modifying the bias term to be trainable. This adjustment allows the bias to be learned alongside the weights during training, enabling the model to better fit the data. The updated code should represent the following equation:

$z = w * x + b$

where b is now a learnable bias parameter.

Let's explore how the MATLAB Test Manager can facilitate efficient implementation of this change through impact-based testing.

Utilizing MATLAB Test Manager to find and run impacted tests

We'll begin by examining what tests the test manager identifies before we make any changes. When you select All Tests in Current Project , it finds, well, all the tests in your MATLAB Project.

In R2025a, the app introduced a new option named Impacted Tests Since Last Commit.

The Source Control panel agrees!

So, let's get to work and start making the changes. We begin by creating a new branch feature/bias using Source Control > Branch Manager

Looks like we are in a good source control state here – right branch for the change.

To turn bias into a model parameter, we need to modify the functions that initialize the weights and calculate the weighted sum.

initializeWeights.m

Before:

function weights = initializeWeights()
% Initialize weights randomly for two inputs and one bias
weights = rand(1, 3) * 0.5;
end

After:

function [weights, bias] = initializeWeights(numInputs)
% Initialize weights and bias randomly
weights = rand(1, numInputs); % Random weights for each input
bias = rand() - 0.5; % Bias initialized to a small random value between -0.5 and 0.5
end

calculateWeightedSum.m

Before:

functionoutput = calculateWeightedSum(weights, inputs)
% Compute the weighted sum
total_input = sum(weights .* [inputs, 1]);
% Activation logic
if total_input > 0 % step function threshold; activate if weighted sum is positive
    output = 1;
else
    output = 0;
end
end

After:

functionoutput = calculateWeightedSum(weights, bias, inputs)
% Compute the weighted sum
total_input = sum(weights .* inputs) + bias;
% Activation logic
if total_input > 0 % step function threshold; activate if weighted sum is positive
    output = 1;
else
    output = 0;
end
end

OK, we made the desired changes. With those changes implemented, we'll now rely on MATLAB Test Manager to identify and run only the tests impacted by our modifications. This targeted testing approach helps maintain efficiency and ensures that we address any regressions introduced by the changes.

Let's select Impacted Tests Since Last Commit in the test manager to identify the relevant tests.

Uh-oh! All tests have failed.

Thankfully, failure diagnostics are provided alongside the results, offering helpful insights for troubleshooting.

Ah, right! We modified the function signatures of initializeWeights.m and calculateWeightedSum.m. Let's address this.

The first two failures are straightforward – they simply require us to update the tests corresponding to those two modified functions.

We re-run the tests in the MATLAB Test Manager.

Progress! The first two tests now pass. However, we still have two failures remaining.

Upon closer inspection, it seems the remaining issues are related to other source files, such as trainPerceptron.m and predict.m. These need to be updated to account for the new trainable bias parameter and ensure it is trained alongside the weights.

OK, we have updated those functions and their corresponding tests as well to address the regression and ensure we have a cohesive set of changes. We'll click Refresh tests in the test manager to get an updated list of impacted tests.

Good news - there are no additional impacted tests. The tests identified earlier are already covering the newly updated source files.

Great! All tests are passing now, giving us confidence that the changes are compatible with the rest of the codebase.

Ideally, we would add new tests to specifically target the new functionality. Even better, we could start with those new tests to drive our implementation. Depending on the complexity of the application we might also consider performing ad-hoc testing to further bolster the rigor of our qualification and ensure full confidence in the changes.

The test manager supports you throughout this process by automatically identifying and running only the tests impacted by the changes in each development iteration. Much like the Find Tests feature we discussed in Part 1, Impacted Tests Since Last Commit also leverages dependency analysis to find impacted tests.

An additional feature I find particularly useful in the test manager is its ability to track the history of test runs. This allows you to review previous test results, providing valuable insights as you iterate through your development process.

Once we've validated our changes, we can commit them using the Source Control panel:

We now have a new commit

After committing, the test manager indicates that no tests have been impacted since the last commit. That makes sense!

If further development is needed for the feature, we continue to implement changes in manageable increments and commit frequently. This approach ensures the integrity of each commit while leveraging the test impact analysis capability for efficient qualification.

Would you consider adopting this capability in your workflow? Are there any aspects of test impact analysis or the workflow itself that we could improve to facilitate a more seamless development cycle for you? We welcome your thoughts in the comments below!

What's Next...

So far, we have conducted cycles of interactive testing using the Find Tests button and MATLAB Test Manager capabilities. Our next step will be to explore how the MATLAB build tool can help automate and further enhance your development workflow. We'll look at that in the next post. We'll specifically examine how it can leverage the same foundational test impact analysis capability to enable incremental testing and faster local automated builds as part of your pre-qualification process.

Get the MATLAB code

Published with MATLAB® R2025a

Welcome back! As promised, I am here to kick things off with Part 1 of our blog series on impact-based test selection. Lezz go!

Imagine you are developing an application and writing code in your MATLAB development environment.

Code changes, even small ones, can introduce unintended consequences in complex software systems. Running an entire test suite to validate every change slows down feedback loops and reduces development efficiency. For new contributors, unfamiliarity with the codebase further amplifies this challenge, making it harder to identify which tests are relevant.

When you make code changes, your focus is usually on a specific file. You want to iterate swiftly and confidently without losing context. To streamline this just-in-time validation process, MATLAB Test R2025a integrates dependency-based test impact analysis directly into the MATLAB Editor. This makes it easier to find and run only the tests related to the file you are actively working on.

At the heart of this is the new Find Tests button on the MATLAB Toolstrip that shows up if you have the MATLAB Test toolbox. MATLAB easily analyzes dependencies for the open file and identifies relevant tests. These are automatically populated in the Test Browser, so you can run just those necessary tests without ever leaving the Editor or searching manually through your project.

Test impact analysis uses MATLAB dependency analysis. Tim Johns wrote a nice post on Dependency-based Test Selection that you may be interested in checking out as well. It also touches on the nuances and limitations of static analysis that one ought to keep in mind.

Contents

Example: Testing a Simple Perceptron

While impact-based testing shines brightest in large projects, let's take a simple example to see how it works in practice. Our example here is a simple neural network, perceptron , that creates a binary linear classification model that simulates the task of decision making. You don't need to fully understand the details of how it works; in fact, it's better if you don't. Like the scenario of a new contributor unfamiliar with the codebase, this lets you see how the Find Tests feature helps you identify which tests are relevant, without having to reverse-engineer everything first.

The project has some source code and tests. The main goal of a perceptron is to make accurate classifications. This is achieved through optimizing the weights of the perceptron so the model can be better trained. We start off by calculating the weighted sum and that is used to train the model and use the trained model for prediction.

Let's make a change

Say, you are interested in determining how changes to the initialization of weights affect the classifier's performance. So, you open the initializeWeights.m file in the MATLAB Editor.

You change the scaling factor from 0.1 to 0.5 and want to validate the change. Luckily the project has tests. In this case, there's only a handful of them, but you are not sure if all of them are relevant to this file and change, and how long it would take to run them all. You want your iterations to be quick. The new Find Tests button to the rescue!

When you click the Find Tests button, it prompts you to choose the folder where tests are located. It runs MATLAB dependency analysis and populates the Test Browser app with just those related tests.

Now you can interact with the Test Browser as you normally would. Click the Run Tests button to run the test suite and see how your changes impact the perceptron performance.

Hmm, changing the scaling factor to 0.5 didn't seem to have much impact. Perhaps that is because this is a simple AND perceptron. But that tighter, faster feedback loop gives you confidence to keep experimenting or moving forward with the change.

Stay Focused. Stay In Context.

When you're editing a file, that file is your primary focus. Everything else can be noise. Find Tests optimizes for that mental context by populating just the relevant tests in a clean Test Browser, reducing distraction and cognitive load.

You can also then enable coverage collection in the Test Browser to see which parts of your modified or new code are exercised by existing tests. This helps you identify and close coverage gaps before pushing your changes. (The Generate Tests button right next to Find Tests can also help here.)

Key Considerations

Test impact analysis is only as effective as our test coverage. If there are no tests or not enough tests in the project, there's little to analyze and benefit from. The richer the test suite, the better the outcome from test impact analysis. The better the test impact analysis, the better the confidence in making code changes.

Find Tests determines dependencies (a.k.a. impact) with static analysis, and like any static analysis technique, it has limitations. It's not perfect or exhaustive. Think of it as a performance optimization that benefits from user vigilance − trust but verify. It is still recommended to run your full test suite at key stages in your development and release cycle.

Tip: If you know of a test that needs to run due to an untracked dependency you can still manually add it via the Test Browser after you "find the tests".

In Summary...

You can find impacted tests manually, but by surfacing the relevant tests just at the right time, MATLAB Test's Find Tests reduces the cognitive load, time and effort required to validate changes—especially in large projects with extensive test suites—and provides immediate guidance for new contributors or when working in unfamiliar areas of the codebase. Ultimately, it helps maintain high confidence in code changes while accelerating development workflows.

How do you see this feature helping your day-to-day development workflows? What other tools or capabilities would you like to see integrated more tightly with Find Tests and the Editor to further assist you in completing your workflows without having to switch too many contexts? What else do you see valuable for a more rigorous test impact analysis?

Let us know in the comments!

Get the MATLAB code

Published with MATLAB® R2025a

One of the biggest hurdles?... Testing.

As applications grow, so does the burden of making sure they still work after every small change. Running all the tests every time you tweak a piece of code can be painfully inefficient. It drags down developer productivity, introduces bottlenecks in CI pipelines, and turns quick feedback loops into long, frustrating waits.

Imagine fixing a minor issue in a single function — only to sit there while an entire suite of unrelated tests run. Not only is it a waste of time, but most of those tests may have nothing to do with the change you just made. It’s easy to see how this adds up over time, leading to diminishing returns and growing developer frustration.

Waste from running unrelated tests is not limited to development iterations. It slows down CI throughput. While it is a good practice to run all the tests at a meaningful cadence to ensure the entire application is always in a deployable state, running them all for every incremental job can be an overkill and hinders CI efficiency. It is counterproductive and makes an organization arguably less agile.

We can be smarter. Let's see how!

R2025a release of MATLAB Test introduced a set of new features designed to ease some of these pain points — both during everyday development and when running automated builds locally or in CI.

We'll dive into these capabilities in this blog series. I am planning a 4-part series at the moment, but we'll add more if we get carried away!

• Part 1: We’ll start with local development workflow, looking at a feature that helps you make changes to your source code with more confidence.
• Part 2: We’ll talk about how to ensure commit integrity using MATLAB Test Manager.
• Part 3: We'll explore how the MATLAB Build Tool can help you leverage incremental testing for faster local builds as part of your pre-qualification workflow.
• Part 4: Finally, we’ll walk through a CI workflow, how you can take advantage of incremental testing and boost your CI throughput.

What’s your testing experience been like? Have you ever felt stuck waiting on test runs that didn’t need to happen? Or maybe you’ve found creative ways to speed up your workflows? We’d love to hear your stories — drop them in the comments below.

We’ll kick things off with Part 1 next week. Stay tuned!

Hello everyone, it has been a long time! I am excited to resurrect the conversation a bit here because we have a number of topics that are in the pipeline. Actually, there have always been plenty of topics in the ol' blog pipeline, but the good news now is that we actually have been able to sit down and write some of them.

Over the next little while Ajay Puvvala will be publishing a series of posts related to some exciting features we have in MATLAB around test selection. Super exciting stuff so be sure to keep an eye out for these.

While you wait, I also wanted to quickly plug a couple of GitHub repositories that we've published to help with your CI workflows.

Contents

The simplest config that works

First, if you want to just get up and going on your CI platform with the simplest of configs, take a look at the CI Configuration Examples repository. This shows a basic "Hello World!" example for GitHub® Actions, GitLab® CI/CD, Jenkins™, Azure® DevOps, and CircleCI®. You can go there right now, fork it, and immediately use cloud hosted runners on GitHub, Azure DevOps, or CircleCI to see the CI process in action. Note, GitHub makes it particularly easy because Actions are just built into GitHub itself.

(Slightly) more advanced configurations

Additionally, for those looking for more advanced configurations, check out the Advanced CI Configuration Examples repository. This includes more complex setups and best practices for integrating MATLAB with these CI/CD tools. In this repository you will currently find two workflows that are demonstrated on 4 CI platforms: GitHub Actions, Azure DevOps, Jenkins, and CircleCI.

Multi-platform Toolbox Creation

The first example goes through the build of a MATLAB toolbox across 3 platforms to produce and release a single .mltbx file. This workflow currently shows:

Multi-release deployment to integrate with Python®

The second example shows how you can leverage the MATLAB Compiler SDK™ to create a python package from this algorithm. It does this across releases and operating systems. Some highlights of this workflow:

It's great to be back! Stay tuned for more and we'd love to hear how you are doing and what you'd like to learn more about!

Get the MATLAB code

Published with MATLAB® R2025a

I could start with monoids, Bird-Meertens formalism, the parallelizaton contract, StarLisp's *map and reduce!! implementations and other esoteric topics when it comes to tracing the root of modern Big Data processing. This dates back to the 1980s and sometimes even earlier. Some of the debate on who did it first is a religious topic and often contested.

2004 - MapReduce: Simplified Data Processing on Large Clusters

The Google system scanned all data on Gmail and on the heels of the “how we store stuff” came a key whitepaper describing “how we analyze stuff”. Their paper on MapReduce: Simplified Data Processing on Large Clusters by Jeff Dean and Sanjay Ghemawat, shed light on how the massive amounts of data was being processed. While one can argue that the concepts for map-reduce pre-date the whitepaper, it is usually accepted that this represented the first accessible implementation for practitioners. A functional programming API designed for ease of use. In operation, this looked like:

It became clear that data being split or chunked could be replicated, stored, and distributed across a cluster. This data could then be analyzed in a distributed way without moving the data. Results of the analysis could be mapped and reduced to the user. In other words when the data is too big to move, you move the program to the data.

This approach was disruptive to the way algorithms are built. As the simplest example, if I ran a mean() operation over every chunk or slice of data distributed across a cluster, I may never get the right result.

To correctly compute a mean, I would need to return the sum of the samples on each chunk and the number of samples i.e. a tuple. This step would be the map() operation and subsequently I could reduce() these two numbers by adding all the totals and then dividing the result by the sum of the number of samples across the chunks.

In short, the structure and staging of my math operation changes at scale.

2014-Present

The development of medical imaging software is a meticulous process that plays a critical role in advancing healthcare diagnostics and treatment. The rapid pace of technological change requires continuous investment in innovation to stay competitive. Companies must constantly update their devices with the latest advancements, such as artificial intelligence for image analysis. To accomplish that, the lifecycle of such software development usually encompasses several stages, from conception and design to deployment and maintenance. Each phase presents unique challenges that developers and project managers must overcome to ensure the delivery of high-quality, reliable, and compliant software in the shortest amount of time.

The Power of a Common Platform

To navigate these complexities effectively, a strategic and unified approach is essential. A common platform for medical imaging software development emerges as the key solution, offering a connected ecosystem where all stages of the lifecycle are seamlessly integrated. This enables efficient collaboration among diverse teams – software developers, verification engineers, regulatory specialists, and more – streamlining the entire process.

MATLAB is one such platform that has an integrated workflow for tying all the stages of the software development lifecycle together. Maintaining a digital thread throughout the development lifecycle is one of the ways to significantly shorten the software development timeline.

Figure 1: Bring all the software development lifecycle stages to one platform

Let's explore how a common platform tackles the specific challenges encountered throughout the development lifecycle:

1. Conceptualization and Planning: Medical imaging software is usually focused on three areas: image formation and reconstruction, pre-processing, or analysis algorithms. Defining clear and achievable objectives is necessary to clearly state the current capability being reformed or new functionality being added. Given the complexity of medical imaging needs and the diversity of clinical environments, setting realistic goals can be challenging. Defining requirements of the new feature - considering integration with the existing system and/or redefining the requirements of the existing system due to the new feature to ensure integrity - often becomes a challenging problem.

Solution:  Starting with conceptualization, MATLAB offers solutions for requirements analysis and software architecture, which help define clear objectives and deliverables for the new feature. These tools integrate with various third-party requirements and system engineering tools so that the software development can also be linked to the entire system development workflows (electrical, mechanical, fluids, etc.).

2. Design and Prototyping: During the design phase, developers must create a design concept that balances advanced imaging functionality with user-friendly interfaces. Complex features should not compromise the ease of use for healthcare professionals. For that reason, several iterations usually happen during the prototyping phase to figure out the best possible version. This prototyping and iterative feedback from the end-users phase can sometimes be long and impact the overall product timeline.

Solution: MATLAB offers an extensive suite of image processing, signal processing, analysis, and AI capabilities that simplify the creation of complex algorithms. With the high-level language and interactive environment, developers can quickly prototype and iterate on sophisticated image formation, pre-processing, and analysis algorithms, reducing development time and increasing efficiency.

3. Testing, Verification and Validation: Medical imaging software must be rigorously tested to ensure accuracy and reliability. Simulating the full range of clinical scenarios can be difficult. Again, with regulatory compliance in consideration, there is no shortcut for thorough testing. It is more often required to collaborate with clinical partners to conduct thorough testing, including unit, integration, system, and acceptance testing – which becomes a long iterative process.

Solution: Once the prototypes are developed, testing, verification, and validation are integrated into the prototype development process. The requirements and system architectures are directly tied to the prototype code, which extends to all the testing, verification, and validation suites. MATLAB also allows for the automation of the testing processes with continuous integration workflows, leading to more reliable and repeatable test outcomes. You can also develop Web Apps for sharing and testing the algorithms with clinical collaborators.

4. Deployment and Integration: Integrating new software into existing healthcare IT systems such as RIS or PACS without disrupting clinical workflows is a common challenge. Sometimes the software needs to be deployed on edge systems; other times, the algorithms live on an enterprise system such as a cloud or hospital server. Using established standards like DICOM and HL7 to ensure compatibility is often necessary, and there is a need for comprehensive integration and system testing to ensure the functionality of the overall system is not compromised.

Solution: MATLAB supports interoperability with other programming languages and technologies, making it easier to integrate MATLAB-based algorithms into the healthcare IT infrastructures. The ability to compile MATLAB code into standalone applications or software components also facilitates integration with various systems. MATLAB can generate standalone C/C++/CUDA/HDL code for integration into various edge systems and compile into various libraries, e.g., Python, .NET, Java, etc., for integrating with enterprise systems. MathWorks fully supports cloud integration and integration with PACS databases while ensuring DICOM compatibility. All the deployment steps are usually automated—sometimes just a click of a button, which significantly reduces the overall development time. The requirements, test suites, and prototype code are all tied to the deployed code, making the testing and validation much faster while ensuring true code quality.

5. Regulatory Compliance: Compliance with regulations such as FDA 21 CFR Part 11, IEC 62304/IEC 82304, or the MDR is mandatory for bringing medical imaging software to the market. Keeping up with changing regulations and ensuring compliance can be overwhelming. It is mandatory for companies to implement a quality management system (QMS) that is designed to meet regulatory requirements throughout the software development lifecycle. This is also a necessary process, and sometimes a software development lifecycle can involve several iterations between the development, verification, and software teams. Since in this phase all the product teams get involved, the time taken to converge is often very long.

Solution : MATLAB supports compliance with regulations (FDA/MDR)  and industry standards(IEC 62304) by providing solutions for tool validation, ,traceability from requirements to code, verification, and test report generation for technical files. The MATLAB environment supports the creation of clear and comprehensive documentation that can save significant amount of time and effort for regulatory submission and audits. The digital thread enables tracking any changes to the artifacts, tracing relationships between artifacts, identifying outdated results, performing incremental updates of results, and many more capabilities which greatly reduce the regulatory burden. Every single line of code that becomes part of the medical device can be traced back to the original requirements, architecture, prototype code, and test cases; and other metrics such as code coverage, etc., are also reported.

6. Maintenance and Updates: After deployment and market release, the software must be maintained, updated, and scaled to meet evolving clinical needs and technological advancements. Companies are required to develop a long-term maintenance plan that includes regular updates, patches, and technical support. Designing the software with scalability in mind to accommodate future growth and changes adds to the total development timeline.

Solution: MATLAB's modular structure allows for easy updates and scalability. Developers can modify or extend functionality without overhauling the entire system. MATLAB's compatibility with previous versions also ensures a smoother transition when upgrading software components.

Benefits Beyond Efficiency

Adopting a common platform approach extends beyond just efficiency gains. It fosters collaboration, enhances transparency, and ensures data integrity throughout the development process. This ultimately leads to higher-quality software, faster time-to-market, and improved patient care.

The Path Forward

As the medical imaging landscape continues to evolve, a common platform becomes increasingly essential for navigating the complexities of software development. By embracing this unified approach, companies can unlock their full potential, bringing innovative and reliable solutions to the healthcare industry while ensuring compliance and patient safety.

1. An introduction to MATLAB Test and the Test Manager
2. The use of advanced coverage metrics
3. Measuring and monitoring code quality
4. Equivalence testing.

• runtests(__,DependsOn=val)
• testsuite(__,DependsOn=val)
• matlab.unittest.TestSuite.fromX(__,DependsOn=val) where X can be Class, File, Folder, Method, Name, Package, Project, or Requirements.
• selectIf(suite,DependsOn=Val) where suite is a matlab.unittest.TestSuite array.

1. An introduction to MATLAB Test and the Test Manager
2. The use of advanced coverage metrics
3. Measuring and monitoring code quality.

What is equivalence testing and why do I need it?

Basic syntax

C/C++ code

1. Our test class inherits from matlabtest.coder.TestCase rather than the usual matlab.unittest.TestCase.
2. The new build method builds the C binary that we will execute as part of the test. We specify the name of the entry point function we want to build (myAdd) and some default inputs. These "compile-time" inputs are required for code generation and are similar to codegen’s ­-args flag. It’s an optional argument as it’s possible to write functions with no input arguments.
3. The new execute method executes the C binary using the default inputs. You can also specify different "run-time" inputs here if you so wish.
4. The verifyExecutionMatchesMATLAB method does what it says – it produces a verification failure if the output from the C code does not match that of MATLAB.

.NET, Java, Python

Reusing unit tests for equivalence testing

Overview

Equivalence test for C code

1. Inherit from matlabtest.coder.TestCase in addition to tMATLABTests so that we have access to the equivalence testing functionality that we need.
2. Build the code – it’s most efficient if we do this once for the test class rather than for each test point individually. We can use a TestClassSetup block to do this and store the result as a property. There’s an additional catch that the adjacency input is not of fixed size. We need to tell MATLAB Coder about this using the coder.typeof function.
3. Execute the code for the given inputs.
4. Compare the results to MATLAB.

Equivalence test for Python

1. Inherit from matlabtest.compiler.TestCase (note compiler, not coder!) in addition to tMATLABTests so that we have access to the equivalence testing functionality that we need.
2. Build the code – it’s most efficient if we do this once for the test class rather than for each test point individually. We can use a TestClassSetup block to do this and store the result as a property.
3. Execute the code for the given inputs.
4. Compare the results to MATLAB.

Conclusion

Code listing

shortestPath

tMATLABTests