Prerequisites

• An Octopus Cloud account. If you don't have one, you can sign up for a free trial.
• The Octopus AI Assistant Chrome extension. You can install it from the Chrome Web Store.

:::div{.hint} The Octopus AI Assistant will work with an on-premises Octopus instance, but it requires more configuration. The cloud-hosted version of Octopus doesn't need extra configuration. This means the cloud-hosted version is the easiest way to get started. :::

Creating the project

Teams that have adopted GitOps workflows with Argo CD can use Octopus to commit changes to a Git repository to promote changes through environments. The Update Argo CD Application Manifests step in Octopus commits changes to the files referenced by an Argo CD Application.

Paste the following prompt into the Octopus AI Assistant and run it to create a sample project using the Update Argo CD Application Manifest step:

Create an Argo CD Manifest Update project called "16. Argo CD Manifest Update" with the slug "argo-cd-octopub-manifest" using a Git Connection called "Mock" with a random username and the allowed repository "https://mockgit.octopusdemos.com/*"

This prompt creates an example project called 16. Argo CD Manifest Update with the project slug argo-cd-octopub-manifest. The slug is important because it links an Argo CD Application to an Octopus project.

We then create a Git Credentials defining the credentials required to interact with a Git repository. In this example, we are using a mocked Git repository hosted at https://mockgit.octopusdemos.com. This Git repository lets us use Argo CD steps in Octopus without providing credentials or creating a repository on a platform like GitHub.

Behind the scenes, we also create a mock Argo CD Instance called Mocked Argo CD Instance:

Typically, to create an Argo CD Instance, you must install the Octopus Argo CD Gateway in the Kubernetes cluster where Argo CD is running. This gateway then registers Argo CD Applications with Octopus and monitors the cluster for any changes.

For this demonstration, however, we register a mock Argo CD Instance and several mock Argo CD Applications. The mock Applications contain the information required for Octopus to modify the correct files in the mock Git repository during a deployment, without requiring an Argo CD cluster.

:::div{.warning} Some features of the mocked Argo CD Instance will always report errors or warnings. For example, the Gateway Connectivity tab will always report an error, because there was never a real Argo CD cluster to connect to. The manifests deployed by Octopus will always report Git drift because the Argo CD Applications are never updated to reflect the changes. And any attempt to sync Applications in Argo CD as part of the Update Argo CD Application Manifests step will fail.

These errors can be ignored or the features disabled without preventing Octopus from completing a deployment. :::

The sample step configuration

The Repository URL setting in the Update Argo CD Application Manifests step defines the Git repository that Octopus will commit to as part of the deployment. The value https://mockgit.octopusdemos.com/repo/argocd is matched to the Mock Git Credentials, providing the step with the credentials required to commit to the repository.

The Path setting, set to octopub-manifest/template/octopub.yml, defines the template file that will be read, have any binding syntax replaced, and persisted to the path defined in the Argo CD Application linked to the project and environment (how these Applications are linked is described later).

To see the contents of these files, check out the mock Git repository:

git clone https://somerandomusername@mockgit.octopusdemos.com/repo/argocd

:::div{.hint} The mocked Git repository accepts literally any username. However, commits made by unrecognized usernames are ignored. The Mock Git Connection created by the AI Assistant has unique and recognized credentials that allow Octopus to persist commits. However, the repository contents are reset periodically, so all commits are eventually reverted.

The contents of the Git repository cloned with the credentials above do not include the commits made by Octopus, as the mock Git server treats them as two separate repositories. :::

View the contents of the sample Argo CD Application:

cat argocd/octopub-manifest/octopub-development.yml

This is the example Argo CD Application manifest.

Note the repoURL field matches the Repository URL setting. The path field, set to octopub-manifest/application/development, specifies the location in the Git repository where Argo CD finds the manifest files to apply to the cluster.

Also note the annotations argo.octopus.com/project.<application name> and argo.octopus.com/environment.<application name>. These annotations link an Argo CD Application to an Octopus project and environment. The step references only Argo CD Applications that match the project's and environment's slugs. The prompt to create the project specified the slug argo-cd-octopub-manifest because this value is hard-coded in the example YAML:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: octopub-manifest-parent-development
  namespace: argocd
  annotations:
    argo.octopus.com/project.octopub-manifest-parent-development: "argo-cd-octopub-manifest"
    argo.octopus.com/environment.octopub-manifest-parent-development: "development"
spec:
  project: default
  sources:
    - name: octopub-manifest-parent-development
      repoURL: https://mockgit.octopus.com/repo/argocd
      # This is the destination folder where the template manifest files are placed
      path: "octopub-manifest/application/development"
      targetRevision: main

  destination:
    server: https://kubernetes.default.svc
    namespace: octopub-manifest-parent-development
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Display the octopub.yml file in the directory specified by the path field:

cat argocd/octopub-manifest/application/development/octopub.yml

This is the file applied by Argo CD:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: octopub-manifest-development
  namespace: argocd
spec:
  project: default
  sources:
    - name: octopub-development
      repoURL: https://mockgit.octopus.com/repo/argocd
      path: "octopub/octopub-frontend"
      targetRevision: main
      helm:
        values: |
          image:
            repository: ghcrfacade-a6awccayfpcpg4cg.eastus-01.azurewebsites.net/octopussolutionsengineering/octopub-frontend
            tag: latest
          mockBackend: true
          overrideTheme: blue

  destination:
    server: https://kubernetes.default.svc
    namespace: octopub-manifest-development
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Now display the template file defined by Path field on the Octopus step:

cat argocd/octopub-manifest/template/octopub.yml

Note that it includes the binding syntax #{Octopus.Environment.Name | ToLower} and #{Project.Frontend.Theme}:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: octopub-manifest-#{Octopus.Environment.Name | ToLower}
  namespace: argocd
spec:
  project: default
  sources:
    - name: octopub-manifest-#{Octopus.Environment.Name | ToLower}
      repoURL: https://mockgit.octopus.com/repo/argocd
      path: "octopub/octopub-frontend"
      targetRevision: main
      helm:
        # Octopus will replace the value for overrideTheme during deployment
        values: |
          image:
            repository: ghcrfacade-a6awccayfpcpg4cg.eastus-01.azurewebsites.net/octopussolutionsengineering/octopub-frontend
            tag: latest
          mockBackend: true
          overrideTheme: #{Project.Frontend.Theme}

  destination:
    server: https://kubernetes.default.svc
    namespace: octopub-manifest-#{Octopus.Environment.Name | ToLower}
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

:::div{.hint} The Project.Frontend.Theme variable is defined on the project. The Octopus.Environment.Name variable is provided as a system variable. :::

:::div{.hint} The pipe in the binding syntax #{Octopus.Environment.Name | ToLower} implements the ToLower filter. :::

The purpose of the Update Argo CD Application Manifests step is to:

• Read the template file (argocd/octopub-manifest/template/octopub.yml)
• Replace any binding syntax with the value of the associated Octopus variable (Project.Frontend.Theme and Octopus.Environment.Name)
• Commit the new file to the file of the same name (argocd/octopub-manifest/application/development/octopub.yml) referenced by the Argo CD Application linked via the argo.octopus.com/project.<project slug> and argo.octopus.com/environment.<project slug> annotations

This allows Octopus to inject environment-specific configuration into the files referenced by an Argo CD Application as a deployment progresses through environments:

:::div{.hint}

The Argo CD Application linked to a project and environment is not modified by Octopus. Only the files referenced by the Application are modified. Typically, these files will be plain Kubernetes manifests.

However, using the Argo CD App of Apps pattern, it is possible to modify a child Application referenced by a parent Application. This is the pattern used in this post.

:::

Performing a deployment

Create a new release for the project and deploy it to the Development environment. Note that Octopus commits a change reflecting the new value of the Project.Frontend.Theme and Octopus.Environment.Name variables replaced in the template YAML file:

As you promote the deployment through environments, each new environment will inject a unique value for the Project.Frontend.Theme and Octopus.Environment.Name variables into the child Argo CD Application manifest.

In a production scenario, Argo CD will detect the changes and apply them to the Kubernetes cluster. Of course, in this example, there was no Kubernetes cluster, since everything is mocked, but you can still observe commits being made to the Git repository.

What just happened?

You created a sample project with:

• A mock Argo CD Instance and a number of mock Applications
• A mock Git Credentials pointing to a mock Git server
• An Update Argo CD Application Manifests step that commits a processed template file back to the mock Git repo

It drops. Not because they forgot how to write code, but because the organization around them has a different risk tolerance. That's the part of the AI productivity conversation we're missing. We keep talking about developer velocity, but rarely talk about compliance velocity. But if you want to increase one, you have to increase the other.

It's not a code-writing problem

The constraint on large software teams hasn't been how fast engineers can write code. Most engineers could ship a weekend project to production in an afternoon. Put those same engineers inside a mid-sized or large company, and the same change takes days or weeks. That's not because the tooling is worse, but because the risk and compliance process around every change is doing exactly what it was built to do.

That's why an initiative that seeks to "generate changes faster with AI" fails to solve our problem. It was already possible to write code quickly; what we struggled with was managing the risk of shipping changes.

Every action has a reaction

This is where physics is a useful metaphor. For every increase in the volume of change, especially in a risk-averse environment, there's an equal and opposite reaction from the organization's risk and compliance functions.

Here's an example. A team is making 100 changes a day. They introduce AI and increase this to 200 changes per day, but production starts falling over twice as often, even though the change failure rate is the same. That causes risk and compliance to take an interest and introduce processes and policies to reduce the failures.

Within a few weeks, the team is back to 100 changes a day. Half of them are AI-authored, but there's no productivity gain to show for it. The system has found its new equilibrium, and it looks a lot like the old one, except for all the new processes that compliance wrapped around it.

There's a second version of this reaction, and it's less about infrastructure and more about experience. Small teams build up deep, shared context about the problem they're solving and the customer they're solving it for. Most software has many authors but one user at a time living through the whole journey. As a team grows, holding that cohesion gets harder, but at least the humans on the team still talk to each other.

Now give every one of those engineers their own AI coding assistant. Each engineer becomes far more productive individually, but each assistant has even less shared context than the humans did. The result is the same pattern: a high volume of change, but a more disjointed product experience, with the end-to-end story getting lost.

The equal-and-opposite reaction to that fragmentation is usually a heavy-handed swing back toward centralization. We saw that a decade ago, when "let every team choose its own tools" gave way to standardized platforms, because nobody could move between teams without relearning their entire stack.

These reactions are rational. They are the result of a system correcting for risk that increases faster than anyone can account for.

The one-way ratchet

There's something even more crucial than the equilibrium problem. Compliance only ever ratchets up; it never ratchets down.

Once a new rule shows up, like a mandatory review step, an extra test suite, or a new sign-off, it rarely gets removed, especially once it's on a regulator's radar. Velocity can go up and down as the team changes. Compliance doesn't work that way. It accumulates.

Imagine a team gets excited about AI and starts generating twenty changes a day instead of one per engineer. Production instability goes up. A data leak that used to happen once a year now happens twenty times. New compliance requirements get bolted on to stop the bleeding. Eventually, the team realizes the strategy isn't working and reverts to its old habits, but the compliance burden doesn't go away with them. They're now doing less, with more overhead, than when they started.

:::figure

:img{ src="/blog/img/the-compliance-ratchet/equal-and-opposite-reaction-diagram.png" alt="Diagram shows change volume increasing, then compliance overhead ratcheting up in response. This results in change volume returning to a lower volume, while the compliance overhead remains." loading="lazy" }

::figcaption[The compliance ratchet in action.]

:::

This is also how bad global solutions get applied to local problems. If one engineer had production access they shouldn't have had and made a mistake, the right fix is local: work out why they had that access and fix it. The wrong fix (but the one organizations reach for when they're not being deliberate) is global: nobody gets production access, ever, for anything. It solves the immediate problem and creates a dozen new ones.

Sometimes it goes even further than the company. A handful of businesses misuse a technology, a regulator responds with a blanket rule, and now everyone on the internet has to click through a cookie banner. That's what happens when a few actors are irresponsible with something new, whether that's self-driving cars, AI, or otherwise. Regulators aren't looking for a reason to step in, but companies force them to through irresponsible decisions.

Ship safer, not just faster

If your company has a new initiative to boost AI-driven productivity, here's the thing worth saying out loud before it starts: We already know risk and compliance will find equilibrium with whatever volume of change you produce. So plan for that from day one.

If every engineer on the team is using AI purely to generate more code, faster, you will accelerate the compliance ratchet. The healthier split is roughly half and half. Half the team should focus on how to make changes faster, and the other half on how to make those changes safer, more compliant, and less risky, with AI doing some of that work, too.

That second half of the work is genuinely interesting, and it's underrated. AI is well-suited to reasoning about risk, not just generating change. It could review a pull request and decide whether a one-line CSS tweak needs a human reviewer at all, while flagging any change that touches the payments pipeline for real scrutiny. It could review the build and decide which of the three hours of automated UI tests are relevant to a README update, rather than running the whole suite, which is what the pipeline does.

None of that reduces the volume of change getting shipped. It reduces the risk associated with it, which is what's actually been holding teams back.

The foundation of every software business is customer trust, and trust is not built by shipping more. If AI makes you ship faster without making you ship safer, you're not getting more productive, you're just winding the compliance ratchet a little tighter, and that one doesn't wind back.

If so, we're excited to share a feature we've been incubating — Feature Flags in Octopus.

Feature flags let teams control when and where new features are enabled, without needing to redeploy. You can experiment, test safely, roll back instantly, and ship continuously with confidence.

With Octopus Feature Flags you can:

• Toggle features on or off instantly — no redeployment required.
• Progressively roll out changes — deliver an upgrade to a single environment, to 10% of your tenants, or to a specific cohort of users.
• Develop on your main branch, keeping unfinished features safely hidden behind flags.

Built on OpenFeature

OpenFeature is an open standard that provides a vendor-agnostic, community-driven API for feature flagging that works with your favorite feature flag management tool or in-house solution.

When we came to build Octopus Feature Flags, we chose OpenFeature as the client SDK.

This means you get a robust, battle-tested SDK designed by the best minds in the feature flag business. More importantly, it means no vendor lock-in. If at some point you want to switch feature flag providers (we hope you never do), it's a single line of code to change the registered provider. You can even register multiple providers. And we get to contribute to a thriving project.

As OpenFeature offers SDKs for all popular programming languages, we only need to create an Octopus OpenFeature provider for each language.

Here's an example of configuring OpenFeature to use the Octopus Provider and evaluate a flag:

// configure OpenFeature to use Octopus as the feature flag provider
OpenFeature.setProvider(new OctopusFeatureProvider({ clientIdentifier: 'my-client-id' }));
const featureFlags = OpenFeature.getClient(); 

// evaluate a feature flag
const withDarkMode = await featureFlags.getBooleanValue('dark-mode', false); 

if (withDarkMode) {
    // enable dark mode option
} else {
    // light mode only 
}

Built into Octopus for seamless Continuous Delivery

Octopus Feature Flags live in Octopus projects. They're not a stand-alone product — they integrate deeply with the Octopus features you already use, like environments, tenants, and releases, and they're designed for releasing changes.

Environment targeting

Deliver a different experience in each environment. Each environment has its own feature flag settings, so you can turn a feature on in Development and Test for your team to trial, while it stays safely off in Production.

:::figure :::

Tenant rollout

Deliver an upgrade to 10% of your tenants, and dial it up from there. You can enable a flag for specific tenants, for tenants matching a tenant tag, or for a percentage of tenants. You can even exclude specific tenants; maybe your tenants represent your customers, and there are select customers you'd prefer not to include as early adopters.

:::figure :::

Minimum version targeting

Because Octopus knows which versions of your project are deployed to each environment, you can configure a minimum version for a feature flag. The flag is enabled only after that release version (or a later one) has been deployed to the environment.

:::figure :::

Client rollout and Segments

Release a feature to 5% of your users, or only to a chosen cohort — your staff, a geographic region, or a license type. Octopus Feature Flags support enabling a change for a percentage of your users (client rollout), or for a specific cohort via Segments.

:::figure :::

Segments are built on top of OpenFeature's Evaluation Context concept. This lets your application pass context (for example information about the current user like geography, or license type) that feature flag providers, Octopus in this case, can use when evaluating a flag.

The following code sample demonstrates adding context values from your application code:

// values can be added at one of three different levels: global, client, or invocation

// add a value to the global context
// an example of a global context value might be if you have multiple sites, for different geographic regions
OpenFeature.setContext({ site: 'octopetshop.com.au' });

// add a value to the client context
// an example of values typically added to the client context are those associated with a specific web request, such as the user details
const client = OpenFeature.getClient();
client.setContext({ 
    email: webRequest.user.email,
    licenseType: webRequest.user.license.type 
    });

// add a value to the invocation context
// the invocation context may be used to add values specific to the flag being evaluated
// for example a pet shop may have a feature specifically for dogs, and add a context 
// value to indicate if dog food has been added to a shopping cart 
const context: EvaluationContext = {
  cartContainsDogFood: 'true',
};

const flagValue = await client.getBooleanValue('dog-profile', false, context);

Read more about how Segments are evaluated in our docs.

Supported languages

We currently have OpenFeature providers for the following languages:

• TypeScript/JavaScript (web client)
• .NET (server)
• Java (server)

If you're interested in using Octopus Feature Flags, and your language isn't currently supported, please register your interest below and tell us which language you need.

Try Octopus Feature Flags

Octopus Feature Flags are now available to Octopus Cloud customers as a Public Preview. To get access, register your interest.

Feature Flags are available in all license tiers, including the Free tier. They're currently available for Octopus Cloud only; we plan to make them available to self-hosted Octopus Server customers soon.

We've been releasing features in Octopus Deploy using Octopus Feature Flags for the past 12 months, and they've improved our ability to release progressively, and to roll back instantly, meaning our customers — that's you — experience a more stable product than ever. We would love to help you do the same for your customers.

Happy deployments!

We've migrated target tags to be managed with tag sets, which means you can now create, edit, and delete tags on your deployment targets the same way you manage tag sets elsewhere in Octopus.

What this means for those already using target tags

Target tags still work the same way for filtering deployments and scoping steps to specific targets but they're now backed by the tag set system, giving you improved tag maintenance. You can now:

• Rename a tag when your naming conventions change
• Use SingleSelect or MultiSelect tag set types to enforce data integrity.
• Improve the organization of target tags by splitting these out into different sets.

:::figure :::

Use cases

Organize and provide context - for teams running multi-cloud infrastructure, it's common to need both the provider and the workload type when targeting deployments. For example:

• Cloud Provider (SingleSelect): aws, azure, gcp, on-premises
• Workload (MultiSelect): containerized, vm, serverless

Deploy to specific tags - trigger a runbook that restarts services only on targets tagged containerized, leaving VM-based workloads alone during an incident.

Exclude specific tags - exclude targets that are currently in use to ensure deployments don't result in downtime during high traffic periods

Deploy to tags across multiple tag sets - Deploy to targets tagged production (Tier) OR regulated (Compliance) to make sure a critical security patch reaches everything that's either customer-facing or subject to compliance requirements.

Getting started

If you're already using target tags, they've already been migrated to a new tag set 'Default Target Tags'. Head to Library > Tag Sets to see them and start managing them alongside your other tag sets. If you're setting up target tags for the first time, take a moment to think about what dimensions matter to your infrastructure; role, cloud provider, team ownership, compliance scope, and model those as separate tag sets rather than trying to pack everything into one.

Learn more

For guidance on designing tag sets, check out our documentation on tag set best practices.

For more information on deploying to, or excluding tags read more our docs.

Happy deployments!

Octopus comes in two flavours: Octopus Cloud, where we host and manage everything for you, or Octopus Server, where you install and run it on your own infrastructure.

Running an outdated Octopus Server is a quiet risk. This post covers why staying current matters, and why Octopus Cloud is worth considering if you'd rather focus on shipping software than maintaining the tool that ships it.

Below, I cover what Octopus updates actually contain, because security patches, bug fixes and more, matter just as much as new features.

Performance improvements

Performance is key not only for our customers but also for us. Every time we bake a new build internally, we test it to ensure it's not worse than the previous version. We run tests against our software, and our staff get to try it out first before it propagates upwards.

You might be asking yourself, "What does this look like?" Bob Walker, Field CTO at Octopus, covers this in a great talk, available on YouTube. The talk covers how we ship changes, who gets to test them, and when these changes land for our self-hosted customers.

If you don't already know, Octopus Cloud is one quarter ahead of Octopus Server.

So what does it look like? If the build is happy against automated tests:

• Staff - We dog food our own app, our main deploy instance is the first one that's updated
• Canary Customers (+3 days) - We deploy to a random subset of Octopus Cloud customers, about 5% of active instances, which is randomized each time
• Stable (+2 days) - We then deliver to the majority of our Cloud customers' instances
• Laggards - By this point, the release has already been running in production across thousands of instances, and we update the remaining Octopus Cloud instances
• Octopus Server - Eventually, self-hosted Octopus customers will be able to download the latest version of our software

Should you want to learn more about performance for your self-hosted instance of Octopus Server, be sure to check out the documentation on performance. We handle performance for you when you're on Octopus Cloud.

Improved compatibility & integrations

We're always ensuring Octopus remains compatible with the most popular software tools and integrations. We have documentation on compatibility that goes into detail, as well as our integrations page on our website.

By staying up to date, you ensure you are ready for whatever the world throws at you and can adopt and implement new integrations as they become available.

Bug fixes

No one likes bugs; these can be nuanced and overlooked. You can view our release notes on our website, and you can also track which version you are on and which target version you plan to upgrade to using the compare versions option.

We present information clearly about breaking changes, bugs, and more.

Quality of life tweaks

We believe in listening to our customers' feedback; it's quite important to us and one of our core values for every Octonaut at Octopus.

Our customers help us sharpen our product even further, and we often revisit features, integrations, and more to refine them and make their lives easier.

You can learn more about Octopus's core values in our handbook.

Security patches for CVEs

Everyone knows that a core part of keeping your software up to date is addressing security issues, and we publish this information in various places:

• Release notes
• Security Advisories page

We also have a Security Disclosure Policy.

Compliance

Building on security patching, you have to ask yourself: how do you remain compliant if you don't update your software regularly?

It's important that you understand and upgrade your software regularly; this isn't just an Octopus need, it's for all software. By staying up to date, you ensure your business remains compliant and that audit checks pass with flying colors.

Enhancements to existing features

Just because we've shipped a feature doesn't mean it's done and forgotten; we're always listening and looking for ways to improve what we offer.

You'll always gain by updating Octopus software, and more often than not, you'll learn about these in our blog section on our website. You can also subscribe, so you're always up to date with any new blogs that are published.

A great example of this is Platform Hub, where we are releasing this functionality to our customers. You might be asking what Platform Hub is? Check that out in this blog post. We also have a feature page, should you be interested.

New features

We love building features that make developers' lives easier.

You can see what we're up to by visiting our public roadmap, leave us a signal on what's important to you by voting, and you can also submit ideas directly to our product and engineering teams for consideration.

Staying within vendor support

Every Octopus software release receives six months of critical patches.

You can learn more about this on our blog post, where we discuss it, and in our documentation, which covers it for self-hosted Octopus customers.

Should you ever need to contact the Octopus Deploy Support Team, the team will be happy to help.

Deprecations

Occasionally, Octopus will deprecate features that will no longer be supported. These features are eventually removed.

Staying up to date means you're protected against known vulnerabilities as soon as fixes are available, and you're not carrying risks that have already been solved.

You can learn more about this in our docs.

Conclusion

The longer you leave it (not updating regularly), the more risk you carry, the more it costs to resolve when something breaks, and the longer it takes to get back on track. Staying current keeps your attack surface small.

If you prefer that Octopus handle upgrading Octopus for you, then I'd recommend Octopus Cloud. Octopus Cloud reliably hosts thousands of Octopus Deploy customers.

Octopus Cloud is the easiest way to run Octopus Deploy. It has the same software and functionality as Octopus Server, except we host it for you and we call it a Cloud instance. You don't need to download, install, or manage it yourself. You can get started with a free account to try it out.

You can learn more about its architecture in this blog post.

Happy deployments!

Don't worry about code reviews, Gene. Code reviews and approvals have always involved a lot of theater. We just need to perpetuate that illusion a little longer and keep pretending that humans are actually reviewing all that agent-generated code.

Jez is absolutely right; code reviews do involve a lot of theater. Especially now in the era of AI-generated code. In the short amount of time since this post, this trend has become more pronounced. Code review used to be considered a solid approach to ensuring quality and compliance. It just isn't anymore, and we need to be honest about its effectiveness for development teams today.

The chocolate belt wrappers

Consider the all-too-familiar process of reviewing a pull request (PR). The notification bell icon lights up, indicating that you have something to review. You open it, review the code, slap "LGTM" on it, and click "approve." It compiles. Ship it.

Now consider the scenario in which agents write the majority of the PRs. You probably know how this ends up if you've ever seen the "Job Switching" episode of I Love Lucy.

Lucy and Ethel wrapping chocolates

In the episode, Lucy and Ethel take jobs on an assembly line wrapping chocolates. Everything starts fine until then the belt speeds up. Lucy and Ethel can't keep pace, so they start hiding chocolates wherever they can. The chocolates keep coming. The wrapping of chocolates, what we call code review, becomes theater.

In our world, these chocolates are PRs, AI coding agents are the belt, and code review is Lucy, frantically trying to keep up while the quality of what's getting through drops with every passing minute. A lot of what's coming off that belt can be slop. It compiles. (Or, sometimes not.) If you're lucky, it passes your test matrix. Looking closely at the code, it looks fine until you realize the model copied a pattern from its training data that doesn't actually fit your problem. The person reviewing would likely not realize this. The reviewer would likely check whether the syntax and structure are correct, not whether the code should exist in the first place.

Agents can produce huge chunks of code in the time it takes to read this sentence. The PRs reflect this. Now consider the burden this places on a reviewer. Is it reasonable to evaluate a 40,000-line change? Does it get better if we atomize it into 4,000 tiny 10-line diffs? You can read each diff and still miss whether it's the right change. That's because you weren't part of the reasoning that produced it. You have no context whatsoever. It's like flipping to the middle of a book and claiming you know where you are in the story.

Yes, AI makes producing code much, much faster. However, reviewing that code has become much, much harder. As an industry, we tout and celebrate the speed. But we don't talk about the PRs piling up, putting everyone downstream under pressure.

The chocolate belt speeds up

If you take a look at the 2026 DORA report, 90% of developers now use AI tools at work. Developers are spending 2+ hours a day with these tools, completing 21% more tasks and merging 98% more pull requests.

With great power comes great responsibility. The average number of bugs per developer is up 54%. Faros AI's analysis of 10,000+ developers found incidents per pull request are up 242.7%. We've essentially doubled our merge rates while breaking things three times as often. We see the impact of AI-generated code in our own data, too. Our 2026 AI Pulse report found that AI reduces task hours across every part of the delivery pipeline except for code review. 72% of developers use AI to write code, but only 56% bother using it for their reviews. The chocolate belt is accelerating, and Lucy and Ethel are starting to look nervous.

To be fair, Daniel Stenberg, the creator of curl, recently noted that AI-generated contributions have gone from slop to genuinely good. Problem solved, right? Not quite. PRs are arriving faster than his team can review them. We have better chocolates, but the same belt speed problem. Our review queue is starting to resemble a backlog.

So what do we do about it? The prevailing sentiment right now is to chuck AI at the review problem, too. Make Ethel check Lucy's work. But think about that. They're standing at the same belt and they've trained on the same data. They have the same blind spots. "AI reviewed it so we're good" is the new "the dog ate my homework." Except now the dog wrote the homework, ate it, barfed it up, and gave it an A+.

That's the real takeaway from the DORA data. AI is an amplifier. It can amplify our intelligence or our stupidity. We need to be careful. Right now, a lot of us have the chocolate belt of PRs cranked up to full speed.

Enter the wrapping machine

The chocolate belt does exactly what it's supposed to do. The wrapping process (code review) is what failed. And the fix has been staring us in the face since the Continuous Delivery (CD) movement began. Our deployment pipeline is the assurance mechanism, not the human with the approve button. If quality and security requirements are missing from the pipeline as automated checks, code review will never ensure they are met. We are simply hoping that a human – somewhere in the chain – might catch the problem.

Yes, we still need people who can look at a system and say, "This is the wrong approach." That's not going away. But we're expecting that same person also to be the last line of defense against every bug and every security gap in every deployment. That was never going to work. We just didn't have a reason to admit it until now.

What's actually in the chocolate

Let's stop pretending code review is something it isn't.

Code review is great for knowledge sharing and catching design-level issues. But it's horrible at catching every bug in a 40,000-line diff. Bugs matter when code is shipping to production.

So the question we should be asking ourselves isn't "how do we make code review scale?" It's "how do we build a pipeline that can verify what it's shipping, regardless of who or what wrote the code?"

Policy-as-code is one way to get there. We write rules that define our deployment standards, and the pipeline checks every deployment against them. The developer sees what went wrong and how to fix it. There's no waiting around for someone to review a diff.

Learning to wrap chocolate

It would be foolish of me not to mention the fact that there's something a chocolate wrapping machine can't teach you. And that's the process of wrapping chocolate. In our world, that's the act of conducting a code review. It's how junior engineers develop judgment.

Mentorship comes from reading other people's code, getting feedback on your own, and absorbing the unwritten reasons behind certain decisions. That pipeline is already breaking. 73% of organizations have reduced junior developer hiring in the past two years. Junior devs dropped from 32.8% to 24.8% of Stack Overflow respondents between 2024 and 2025. If we let that continue without figuring out another way for juniors to learn, we're in trouble. We end up with a generation of engineers who can prompt effectively but can't reason about a system's design.

I'm not saying we need to remove code review. But we need to stop kidding ourselves that it's the all-seeing, all-knowing quality gate we've built it up to be.

Wrapping up

To reiterate, Jez was right. Code review has worked well enough when humans are involved in the volume of code being reviewed. It was good enough when a team merged a handful of PRs a day. However, it's not good enough when AI is generating them.

The answer isn't a better performance. It's a better pipeline. One that can prove our software works before it hits production. The CD community has been saying this for years. Most of us just didn't have a reason urgent enough to listen. But with the advent of AI and code generation, we're now compelled to.

If our quality gates live in our pipeline, it doesn't matter whether the code was written by a human, an AI, or a very determined cat walking across a keyboard.

If the quality of our code reviews is determined by a human's abilities, we're in trouble, because AI sped up the belt, and the chocolates aren't going to wrap themselves.

Happy deployments!

Prerequisites

• An Octopus Cloud account. If you don't have one, you can sign up for a free trial.
• The Octopus AI Assistant Chrome extension. You can install it from the Chrome Web Store.

:::div{.hint} The Octopus AI Assistant will work with an on-premises Octopus instance, but it requires more configuration. The cloud-hosted version of Octopus doesn't need extra configuration. This means the cloud-hosted version is the easiest way to get started. :::

Creating the project

Ephemeral environments support the creation and destruction of short-lived environments and their associated resources. These environments are often used to deploy feature-branch builds, allowing developers to interact with their work before it is merged into a mainline branch.

Paste the following prompt into the Octopus AI Assistant and run it:

Create a Kubernetes project called "K8s Web App with Ephemeral Environments", and then:
* Use client side apply in the Kubernetes step (the mock Kubernetes cluster only supports client side apply).
* Disable verification checks in the Kubernetes steps (the mock Kubernetes cluster doesn't support verification checks).
* Enable retries on the K8s deployment step.
* Add support for ephemeral environments, with the Parent Environment and Ephemeral Environment channel both called "Features"

---

Create a token account called "Mock Token".

---

Create a feed called "Docker Hub" pointing to "https://index.docker.io" using anonymous authentication.

---

Create a Kubernetes target with the tag "Kubernetes", the URL https://mockk8s.octopusdemos.com, attach it to the "Development", "Test", "Production" environments and the "Features" parent environment, using the health check container image "octopusdeploy/worker-tools:6.5.0-ubuntu.22.04" from the "Docker Hub" feed, using the token account, and the "Hosted Ubuntu" worker pool.

:::div{.hint} The document separator (---) is used to split the prompt into multiple sections. Each section is applied sequentially, which allows you to create different types of resources in a single prompt. :::

As we did in the last post, the AI Assistant creates a functional Kubernetes project pointing to a mock Kubernetes server.

We then added support for ephemeral environments, which requires:

• A parent environment called Features
• A channel that deploys to ephemeral environments, also called Features

Create a new deployment of the project, select the Features channel, and define the FeatureBranch custom field to the name of a feature branch like features/font-change:

The value assigned to the custom field is used as the name for the new environment. Any invalid characters, like the backslash, are automatically sanitized to provide a valid environment name.

The deployment is visible in the Ephemeral Environments section:

Because the Kubernetes step is configured to deploy resources to the namespace #{Octopus.Environment.Name | ToLower}, the ephemeral deployment creates resources in a namespace based on the ephemeral environment name, keeping it separate from the traditional deployments to the Development, Test, and Production environments.

You'll also notice that the Kubernetes target was selected for the deployment because it was attached to the parent Features environment. This demonstrates how durable Octopus resources are linked to ephemeral environment deployments.

Runbooks can be used for those scenarios where you need to provision and deprovision the environment.

Run the following prompt to add two runbooks to the project:

Create a runbook called "Provision Environment" in the project "K8s Web App with Ephemeral Environments".
Allow the runbook to be run from the "Features" environment.
Add a "Run a kubectl script" step run against the target tag "Kubernetes" and echo the text "Provisioning the environment" from a bash script.
Run the step from the "Hosted Ubuntu" worker pool.

---

Create a runbook called "Deprovision Environment" in the project "K8s Web App with Ephemeral Environments".
Allow the runbook to be run from the "Features" environment.
Add a "Run a kubectl script" step run against the target tag "Kubernetes" and echo the text "Deprovisioning the environment" from a bash script.
Run the step from the "Hosted Ubuntu" worker pool.

In the Ephemeral Environments project section, open the Settings tab, and select the new runbooks from the Provisioning runbook and Deprovisioning runbook fields:

Both runbooks must have a published snapshot. Open each runbook, click the Publish button, and click the Publish button again:

Now, when you deploy to an ephemeral environment, the Provision Environment runbook is executed. After one week, the Deprovision Environment runbook is automatically executed, or you can manually deprovision an environment in the Ephemeral Environments section under the Overview tab:

What just happened?

You created a sample Kubernetes project with:

• A channel called Features to deploy to an ephemeral environment based on a custom field value
• A Parent Environment called Features
• A Kubernetes target linked to the Features parent environment
• Two runbooks: one to provision the ephemeral environment, and another to deprovision it

What's next?

The next step is an example of deploying an Argo CD Application with a GitOps based workflow.

That means there's more art to change approvals than most people realize, and it's a threat to an organization's ability to deploy and operate software if they don't have change finesse.

Watch the episode

You can watch the episode below, or read on to find some of the key discussion points.

Watch Continuous Delivery Office Hours Ep.6

Organizational trauma

Change approvals don't arrive without reason. If you have a heavyweight change approval process and frequent or extended change freezes, the chances are that they were introduced after a major incident. If you break financial software around tax year-end, banning deployments for a month before and a month after is, in theory, a reasonable resolution.

Almost every industry has a cadence it wants to protect from instability. Retail has seasonal sales events, the music industry has superstar ticket launches, and finance has a peak as the end of the tax year approaches. The goal is to make sure you can operate your business during these times.

With that goal in mind, we have to bust the myth of change approvals as the mechanism to achieve it. Attempting to protect these key moments with change freezes or cumbersome approval processes has one result: increased instability.

Heavyweight change approvals

Heavyweight change approvals make things less stable by delaying work and causing batches of unreleased changes to accumulate. Meanwhile, developers are starting new work and are losing the immediate familiarity with the oldest changes as they press ahead. One of the ways approvals gain weight is through approval chains, which we looked at in depth in the Compliance through Continuous Delivery report.

Large batches also come with admin that can introduce further problems. Testing becomes more difficult, the likelihood of merge issues increases, and pinpointing the source of a problem is far harder.

This is why the DORA research placed streamlined change approvals in their core model for software delivery. Centralized change approval boards don't work, and process is never the solution to your stability problems.

Streamlining

There are some easy ways to streamline change approvals. Most of these don't look like traditional change management, which is good because we know that doesn't work.

The first way to trim the process is to automate your verification stages. At every level of review, tasks can be automated, whether it's automatically linting and formatting code (instead of debating it), running automated builds and tests, or validating your SBOM is free from insecure dependencies.

Where you need a human review, use a peer-review process for individual changes, enforced on commit, with humans brought in only after the automated validation has passed. If you have advanced change approval needs, categorizing changes by risk lets you apply your people to the changes that most need their perspective.

You won't achieve all of this in one day. It's part of your continuous improvement process. You may improve your chances of stripping the bureaucracy if you follow the capability culture cycle pattern.

Small batches, again

If you follow industry experts or the research, you'll notice that small batches keep cropping up as the answer to many kinds of dysfunction. That's not a coincidence. Large batches cause far-reaching problems that build superlinearly as more changes collect unreleased.

Anything that causes batch size to increase, including change approvals, must be subject to fierce improvement.

Happy deployments!

:::div{.hint}

Continuous Delivery Office Hours is a series of conversations about software delivery, with Tony Kelly, Bob Walker, and Steve Fenton.

You can find more episodes on YouTube, Apple Podcasts, and Pocket Casts.

:::

Securing these agents is a top concern for enterprises. One common solution to improve the security of AI agents is to run them in a sandboxed environment. In this post, I'll take a look at what it means to "sandbox" an AI agent in a production environment.

In brief

• Local AI agents are general-purpose assistants that can perform almost any action on behalf of a user.
• Local AI agents benefit from sandboxes as a countermeasure to their broad access to CLI tools, local files, and networks.
• Shared AI agents are designed for specific tasks.
• Shared AI agents should be decomposed into the agent harness and the tools called by the agent.
• The tools called by shared AI agents are typical web services.
• The term "sandbox" has little meaning for shared AI agents, as the tools can be secured with existing security policies and practices.

Distinguishing between local and shared agents

Before discussing what it means to sandbox an agent, it is important to distinguish between local and shared agents.

Local agents are the result of bespoke configuration in an individual's own workspace. It is the coding agent with a mishmash of MCP servers and personal credentials that a developer has set up to help them with their work. Or the OpenClaw style agent that runs in the background automating tasks like monitoring emails, browsing the web, or organizing files.

To use the pets/cattle analogy (where pets have names and are lovingly cared for while cattle are interchangeable), local agents are pets. Local agents must support a wide range of tasks, including code generation, running scripts, manipulating files, and answering questions. They were never intended to be distributed, and little thought is put into how they might be recreated. Each developer is responsible for their own local agent. In fact, much of the functionality provided by a local agent likely relies on MCP servers exposed by an IDE, which are not available outside a local development environment.

Shared (or managed) agents are designed to perform specialized tasks. They must be secure, testable, deployable, and supported. Shared agents will often be hosted as web applications, perhaps using protocols like the Model Context Protocol (MCP).

Local agents have unique security concerns. It is mesmerizing, and slightly horrifying, watching a local agent query the contents of your /etc/environment file to get the credentials required to execute a curl command as it doggedly attempts to upload a file to a remote server. Local agents are like sharing your keyboard with the most brilliant and amoral entity in the known universe.

Because local agents are general-purpose AI tools, they tend to have broad access to the CLI, local files, and networks. So it makes sense to run local agents in an isolated environment to distinguish between the trust granted to a user and the trust granted to the local AI agent.

Shared agents have a far narrower scope than local agents. Shared agents are designed to solve specific tasks and interact with the world through a small window. The limited scope of shared agents has implications for their security.

The focus of this post is on shared agents. This is not to diminish the security implications of local AI agents, but rather to note that shared agents, iteratively developed and deployed to a production environment, align very closely with the core functionality provided by Octopus.

But before we can understand what it means to sandbox a shared agent, we first need to understand the architecture of shared agents.

Shared agent architecture

At the heart of every AI agent is an LLM making decisions about how best to achieve its task.

For all their wonder and complexity, it is best to think of LLMs used by shared agents as string functions: the prompt string goes in, the response string comes out. (I'm going to ignore the social engineering security aspect of LLMs here, as the generated output of an LLM used by a shared agent is not typically consumed by a person.)

That is it. LLMs cannot, on their own, interact with the world. They cannot browse a web page, read a file, or save a record in a database.

Because LLMs can’t interact with the world, there is very little to contain in a sandbox.

However, this inability to interact with the world severely restricts the problems that LLMs can solve. A chatbot is about as complex a solution as you can build with an isolated LLM. To build useful AI agents, LLMs must be able to act.

This is where the concept of tool calling comes in. Tools are just a fancy way of describing code exposed to an LLM that can interact with the world. MCP is the most common interface through which LLMs learn about and execute tools.

When a tool like switch_lightbulb_on is exposed by an MCP server to an LLM, a prompt like Switch on the lights will cause a physical light bulb to turn on.

Treating the LLM and the tools it calls as separate concerns is crucial to understanding how sandboxes apply to shared AI agents.

Sandboxing the tools

There are many industry examples demonstrating the pattern where the LLM is run as a regular service while the tools called by the LLM are isolated within a sandbox environment.

:::div{.hint} Some of these quotes have been edited for clarity. :::

Claude describes the LLM as the brain and the tools as the hands of an AI agent:

The solution we arrived at was to decouple what we thought of as the “brain” (Claude and its harness) from both the “hands” (sandboxes and tools that perform actions) and the “session” (the log of session events).

Notably, in this description, the hands include sandboxes.

Red Hat describes the separation of the brain and the hands, with the hands running in a sandbox, as "the right choice for multi-tenant agent platforms and production workloads":

The agent's "brain" (reasoning and orchestration) is decoupled from its "hands" (tool execution and code). The platform orchestrates the agent loop and delegates execution to disposable, stateless sandboxes that you control. Credentials are physically separated from the execution environment, injected at the network boundary rather than stored where agent-generated code can reach them. Both the Responses API and Anthropic's Managed Agents follow this pattern, whether the sandbox runs in the provider's cloud or on your own infrastructure through self-hosted environments. This is the right choice for multi-tenant agent platforms and production workloads.

How 11x Rebuilt Their Alice Agent: From ReAct to Multi-Agent with LangGraph notes that agents work best when tools do the heavy lifting:

Tools are preferable over skills. Don't try to make your agent too smart. Just give it the right tools and tell it how to use them.

In the video Securing MCP in an Agentic World with Arjun Sambamoorthy from Cisco, Arjun describes the importance of run-time MCP security with sandboxes isolating MCP servers:

We should also sandbox and isolate MCP servers to make sure there's no crosspollination that's actually happening.

Agentic AI Safety & Security by Dawn Song describes the importance of decomposing systems to enforce the principle of least privilege:

The idea is that instead of building one monolithic agent with different components in one system, one can actually separate the overall agent system into separate components where each component can run its own, for example, container or context such that each separate component can have its own set of privileges depending on its needed capabilities and so on and hence enable and help enforce principle of least privilege.

OpenAI describes when to use a sandbox, and notes that "the sandbox stays focused on provider-specific execution":

Use sandboxes when the agent needs to manipulate files, run commands, mount a data room, produce artifacts, expose a service, or continue stateful work later.

The key split is the boundary between the harness and compute. The harness is the control plane around the model: it owns the agent loop, model calls, tool routing, handoffs, approvals, tracing, recovery, and run state. Compute is the sandbox execution plane where model-directed work reads and writes files, runs commands, installs dependencies, uses mounted storage, exposes ports, and snapshots state.

Keeping those boundaries separate lets your application keep sensitive control plane work in trusted infrastructure while the sandbox stays focused on provider-specific execution.

Azure Container Apps Sandboxes provide a managed service where:

Agents can run anything safely - an agent spawns a sandbox, executes work inside it, and returns the output with no agent host privileges required.

AWS provides the Amazon Bedrock AgentCore Code Interpreter, which similarly provides a sandbox where untrusted code is run:

With the AgentCore Core Interpreter, AI agents can write and execute code securely in sandbox environments, enhancing their accuracy and expanding their ability to solve complex end-to-end tasks.

The provided diagram clearly shows the Agent and LLM sitting outside the sandbox, and the code being executed inside it:

What is clear from these examples is that the LLM is hosted separately from the tools it calls, and it is the tools that are sandboxed, as this is where the real work is done.

What even is a sandbox?

When taking the approach of sandboxing tools, the next decision is which guardrails the sandbox must provide.

At the extreme end, a sandbox provides an environment in which untrusted scripts can run. An example of this is Intel DeepMath, which is a lightweight agent that specializes in solving mathematical problems by running small, sandboxed Python scripts that support and enhance its problem-solving process:

Instead of verbose text, the model emits tiny Python snippets for intermediate steps, runs them in a secure sandbox, and folds the results back into its reasoning, reducing errors and output length.

Your local coding assistant AI agent may even have produced Python scripts to modify files in bulk or search for text.

Because you can do almost anything with a Python script, you need a robust sandbox to prevent any malicious or undesirable actions from being executed.

Running untrusted code is an extreme example, though. Most tools will be far more routine, performing deterministic actions like returning data, sending messages, triggering a workflow, approving a request, etc. Indeed, most of the tools called by a shared AI agent are just wrappers around existing APIs.

The sandbox around these tools must address the same cross-cutting concerns as any web service container, like authentication, authorization, rate limiting, PII redaction, observability, CPU and memory limits, firewalls, etc.

At this point, it may not even make sense to talk about sandboxes at all. Any modern Platform as a Service (PaaS) or orchestration platform has almost certainly addressed these common security concerns, usually without using the term "sandbox."

Do sandboxes make sense?

General-purpose local AI agents running in an individual's workspace absolutely benefit from a sandbox. The fact that a local AI agent can and will do anything you ask (and sometimes things you don't) means a specialized sandbox is a valid countermeasure.

In OpenClaw + Windows, Microsoft demonstrates how OpenClaw is prevented from making unwanted changes to the system by running it in a sandbox:

And you'll notice down here in the corner we've got lots of permissions options along with our sandbox configuration. Now, this sandbox is really interesting because this is using MXC, the Microsoft Execution Containers.

You've got full support about what files and folders you want OpenClaw to have access to, and really granular security features like clipboard access or talking to the internet itself.

OpenClaw already has a rich safety layer, and that layer is only augmented more by appropriate containment that can be managed by me or policies applied by IT.

The concept of a sandbox is also applicable for the execution of generated scripts, which administrators must assume can perform any action.

However, the concept of a sandbox is less meaningful when used to isolate specific, deterministic tools required by shared agents. The security layer built into any modern PaaS offering already supports the cross-cutting security concerns required to host web-based services, authentication and authorization policies are available on APIs exposed by tools, and individual tools can be turned on and off as needed in an MCP server.

You could make a good argument that this collection of controls effectively serves as a sandbox. For example, agent-sandbox combines existing Kubernetes features to provide an AI agent sandbox.

But using the term "sandbox" feels more like a distraction from the implementation of existing, standard security controls applied to any web service because it implies that there is some unique security layer that is specifically required to support AI agents.

Conclusion

The term sandbox is thrown around a lot these days. You don't have to look hard to find examples of AI agents going rogue and deleting files or trashing databases, and it is natural to assume that some kind of sandbox is required to rein in freewheeling AI agents.

But it is important to distinguish between general-purpose local AI agents that are incentivized to support any kind of action and specialized shared AI agents that are designed for a very specific purpose. Further decomposing shared AI agents into the agent harness and the tools highlights that it is the tools that need to be constrained. And centrally managed tools exposed as web services (with an MCP server being a specialized web server) already have a wealth of existing, comprehensive security controls available to secure them.

Enterprises should focus on constraining the tools used by shared AI agents, rather than being distracted by hype around sandboxes. Your existing best practices can be applied to centrally managed tools; there is no need to shoehorn in an additional security layer under the guise of a sandbox.

Happy Deployments!

Kubernetes's default rolling update strategy gradually deploys new pods and retires old ones, but during the swap, your service runs both versions side by side, and a regression in the new image affects every request that lands on a new pod.

Progressive delivery patterns like blue/green have long existed: you stand the new version up alongside the old, prove it's healthy on a separate preview endpoint, then flip user traffic across. Blast radius shrinks to nothing in the bad case, rollback is a single command, and your release stops being a held-breath moment.

In this post, you'll set that up with Argo Rollouts, the controller behind progressive delivery in the Argo ecosystem (which graduated from the CNCF in 2022).

What is Argo Rollouts (and why you need it)

Argo Rollouts is a Kubernetes controller and a set of CRDs that bolt blue/green, canary, and other progressive delivery strategies onto your cluster. The primary CRD is Rollout, a drop-in replacement for the standard Deployment.

You convert an existing Deployment by changing the apiVersion to argoproj.io/v1alpha1 and the kind to Rollout, then adding a strategy.blueGreen or strategy.canary block that describes how a new revision should roll out.

A major reason to reach for it is that there's no built-in way to do this kind of traffic control in Kubernetes (you can't decide where requests go independently of which pods are Ready). There's no easy rollback to the previous version once the update has started.

Argo Rollouts fills in everything around that. It plugs into ingress controllers (Traefik, ALB) and service meshes (Istio, Linkerd, SMI) for real traffic shaping. It can query metrics providers (Prometheus, Datadog, CloudWatch, New Relic) to gate promotions on hard numbers, and it tracks every revision as its own ReplicaSet, so flipping back is instant.

Prerequisites

This tutorial assumes some familiarity with Kubernetes. You'll also need:

• A working Kubernetes cluster (EKS, GKE, AKS, or local like Minikube/Kind).
• kubectl installed and pointed at the cluster (kubectl get nodes should return at least one Ready node)
• helm v3 installed
• curl for hitting the demo app

Step 1: Installing the Argo Rollouts controller

Argo Rollouts ships as a controller that runs in its own namespace, along with a kubectl plugin you'll use on your laptop to inspect and steer rollouts.

To install the controller, follow these steps:

1. Add the Argo Helm repo and install the controller:

   helm repo add argo https://argoproj.github.io/argo-helm
   helm repo update
   helm install argo-rollouts argo/argo-rollouts \
     --namespace argo-rollouts \
     --create-namespace \
     --wait
   

2. Confirm the controller pods are up:

   kubectl -n argo-rollouts get pods
   

   You should see something like:

   NAME                             READY   STATUS    RESTARTS   AGE  
   argo-rollouts-dcd465dfc-8m2ql    1/1     Running   0          79s  
   argo-rollouts-dcd465dfc-q92k4    1/1     Running   0          79s
   

Step 2: Installing the kubectl argo rollouts plugin

The controller is running, but the most ergonomic way to drive a rollout — inspecting state, setting images, promoting, rolling back — is the kubectl argo rollouts plugin. It's a separate binary that drops onto your PATH, and kubectl picks it up automatically.

To install the plugin, follow these steps:

On macOS with Homebrew:

brew install argoproj/tap/kubectl-argo-rollouts

On Linux:

curl -sLO https://github.com/argoproj/argo-rollouts/releases/latest/download/kubectl-argo-rollouts-linux-amd64
chmod +x kubectl-argo-rollouts-linux-amd64
sudo mv kubectl-argo-rollouts-linux-amd64 /usr/local/bin/kubectl-argo-rollouts

Verify it's wired up:

kubectl argo rollouts version

You should see something like kubectl-argo-rollouts: v1.8.3+.... From here on, we'll use kubectl argo rollouts ... subcommands to drive the rollout.

Step 3: Defining the Rollout

Argo Rollouts' core idea is that you swap your Deployment for a Rollout resource. The pod template inside it is identical to a Deployment's.

What changes is the spec.strategy block, which describes how a new revision should roll out.

For blue/green, you need three things:

1. A Rollout with spec.strategy.blueGreen configured
2. An active Service, which always points to whichever ReplicaSet is currently serving production traffic
3. A preview Service, which points at the new ReplicaSet before it gets promoted, so that you can test it in isolation

Argo Rollouts injects the rollouts-pod-template-hash label into each Service's selector at runtime, which is how it switches traffic without you ever editing the Services.

Write the manifest:

cat > rollout.yaml <<'EOF'
apiVersion: v1
kind: Service
metadata:
  name: rollouts-demo-active
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
  selector:
    app: rollouts-demo
---
apiVersion: v1
kind: Service
metadata:
  name: rollouts-demo-preview
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
  selector:
    app: rollouts-demo
---
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: rollouts-demo
spec:
  replicas: 2
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: rollouts-demo
  template:
    metadata:
      labels:
        app: rollouts-demo
    spec:
      containers:
        - name: rollouts-demo
          image: argoproj/rollouts-demo:blue
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          resources:
            requests:
              cpu: 25m
              memory: 32Mi
  strategy:
    blueGreen:
      activeService: rollouts-demo-active
      previewService: rollouts-demo-preview
      autoPromotionEnabled: false
      scaleDownDelaySeconds: 30
EOF

A few fields are worth calling out in the blueGreen block:

• activeService and previewService are the names of the two ClusterIP Services above. Argo Rollouts owns their selectors from here on; you don't edit them by hand.
• autoPromotionEnabled: false is what makes this a manual promotion. The new ReplicaSet comes up, you inspect it on the preview Service, and only then do you flip the active Service over. Set it to true (the default) and Argo will auto-promote the moment the new pods are Ready.
• scaleDownDelaySeconds: 30 keeps the old (blue) ReplicaSet around for 30 seconds after promotion, so if something goes wrong in those first few seconds, you can flip back instantly without rescheduling pods.

Apply it:

kubectl apply -f rollout.yaml

We're using argoproj/rollouts-demo,, a tiny app published by the Argo team that serves an HTML dashboard and a /color endpoint that reports which tagged image is running (blue, green, yellow, etc.). It's perfect for seeing the cutover happen in real time.

Step 4: Checking the initial state

Take a look at the rollout:

kubectl argo rollouts get rollout rollouts-demo

Output:

Name:            rollouts-demo  
Namespace:       default  
Status:          ✔ Healthy  
Strategy:        BlueGreen  
Images:          argoproj/rollouts-demo:blue (stable, active)  
Replicas:  
  Desired:       2  
  Current:       2  
  Updated:       2  
  Ready:         2  
  Available:     2  

NAME                                       KIND        STATUS     AGE  INFO  
⟳ rollouts-demo                            Rollout     ✔ Healthy  34s  
└──\# revision:1  
   └──⧉ rollouts-demo-86c957c6d6           ReplicaSet  ✔ Healthy  34s  stable,active  
      ├──□ rollouts-demo-86c957c6d6-72kjf  Pod         ✔ Running  34s  ready:1/1  
      └──□ rollouts-demo-86c957c6d6-nv3zg  Pod         ✔ Running  34s  ready:1/1

One revision, two pods, both stable and active. Both Services currently point at the same ReplicaSet hash. You can confirm with:

kubectl get svc rollouts-demo-active rollouts-demo-preview \
  -o jsonpath='{range .items[*]}{.metadata.name}{" -> hash="}{.spec.selector.rollouts-pod-template-hash}{"\n"}{end}'

Output:

rollouts-demo-active -> hash=86c957c6d6
rollouts-demo-preview -> hash=86c957c6d6

Step 5: Triggering a new version

Now let's deploy a new revision. We'll change the image tag from blue to yellow:

kubectl argo rollouts set image rollouts-demo \
  rollouts-demo=argoproj/rollouts-demo:yellow

Argo creates a new ReplicaSet (rev 2) for the yellow image and waits, because we set autoPromotionEnabled: false. The active Service still points at blue. The preview Service is re-pointed at yellow:

kubectl argo rollouts get rollout rollouts-demo

Output:

Status:          ॥ Paused
Message:         BlueGreenPause
Strategy:        BlueGreen
Images:          argoproj/rollouts-demo:blue (stable, active)
                 argoproj/rollouts-demo:yellow (preview)
Replicas:
  Desired:       2
  Current:       4
  Updated:       2
  Ready:         2
  Available:     2

NAME                                       KIND        STATUS     AGE  INFO
⟳ rollouts-demo                            Rollout     ॥ Paused
├──# revision:2
│  └──⧉ rollouts-demo-7cf9dff6bb           ReplicaSet  ✔ Healthy  38s  preview
│     ├──□ rollouts-demo-7cf9dff6bb-cbp2c  Pod         ✔ Running  38s  ready:1/1
│     └──□ rollouts-demo-7cf9dff6bb-fn2gt  Pod         ✔ Running  38s  ready:1/1
└──# revision:1
   └──⧉ rollouts-demo-86c957c6d6           ReplicaSet  ✔ Healthy  5m   stable,active
      ├──□ rollouts-demo-86c957c6d6-72kjf  Pod         ✔ Running  5m   ready:1/1
      └──□ rollouts-demo-86c957c6d6-nv3zg  Pod         ✔ Running  5m   ready:1/1

This is the heart of blue/green. The cluster is now running both versions, but only blue is serving real traffic.

Step 6: Proving the split with curl

Forward both Services to your laptop on different local ports:

kubectl port-forward svc/rollouts-demo-active 8080:80 >/dev/null 2>&1 &
kubectl port-forward svc/rollouts-demo-preview 8081:80 >/dev/null 2>&1 &
sleep 3

Hit each one using:

echo "active : $(curl -s http://127.0.0.1:8080/color)"
echo "preview: $(curl -s http://127.0.0.1:8081/color)"

Output:

active : "blue"  
preview: "yellow"

This is exactly the window where you'd run smoke tests, point a staging frontend at the preview hostname, or have Argo run an AnalysisTemplate against Prometheus.

Nothing about production traffic has changed yet.

When you're done with the port-forwards, run the following command:

kill %1 %2 2>/dev/null

Step 7: Promoting

When you're happy, flip the active Service over with one command:

kubectl argo rollouts promote rollouts-demo

Output:

rollout 'rollouts-demo' promoted

Argo updates the active Service's selector to the new ReplicaSet hash.

Subsequent requests should show production traffic is in yellow. The old blue pods stick around for scaleDownDelaySeconds (30 by default) before being torn down, which is what makes the next section possible.

Confirm the cutover by running:

kubectl argo rollouts status rollouts-demo --timeout 60s

You should see Healthy, and the Service selectors should now agree:

kubectl get svc rollouts-demo-active rollouts-demo-preview \
  -o jsonpath='{range .items[*]}{.metadata.name}{" -> hash="}{.spec.selector.rollouts-pod-template-hash}{"\n"}{end}'

Output:

rollouts-demo-active -> hash=7cf9dff6bb
rollouts-demo-preview -> hash=7cf9dff6bb

Step 8: Rolling back

If something goes wrong after the promotion (a metric tanks, you spot an error in the logs, a teammate flags a bug), undo it by running:

kubectl argo rollouts undo rollouts-demo

That brings the previous ReplicaSet back as the new "preview" and pauses, waiting for you to confirm with promote once more, which flips the active Service back to it. Because the old pods were kept warm by scaleDownDelaySeconds, this happens in seconds, not whatever your image pull time is.

Step 9: Cleaning up

Once you're done, you can tear down the demo using:

kubectl delete -f rollout.yaml
helm uninstall argo-rollouts -n argo-rollouts
kubectl delete ns argo-rollouts

How does this fit with Octopus Deploy and Argo CD

Everything we've done so far works on its own. You've got a Rollout, two Services, and a one-command promote/undo loop.

Argo Rollouts is happy to do its job at the cluster level. What it doesn't have is an opinion on how dev becomes staging and then becomes production. Who's allowed to push the button, or what the deployment history looked like six weeks ago.

This is the layer Octopus Deploy is built for. A good mental model is:

• Argo Rollouts owns the cluster-side mechanics: Which ReplicaSet is active, which is preview, when to flip, and when to scale down old pods.
• Argo CD owns the GitOps sync: The Rollout (and its Services) live in a Git repo, and the cluster state is reconciled to match.
• Octopus owns everything above that: Environments, approval gates, release lifecycles, audit trails, and the self-service UI that developers actually click on.

The promotion path between environments is described once in Octopus and reused across every service, instead of being re-encoded in each team's CI script.

Ship green, sleep through the night

If you made it this far, you've got the cluster-side mechanics of progressive delivery sorted: a Rollout flipping between active and preview Services, a manual promotion gate, and instant rollback. That's the hard, hands-on layer done.

What's missing is the orchestration above it, including environments, approvals, audit trails, and the self-service flow your developers actually click. That's where Octopus Deploy slots in, sitting on top of Argo CD and Argo Rollouts to give you a complete progressive delivery stack across every environment, not just one cluster.

Connect your Argo CD instance to Octopus and see how the whole pipeline comes together, or try Octopus free and wire it up against your own cluster.

Happy deployments!

Argo CD 3.5 introduces native, first-class mutual TLS (mTLS) support. By embedding encryption and identity verification directly into its components, it eliminates the need for running service mesh sidecars, writing custom certificate-rotation scripts, or managing complex volume projections just to make sure traffic is secure between the ArgoCD components.

Why mTLS is one step ahead of one-way TLS

Standard TLS provides one-way authentication: the client verifies the identity of the server via a certificate, but the server accepts connections from any client inside the network boundary. So far so good!

Let's take, however, the case of zero-trust architecture, where relying entirely on the network is an antipattern. If an attacker compromises a single pod within the cluster, they can potentially communicate with the repo-server without any authentication.

Mutual TLS (mTLS) addresses this by requiring both sides to authenticate before exchanging any data:

• The client verifies the repo-server certificate to ensure it is communicating with the legitimate repository management layer.
• The repo-server validates the client's certificate against a trusted Certificate Authority (CA) to confirm that the incoming connection originates from an authorized Argo CD component.

:::figure

:img{ src="/blog/img/mtls-support-argocd/blog_1.png" alt="mTLS vs TLS" }

:::

What do you gain by using mTLS:

• First and most important, you stop assuming internal cluster traffic is safe. Every internal component that needs to "talk" to the repo-server must explicitly prove its identity.
• If your environment is regulated by compliance frameworks such as SOC 2, HIPAA, and PCI-DSS, then you are already covered. Native mTLS satisfies the security requirements without adding third-party dependencies.
• By issuing different certificates to different components, you can precisely log which service initiated a connection and establish a foundation for fine-grained access controls.
• Even if an attacker gains access to a pod, they can't actually talk to the repo-server because they lack the required signed client certificate.

The pre-3.5 reality: how operators managed internal encryption

Before native support was introduced in version 3.5, achieving mTLS within an Argo CD deployment forced teams to choose between several complex architectural workarounds.

The service mesh approach

The most common pattern was transferring the encryption responsibility to an external service mesh like Istio or Linkerd. Teams would inject sidecar proxies into their Argo CD pods to intercept traffic and handle the TLS handshake transparently. This architecture was ok-ish. It worked, but if you ask operators, they would probably complain because they had to manage, upgrade, and debug an entirely separate control plane. On the other side, if secure communications is a hard-requirement, then this approach was a great implementation.

:::figure

:img{ src="/blog/img/mtls-support-argocd/blog_4.png" alt="Service Mesh Approach for mTLS in Argo CD" loading="lazy" }

:::

Manual cert-manager integration

Other teams used cert-manager to automatically provision certificates into Kubernetes Secrets, paired with custom Kustomize or Helm patches to manually map those secrets to specific paths inside the Argo deployments. The main point of failure here was certificate rotation. Coordinating pod restarts to ensure components picked up renewed certificates often required writing custom wrapper scripts or relying on auxiliary operators like Reloader, which added more moving pieces to the platform.

Static secret vaulting

In highly locked-down environments, teams synchronized certificates from external stores like HashiCorp Vault using tools like the External Secrets Operator (ESO). While this approach is very auditable-friendly, it creates a tight coupling between the secret store, the sync operator, and the GitOps controller.

In smaller companies, the fallback was often base64-encoding static, long-lived certificates directly into Kubernetes ConfigMaps or Secrets. This bypassed infrastructure overhead but required manual secret rotation which was often a manual process.

Overall, you can see the same pattern in all three approaches described above (there might be more, but they all follow the same basic principles) Internal communication security was handled as an external network problem rather than a native application capability, resulting in fragile configurations challenging to maintain.

What's new in 3.5: built-in mTLS

Argo CD 3.5 simplifies this landscape by moving the handshake, verification, and configuration logic entirely into the application. You no longer need to write complex volume mounts or alter network layers; It's as simple as provisioning a single, specifically named Kubernetes Secret to activate mTLS.

Below you can find a high-level overview of the new features:

1. Super-easy setup (auto-discovery). Argo CD automatically looks for a Secret named argocd-repo-server-mtls. When discovered, the manifests handle the internal mounting and environment configurations out of the box, eliminating manual template patching.
2. The repo-server requires self-directed communication to execute liveness and readiness probes. In a strict mTLS environment, a service can easily block its own health checks. Argo CD 3.5 addresses this by automatically generating memory-lived, ephemeral certificates dedicated exclusively to internal loopback probes, ensuring monitoring remains functional without manual intervention.
3. The feature accommodates both simple and highly complex environments. You can start with a single shared client certificate across all components and transition to unique per-component identities later without re-architecting your underlying deployment strategy.

:::figure

:img{ src="/blog/img/mtls-support-argocd/blog_2.png" alt="mTLS handshake. How Argo CD 3.5 handles authenticates connections between Argo CD components" loading="lazy" }

:::

Now let's take a more detailed look at the setup process for the shared-certificate configuration.

Getting started: the shared-certificate setup

For the majority of production deployments, a single shared client certificate used by all client components is enough and requires minimum configuration

Step 1: Generate the certificates

If you are not using an automated PKI pipeline, you can generate the required CA, server, and client certificates using standard OpenSSL commands:

openssl genrsa -out ca.key 2048
openssl req -new -x509 -days 365 -key ca.key -out ca.crt \
  -subj "/CN=argocd-internal-repo-ca"

openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr \
  -subj "/CN=argocd-repo-server"
openssl x509 -req -days 365 -in server.csr \
  -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out server.crt

openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr \
  -subj "/CN=argocd-core-clients"
openssl x509 -req -days 365 -in client.csr \
  -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out client.crt

Step 2: Construct the mTLS kubernetes secret

Create a secret named argocd-repo-server-mtls in your Argo CD namespace. The keys within the data block must strictly conform to the expected naming convention:

apiVersion: v1
kind: Secret
metadata:
  name: argocd-repo-server-mtls
  namespace: argocd
type: Opaque
data:
  client-ca.crt: <BASE64_ENCODED_CA_CRT>
  client.crt: <BASE64_ENCODED_CLIENT_CRT>
  client.key: <BASE64_ENCODED_CLIENT_KEY>
  server-ca.crt: <BASE64_ENCODED_CA_CRT>

If you are a CLI fan, you can generate and inject this secret directly using kubectl:

kubectl create secret generic argocd-repo-server-mtls \
  --from-file=client-ca.crt=ca.crt \
  --from-file=client.crt=client.crt \
  --from-file=client.key=client.key \
  --from-file=server-ca.crt=ca.crt \
  -n argocd

Step 3: Trigger a rolling restart

As usual, to let the services pick up the new configuration, secrets, and certificates and initialize mTLS, you need to run a rolling restart across your deployments:

kubectl rollout restart -n argocd \
  deployment/argocd-server \
  deployment/argocd-repo-server \
  deployment/argocd-application-controller \
  deployment/argocd-applicationset-controller

If you are running the application controller in a High Availability (HA) configuration, remember to target the statefulset instead:

kubectl rollout restart -n argocd statefulset/argocd-application-controller

Validating the connection handshake

Once the pods are restarted, check the argocd-repo-server logs to confirm successful initialization. You should observe log messages related to the self-generation of certificates for the internal health check as below:

Generated ephemeral health-check client certificate (CN=argocd-repo-server-health)

To verify that mTLS is now enabled, try to execute a direct gRPC or HTTP request to the repo-server from an unauthenticated pod inside the cluster. The connection should terminate immediately during the TLS handshake phase, log an untrusted client error on the server side, and prevent any data exposure.

To the contrary all internal communications to repo-server are now encrypted.

So far we have seen how easy to enable mTLS using a default approach, same client certificate across all components. This probably works for most of the production environments, but there are cases where you might want to use different certificates for different components. The following sections will show you how to do that.

Advanced configurations: per-component certificates

In enterprise environments with strict auditing requirements or multi-tenant architectures, sharing a single client certificate across all services may violate compliance rules. Argo CD 3.5 supports unique and different, per-component certificates through the following two approaches.

:::figure

:img{ src="/blog/img/mtls-support-argocd/blog_3.png" alt="Advanced Configurations: Per-Component Identities" loading="lazy" }

:::

Option A: multiple keys within a single secret

You can store all individual component certificates inside the primary argocd-repo-server-mtls secret using distinct key identifiers. This centralizes your secret management while allowing smooth delivery to each client component via volume projection patches. Check the sample yaml below. You can see that for each component we have a separate key for the client certificate (server-*, controller-*, etc.)

apiVersion: v1
kind: Secret
metadata:
  name: argocd-repo-server-mtls
  namespace: argocd
type: Opaque
data:
  client-ca.crt: <BASE64_CA_PEM>
  server-client.crt: <BASE64_SERVER_CERT_PEM>
  server-client.key: <BASE64_SERVER_KEY_PEM>
  controller-client.crt: <BASE64_CONTROLLER_CERT_PEM>
  controller-client.key: <BASE64_CONTROLLER_KEY_PEM>

Once you have added the above secret to K8s, patch the volume mount configuration of each component deployment, ensuring it maps its custom key to the filename the client expects (client.crt):

spec:
  template:
    spec:
      volumes:
      - name: argocd-repo-server-mtls
        secret:
          secretName: argocd-repo-server-mtls
          items:
          - key: server-client.crt
            path: client.crt
          - key: server-client.key
            path: client.key
          - key: client-ca.crt
            path: client-ca.crt

The main advantage of this approach is that you are still keeping all certificates in a single secret. Obviously the disadvantage is that you need to maintain explicit volume mount overrides across all components that need to talk to repo-server.

Option B: isolated secret objects per component

The second approach suggests breaking the secret configuration to individual, component-specific secrets (e.g., argocd-repo-server-mtls-server, argocd-repo-server-mtls-controller): So for each component that communicates with repo-server, you create a separate secret with the appropriate client certificate and key. For example:

apiVersion: v1
kind: Secret
metadata:
  name: argocd-repo-server-mtls-server
  namespace: argocd
type: Opaque
data:
  client.crt: <BASE64_SERVER_CERT_PEM>
  client.key: <BASE64_SERVER_KEY_PEM>

Then you need to patch again each deployment's volume specification to point to its corresponding secret:

spec:
  template:
    spec:
      volumes:
      - name: argocd-repo-server-mtls
        secret:
          secretName: argocd-repo-server-mtls-server

This is the most clear approach because you keep the secrets for each component isolated from the others. This allows you to rotate keys independently of the primary secret, which is a common practice in multi-tenant environments.

But as you know, there's no free beer. The downside (if you think of it as a downside) of this approach is that you need to maintain separate Kubernetes Secret objects for each component.

Real-world use cases

Comprehensive zero-trust layering

Native mTLS provides a critical mid-tier authentication layer that complements existing security layers. A strong and complete defense-in-depth model for Argo CD typically relies on the following layers:

1. Network policies that restrict pod-to-pod communication paths so that only valid Argo CD components can route packets to the repo-server port.
2. K8s service accounts that define exactly what an authenticated client process is allowed to execute once the connection is opened.
3. Native mTLS which is used to authenticate data exchange between services at the application layer.
4. Application RBAC which allows fine-grained access control to Argo CD resources.

Audit-ready compliance (SOC 2 / HIPAA / PCI-DSS)

When preparing for an audit, proving internal data security can be challenging when relying on third-party service meshes or complex bespoke scripts. Auditors look for easily verifiable, reproducible security controls. Pointing to a native, platform-supported configuration driven by a declarative Kubernetes Secret simplifies the compliance narrative considerably compared to explaining a custom infrastructure mesh setup.

Multi-tenant control planes

For large organizations running multi-tenant internal developer platforms (IDPs), combining per-component certificates with upstream gRPC interceptors allows cluster administrators to log, trace, and isolate internal requests precisely by team or business unit. This level of traceability is highly beneficial for forensics and chargeback metrics.

Migration patterns

Transitioning an active production cluster from a service mesh or custom cert-manager architecture to native mTLS might sound like a complex task but it can be executed with minimal risk using the following, incremental, approach:

1. You can keep your existing Root CA. Use your current certificate authority to generate the initial client certificates and keys.
2. Deploy the argocd-repo-server-mtls secret into the cluster while your existing sidecars or custom mounts are active. The presence of the secret will not change anything until a reload occurs.
3. Execute a rolling restart of all Argo CD components to pick up the new configuration.
4. Verify that the connection handshake (via the native mTLS configuration) is successful and that all internal traffic is now encrypted.
5. Once the connection is established, remove the old configuration, disable sidecar injection, etc.

Well done! – you are now using native mTLS for all internal traffic.

Architectural recommendation

This article cannot cover all the production cases out there but we can summarize the recommendations for two groups of implementations. If you are operating a small-to-medium-sized Argo CD installation looking for an immediate security upgrade with low operational friction and no specific requirement to audit individual internal component traffic streams. On the other side, if you are looking for a more robust and comprehensive security model, because, for instance, you are in a highly regulated enterprise environment, then you should consider the per-component approach that gives you the most flexibility and control.

For most teams, starting with a shared certificate configuration is the most pragmatic approach. The good (excellent) news is that if your compliance needs scale over time, migrating to separate component identities involves adjusting your manifest overlays without replacing your underlying secrets architecture.

For those who have accepted that you can move fast without breaking things, a second myth is emerging: The myth of speed.

When you look at teams that deploy frequently with short lead times, you'll find they are generally more successful than those who work in larger batches. This checks out when you consider the additional work large batches cause. There's also a benefit to the automation you introduce to achieve higher throughput.

But if you think speed alone is driving the success of these teams, their products, or the organization's market share, you're making a common mistake.

You can develop a better understanding of the broader factors that contribute to success using the Bullseye Model, which captures the nuance of product velocity.

A quick archery lesson

Many sports present a target with a bullseye, but we'll use archery as an example today.

When you practice shooting a bow, your goal isn't to hit the bullseye (though that will become the goal later). Instead, you should develop consistency so your arrows land in a tight group. Until you can consistently group your arrows, any bullseye you hit is chance, not skill.

Much of the skill in archery comes from the discipline of form. You have to develop consistency in where you place your feet, how you hold your body, the way you draw the bow, and how smoothly you release the string. You repeat this by making very deliberate moves many times over, working through a checklist in your mind each time, until it becomes automatic.

:::figure

:img{ src="/blog/img/the-bullseye-model-for-product-velocity/grouping.png" alt="Two targets, one with holes scattered across the surface and one with a tight grouping of holes in the top left" loading="lazy" }

::figcaption[While the left target scores more, the right target shows skill and consistency.]

:::

Once you've developed these skills, missing the bullseye becomes part of the feedback loop. When you know you've got all the movements right, you're able to attribute any drift to other sources. You can now adjust your aim to compensate for the many variables, like wind speed, moisture in the air, and distance, and you develop a feeling for when you've damaged an arrow and need to repair it.

If you were betting on an archer based on scores, you'd likely pick the left-hand target. You now know that, over the course of a whole competition, the right-hand target was made by the archer more likely to win. When an archer can group their arrows in a tight bunch, they can move that group closer to the bullseye and achieve far higher scores than those hitting the target randomly.

Why speed isn't enough

Let's return to the topic of building software. We need to establish why speed alone isn't a sufficient strategy. If you've been following the research, you'll be familiar with the insight that deploying more often with short lead times makes your team, software product, and organization more successful (source: DORA). Interpreting this as "speed equals success" is a simplification of the true picture.

Many teams and organizations are working hard to improve their software delivery performance without realizing a crucial element is missing from their approach. Alongside the smooth flow of changes, you need to develop a deep understanding of the user, their goals, and what's causing them problems and friction.

The research captures this in a construct called "user-centric focus," and the Bullseye Model is how you navigate it.

:::figure

:img{ src="/blog/img/the-bullseye-model-for-product-velocity/target.png" alt="An archery target with concentric circles of white, black, blue, red, and a golden bullseye in the center." loading="lazy" }

::figcaption[An archery target with the perfect product represented by the bullseye.]

:::

The bullseye sits at the center of the target. It represents an absolutely perfect product-market fit for your software product. If you imagine the best product your customers could ever want, that's the bullseye. Except you can't imagine it, which is why you need to apply the Bullseye Model.

Product velocity

There are infinite paths for developing a software system. The two most interesting are where you are now compared to the ideal product you could build. The concept of the ideal product is asymptotic, but it's better, faster, safer, and cheaper than your competitor's products.

To map product velocity, you need to know your starting point. What you've built likely isn't the absolute best product, so your customers' needs are not fully met. Your starting point will be somewhere other than the bullseye. It may be close to the gold circle, or it could be wallowing in the very edge of the last white one.

From this starting point, you create a roadmap, which is your hypothesis about what will make your software look more like the ideal product. Each roadmap entry is another arrow hitting the target, and each time you shoot an arrow, you have the opportunity to observe where it landed.

The line between the starting point and the new arrow has a distance and direction. This is your vector, telling you whether you're heading in the right direction and how quickly. You have to watch both, as only a careless organization celebrates speed without paying attention to direction.

:::figure

:img{ src="/blog/img/the-bullseye-model-for-product-velocity/target-vector.png" alt="Two holes with an arrow showing the distance and direction between the shots" loading="lazy" }

::figcaption[Vectors have distance and direction]

:::

The direction of the vector tells you how well your roadmap is servicing the hunt for the ideal product. If the vector points to the bullseye, your assumptions were perfect. More often, you'll find you're making less efficient moves.

When you move away from the bullseye, your hypothesis is invalid, and you need to adjust course. You might back up and try a different approach (returning to the previous starting point), or incorporate feedback that changes the outcome (continue from where your arrow landed, hoping to flip the loss to a gain).

When speed does matter

On the Bullseye Model, speed refers to how quickly you can prepare and shoot each arrow. We can apply the lessons from the research here, as we don't want throughput (lots of arrows) without stability (hitting the target).

Speed does matter, but it's not the goal. Your ultimate aim is to reach the bullseye, but being able to smoothly and quickly deliver arrows creates more opportunities to check your vector and adjust course. Crucially, when you start trending away from the center, you can rapidly reverse course without over-investing in the wrong approach. You might start building an "offline mode" only to discover users are confused by how you've implemented it, or that they don't value the idea at all and would prefer to be notified when they are back online.

Any speed you have in excess of your ability to check your vector and change course is wasted, as it's as likely to move you rapidly away from the bullseye as it is to move you closer. You need to develop feedback cycles in concert with your software delivery throughput, while maintaining stable operations.

Moving in the right direction is the primary concern and high-throughput is a valuable way to check your vector more often.

:::figure

:img{ src="/blog/img/the-bullseye-model-for-product-velocity/vector-comparison.png" alt="An origin hole and two different next shot possibilities, one is a moderate improvement, the other moves away from the bullseye" loading="lazy" }

::figcaption[Modest progress in the right direction always beats a great leap in the wrong direction]

:::

AI makes deployment pipelines crucial

The introduction of AI-assisted software development means teams have the potential to increase their throughput. There are preconditions for this to be true: the techniques and practices described in Continuous Delivery, as evidenced by research programs like DORA.

When you increase change throughput, you must adapt to this rate of change in other areas, or queues will eat all your gains. Manual stages in your deployment pipeline need refinement. You may automate, augment, or prioritize them, but leaving them as they were before AI-assisted development is not an option.

Deployment governance has become crucial in regulated organizations and those working at scale, as fragmented governance processes will result in a total loss of all return on your AI investments. Your default path to production must be the easiest, safest, and most compliant way to deliver changes, with no exceptions or expediting.

The Bullseye Model also reminds you that the rate at which you can collect and absorb feedback is a crucial consideration of how quickly you can introduce changes. If you can't tell if a change is moving you toward the ideal product, you're simply spinning out of control.

While our competitors are counting token use, pull requests, or lines of code, the thing that we should focus on above all else is how rapidly we are closing the gap between where our product is today and the ideal product described in our roadmap.

Happy deployments!

There are many things I like about ephemeral environments, and I think they add significant value to the software development lifecycle. However, I have recently been asked a few times whether ephemeral environments can be skipped as part of the PR process. Once was at NDC Sydney, where I gave a talk on Ephemeral Environments, and another was by a colleague who regularly works on these microsites internally.

I sat down to think about this, and refactored the GitHub workflow to accommodate the request.

Why skip ephemeral environments?

Ephemeral environments are valuable, but as I learned through these recent conversations, they're not always required. Our ephemeral environment process for microsites, which creates an Azure Storage account with Terraform and then deploys the website package to the target storage account, can take 5 to 9 minutes.

Our GitHub repositories have protections on the main branch, and in addition to a required review, we have checks that must pass before the PR can be merged. Some of those checks check spelling and Markdown syntax, but another checks that the build process completed successfully. Part of that build triggers the ephemeral environment deployment in Octopus Deploy, so deploying the ephemeral environment can delay the build's completion.

Another reason that came up in a community discussion was the issue of busy repositories and the amount of infrastructure that may be provisioned when environments are spun up for each PR. Especially if those environments have high costs, there could be real cost implications at scale. Though I'd also argue that there are other factors to investigate, like the duration of opened PRs, to see if optimizations can be made elsewhere.

The common thread across all these scenarios is that the decision to deploy an ephemeral environment is contextual; it depends on the nature of the change and the needs of the people reviewing it. Rather than removing ephemeral environments from the process entirely, what I needed was a way to make them optional on a per-PR basis, with minimal friction for the developer raising the PR.

It turned out that GitHub's PR labels were the perfect tool for the job.

Skipping ephemeral environments with a PR label

The original GitHub workflow file only had one job. All steps were within the single job, and the required checks for the GitHub repo were configured to verify a successful run of this single job before allowing the PR to be merged.

While you can make steps in a workflow conditional, the more I looked at it, the more I realized it made sense to break the workflow into multiple jobs to achieve different outcomes.

• build — always runs on every PR. It installs dependencies, builds the site, runs tests, packages the artifact, and uploads it for use by subsequent jobs.
• deploy-ephemeral — picks up the artifact from the build job, pushes it to Octopus Deploy, creates the ephemeral environment, and deploys the release. This is the job we want to make optional.
• pr-ready — a lightweight gate job that acts as the required status check for the main branch. It verifies that build succeeded and that deploy-ephemeral either succeeded or was deliberately skipped.

The key to making deploy-ephemeral optional is a single if: condition on the job:

deploy-ephemeral:
  runs-on: ubuntu-latest
  needs: build
  if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip-ephemeral') }}
  ...

This checks whether the PR has a label named "skip-ephemeral". If the label is present, the entire job is skipped. If it isn't, the job runs as normal. Adding the label to a PR before opening it is all that's needed to skip the deployment to the ephemeral environment.

But by breaking the workflow into two jobs, I introduced another problem I needed to solve.

Using a gate job to keep branch protection intact

Previously, the required check on the main branch pointed directly at the single build job, and if it passed, the PR could be merged. With the refactored workflow, I couldn't configure the required check to check the status of deploy-ephemeral anymore, because that job might be intentionally skipped. A skipped job registers as a failed required check in GitHub, which would block the PR from merging even when everything went exactly as intended.

I needed a reliable way to signal whether the overall workflow succeeded or failed, regardless of whether the ephemeral deployment ran. That's where the pr-ready gate job comes in.

The pr-ready job runs after both build and deploy-ephemeral, and uses if: always() to ensure it runs regardless of what happened upstream. Inside the job, a single step checks two conditions:

• build must have succeeded, as there's no scenario where a failed build should result in a mergeable PR
• deploy-ephemeral must have either succeeded or been deliberately skipped. A failure in the deployment job should still block the merge, but a skip should not

If either of those conditions isn't met, the step runs exit 1, which fails the job and blocks the PR from merging. The required check on main now checks the pr-ready job rather than either of the other two, so it always has a clear, predictable result to work with.

pr-ready:
  runs-on: ubuntu-latest
  needs: [build, deploy-ephemeral]
  if: always()

  steps:
    - name: Verify required checks
      run: exit 1
      if: >-
        needs.build.result != 'success' ||
        (needs.deploy-ephemeral.result != 'success' &&
        needs.deploy-ephemeral.result != 'skipped')

Making it simple to use

I wanted to make it as easy as possible for developers to skip an ephemeral environment, should they want to. When creating a PR, all they need to do is add the label skip-ephemeral to the PR. When the PR is opened, the GitHub workflow will start, read the labels assigned to the PR, and skip the deploy-ephemeral job. The build still runs as expected, and the pr-ready job will execute, which is the job being tracked by required checks for PRs.

:::figure

::figcaption[Adding the label during PR creation.]

:::

It's worth noting that the label needs to be present before the workflow triggers. If you open a PR without the label and the workflow starts running, adding the label mid-run won't affect that run. The label state is read from the event payload when the workflow was triggered. If you push a subsequent commit, the workflow will re-trigger and read the current label state at that point, so the skip will take effect from your next push onwards.

One change, every team benefits

One of the things I enjoy about working with a Platform Engineering mindset is that the improvements I make to shared infrastructure benefit everyone who relies on it. In this case, I was responding to a request from one of our application teams, but because our microsite build process is templated and shared across multiple teams, the change I made is immediately available to every team using the same workflow. At the same time, the platform team can also decline a feature request. If skipping ephemeral environments conflicted with the organization's risk, compliance, or deployment policies, we could simply choose not to implement it, and every team consuming the shared workflow would remain compliant by default.

This is the value of treating your pipelines as shared, versioned infrastructure rather than copies of a golden template. When every team has its own copy of a pipeline, improvements stay local to the team, making it harder for the platform team to push changes out. When teams consume a shared workflow, a single change propagates everywhere, everyone benefits, and the platform team can focus on building improvements rather than coordinating rollouts.

Wrapping up

Ephemeral environments are a powerful tool for reviewing changes in a real environment before they're merged, but there are scenarios where developers may want to skip the ephemeral environment process. It's not something I'd lean towards myself, and I definitely wouldn't make that the default behavior.

By breaking the workflow into separate jobs and using a simple-to-use PR label as a condition, it's possible to make the ephemeral environment deployment optional without compromising the integrity of the build process or the branch protection rules that rely on it.

Happy deployments!

In 2024, we launched the Octopus Deploy App in the GitHub Marketplace. It has been widely adopted, unsurprisingly, as it is the most seamless way to connect Octopus Deploy and GitHub. At launch, however, it did not support self-hosted Octopus Server customers.

This means until now, if you self-host your Octopus instance and you integrate with GitHub, you've been doing it with Personal Access Tokens (PATs). PATs work, but they've never been ideal for a critical integration like this, suffering from a number of problems:

• Someone leaves the team and their PAT goes with them
• A token quietly expires at 2am and a deployment falls over
• PATs are user-scoped, long-lived, and often have more permissions than they strictly need

We've received plenty of feedback requesting support for the Octopus Deploy App for GitHub from our self-hosted customers, and we're happy to say that as of release 2026.2 it's available.

What's shipping

GitHub Connections are now available for self-hosted Octopus Deploy. You can install the Octopus Deploy App for GitHub, connect it to your self-hosted instance, and use it anywhere the product currently requires GitHub credentials.

Under the hood, GitHub Connections use OpenID Connect (OIDC) to exchange a signed token from your Octopus instance for a short-lived GitHub token, scoped to the repositories you've granted the app access to. There are no long-lived secrets to store, rotate, or leak. The permissions are managed in GitHub against the app installation, not against a user account. When someone leaves the team, nothing breaks.

What about Octopus instances behind a firewall?

This is the part that took the most thought.

OIDC requires GitHub to fetch the public signing keys from your Octopus instance to verify the tokens it issues. That works fine on Cloud, and it works fine on a self-hosted instance that's reachable from the public internet. It doesn't work if your instance lives behind a corporate firewall, on a private network, or otherwise isn't reachable from github.com, and this is true for many self-hosted installs.

So we've added a second option. On the Signing Keys settings page, you'll find a new Externally Hosted option. You generate your signing keys in Octopus as usual, then publish the public keys to any web host you like, such as an S3 bucket or Azure blob storage, whatever fits your infrastructure, and tell Octopus the OIDC issuer URL where GitHub can fetch them. Octopus signs tokens with that issuer URL, GitHub validates them against the published keys, and your private instance never has to take an inbound connection from the internet.

The signing keys docs walk through the options.

Getting started

GitHub Connections are available to all self-hosted customers from 2026.2. See the GitHub Connections docs for the details of how to configure the integration between Octopus Deploy and GitHub, without Personal Access Tokens.

:::div{.hint} Note: GitHub Connections work with repositories hosted on github.com. They don't yet support GitHub Enterprise Server (self-hosted GitHub). If your repositories live on GitHub Enterprise Server, you'll need to keep using Personal Access Tokens for now.

If you use GitHub Enterprise Server, and this is important to you, please register your interest on the roadmap card. :::

Happy deployments!

With the release of Argo CD v3.x the project has finally taken a clear stance on secret management. Today, the documentation page clearly recommends external secret controllers over manifest generation plugins. At the time of writing, the following solutions are recommended:

• Sealed secrets
• External Secret Operator
• Kubernetes Secret Store CSI driver

We've already covered the first two solutions in our previous guides. Today, we'll complete the trilogy by showing how you can secure your Argo CD secrets with the Secret Store CSI driver.

Using an external secret solution

Before we dive into the secret store driver, it's important to understand the concepts behind GitOps Secrets. Even though in theory there are several different secret solutions, in reality most of them fall under two categories:

1. Secret solutions that are implemented by external controllers without any coupling to Argo CD
2. Secret solutions that are tied to Argo CD and the manifest generation process

The first category is what the new Argo CD documentation recommends. This is the case where two different tools handle secret management and application deployments with completely different lifecycles.

:::figure

:img{ src="/blog/img/gitops-secrets-csi-driver/approved-gitops-secrets.png" alt="Approved GitOps secrets" }

:::

In this setup, Argo CD doesn't even know what the real secret values are. It deploys the application with several secret references/pointers. An external secret operator is then responsible for converting secret references into real secrets. Each tool does what it does best and more importantly, any secret rotation process doesn't need application re-deployments or sync operations in Argo CD.

On the other hand, we have secret solutions that are tied to Argo CD, either in the form of plugins or any other tools that modify manifests in place when a sync operation takes place. These are not recommended by the Argo CD team.

:::figure

:img{ src="/blog/img/gitops-secrets-csi-driver/reject-gitops-secrets.png" alt="Rejected GitOps secrets" loading="lazy" }

:::

Having a secret solution tightly coupled with Argo CD forces Argo CD into the realm of secret management. Not only does it make Argo CD aware of the actual secret values, but any kind of rotation needs to pass through Argo CD and its synchronization process.

The secret store CSI driver is a solution that follows the first approach and thus enjoys the implicit approval from the Argo CD team.

Avoiding Kubernetes secrets by mounting files directly on pods

If you have read our previous guide or you already know how the external secret operator works, you might wonder what's the advantage of using the Secret Store CSI driver. At first glance, both approaches work in a similar way

• Both solutions are decoupled from the Argo CD sync process
• Both fetch secrets from an external secret source and bring the secrets into the Kubernetes cluster
• Both use a secret pointer/reference CRD
• Both support mounting secrets as files on the workload pods
• Both can automatically refresh the files when a secret changes

The biggest difference between them is that the CSI driver does not use Kubernetes secrets.

:::figure

:img{ src="/blog/img/gitops-secrets-csi-driver/gitops-secrets-comparison.png" alt="GitOps Secrets comparison" loading="lazy" }

:::

The External Secret Operator always uses Kubernetes secrets, which are then mounted as files using the standard Kubernetes volume mechanism. The CSI driver completely bypasses this step and mounts the secrets directly as files without any Kubernetes secrets on the cluster.

Whether this feature is important to you depends on your security policy. Several companies have different security constraints and requirements that affect how their workloads run in a Kubernetes cluster. If part of the security policy is to never have Kubernetes secrets anywhere, then the CSI driver is a better option for you.

The end result in both cases is the same, the secrets reach your Argo CD application as files and are automatically rotated if the secret provider changes them. In all cases, we assume that your application is loading its secrets from files and not environment variables.

So if you already have a working solution with the External Secret Operator and your security team is happy with it, there's no real technical reason to adopt the CSI driver.

The example secret application

You can find the source code and Kubernetes manifests for our application in the argocd-csi-secret-store-example repository on GitHub.

It's a very simple application that reads a database connection credential from /secrets. There's no real database of course. The application just prints its own secrets so that you can see the flow of information from the secret store to the Kubernetes pods

:::figure

:img{ src="/blog/img/gitops-secrets-csi-driver/02_app_before_rotation.png" alt="Simple GitOps application with Secrets" loading="lazy" }

:::

Even though this is a demo application, it has several important characteristics that help with secret management.

1. It loads secret information from files and not environment variables.
2. It shows where it loads secrets from. This makes debugging very easy.
3. It automatically reloads secrets if they change. We'll see this later in the secret rotation scenario.

Installing HashiCorp Vault and the CSI secret store driver in your Kubernetes cluster

Let's start by deploying our "infrastructure" applications first. You can install Vault using the public Helm chart.

argocd app create vault \
--project default \
--repo https://helm.releases.hashicorp.com \
--helm-chart vault \
--revision 0.28.0 \
--sync-policy auto \
--sync-option CreateNamespace=true \
--parameter server.dev.enabled=true \
--parameter injector.enabled=false \
--parameter csi.enabled=true \
--dest-namespace vault \
--dest-server https://kubernetes.default.svc

Just for this demo we install Vault with server.dev.enabled so that we don't deal with sealing/unsealing. The default admin token is "root" and we'll use it later to log in to the web UI. We also pass csi.enabled=true for CSI integration.

In a production environment, Vault should be handled by your security team and have proper credentials. We create applications using the Argo CD CLI for simplicity here. In a production setup, all applications should be stored in Git. See anti-pattern 2 in our comprehensive Argo CD guide.

The next step is to create the credentials that the application is using. In this contrived example, it's database credentials for an imaginary MySQL instance.

You can do this from the vault CLI or from the Web interface:

:::figure

:img{ src="/blog/img/gitops-secrets-csi-driver/03_vault_secret_view.png" alt="View Secret in Vault" loading="lazy" }

:::

This concludes the "demo" installation for HashiCorp Vault. As a reminder, Vault is just one of the many providers supported by the CSI driver.

Fetching external secrets from HashiCorp Vault

With the vault installation ready, we can now install the CSI driver. This step depends on how you create your Kubernetes cluster and on any Infrastructure-as-Code tool you already use.

See also the official installation instructions.

The last piece of the puzzle is to actually set up the integration between the CSI driver and vault. This happens via a dedicated SecretProviderClass. See the example in our application repository.

You can easily deploy it like any other Argo CD application:

argocd app create vault-secret-store \
--project default \
--repo https://github.com/kostis-codefresh/argocd-csi-secret-store-example.git \
--path "./manifests/vault-integration" \
--sync-policy auto \
--dest-namespace default \
--dest-server https://kubernetes.default.svc

Notice that, like the external secret operator example, Vault is set up to trust the Kubernetes cluster it is running on. There's no other token or secret stored anywhere (either in Git or the application itself)

Everything is ready now regarding secret management. Vault is running, and the CSI driver connects to it to fetch secrets.

Passing secrets as normal files without Argo CD involvement

Finally, let's deploy our application. It's very similar to the one we used in the External Secrets Operator post. It's a very simple Web application that prints a "secret". Again, you can do this with Argo CD and verify that everything is synced and healthy:

:::figure

:img{ src="/blog/img/gitops-secrets-csi-driver/01_argocd_dashboard.png" alt="Argo CD dashboard" loading="lazy" }

:::

Now if you visit the Web UI of the application (using port forwarding or any other networking method), you'll see the full result. An application reading secrets from Vault without any hardcoded token anywhere.

:::figure

:img{ src="/blog/img/gitops-secrets-csi-driver/02_app_before_rotation.png" alt="Simple GitOps application with Secrets" loading="lazy" }

:::

This is how the whole process works:

1. Vault is running inside Kubernetes and also has a trust relationship with the same cluster.
2. The CSI driver is also active on the same cluster and can read secrets from Vault.
3. The secrets are mounted directly as files under /secrets.
4. The application is just reading files without any knowledge of how/where these secrets are stored.

The main advantage of this setup is security and simplicity:

• There are no Kubernetes secrets in the cluster
• There are no application tokens that are used for vault access
• The source code of the application just reads files at /secrets
• The communication between the vault and the CSI driver is transparent to the application

We've now explained the initial deployment of the application. But what happens when a secret needs to be rotated?

Refreshing secrets without restarts and any Argo CD sync operations

The Secret Store CSI driver has a similar capability to the external Secret Operator for automatically refreshing secrets without application restarts. This means that you can change a secret value, and if the application is correctly wired it can auto-reload the secret on its own.

We can test this scenario easily by changing the secret values in Vault (using the CLI or the Web interface).

:::figure

:img{ src="/blog/img/gitops-secrets-csi-driver/03_vault_secret_edit.png" alt="Update GitOps Secret" loading="lazy" }

:::

After the secret changes in the vault, the Secret store driver detects it (the refresh period is configurable) and can automatically update the secret's file contents in the mounted folder.

For the application, everything is transparent. The source code just sees a file change on the filesystem. Our example application is already configured correctly to automatically reload secrets on the fly.

:::figure

:img{ src="/blog/img/gitops-secrets-csi-driver/04_app_after_rotation.png" alt="Secret rotation" loading="lazy" }

:::

Secret rotation works with zero effort and without any involvement from Argo CD. No sync operation is required, or killing pods manually. One of the biggest challenges in a big organization when it comes to secret rotation is not only understanding which applications use a specific secret, but also how exactly to make the applications "see" a secret change.

With the CSI Secret store driver, secret rotation is straightforward. There's no need to hunt down individual applications for restarts or missing an application that was never part of the rotation process.

If your application can't refresh secrets on its own, you can use any external controllers such as Reloader to achieve the same result. The CSI Secret store driver can be configured to use Kubernetes Secrets as a configuration option. Secret rotation can then follow the traditional process where you must restart applications in order for the new secret to take effect.

Conclusion

In this guide, we've explained how to use the CSI Secret driver for secret management of Argo CD applications. We've seen how you can rotate secret values without any sync operations or application restarts.

We've now covered the full trilogy of how to handle secret management with Argo CD applications. The updated Argo CD documentation now has a clear recommendation, and there's a comprehensive guide for each one of them

• Secret Store CSI driver (this post)
• External Secret operator (previous post)
• Sealed Secrets controller (previous post)

If you need help in your Argo CD onboarding process or just need advice on how to improve your Argo CD promotion workflows, check out our Enterprise Argo support offering.

Happy deployments!

When they actually do, the plan is typically a runbook in a wiki, or the fact that, in theory, we can always revert to the last working commit. When you run Kubernetes, Argo CD is instrumental in keeping your resources in sync.

Argo CD is good at reconciling whatever's in your Git repo to your cluster. When a deploy goes sideways, "roll back" turns into "revert the right commit, in the right repo, without breaking the other applications syncing alongside it", which is fine on paper and miserable in practice.

In this post, we'll explore how to manage releases in Octopus Deploy, how it integrates with Argo CD, and discuss, as well as implement, a rollback.

What Argo CD doesn't do (and why it matters)

Argo CD is great at one job: take whatever's in your Git repo and reconcile a cluster to match it. It'll show you what's drifted, when it last synced, and which resources are unhealthy.

However, what it deliberately doesn't do is anything above the cluster.

There's no concept of a release. Each Application is an independent entity, and its current state is whatever commit HEAD happens to be on. If Tuesday's image tag turns out to be a regression, your options are: revert the commit in Git and wait for the next reconcile, or open the Argo CD UI and sync to a previous revision by hand.

More importantly, the processes around what constitutes a release that can be synced are something Argo simply wasn't built for.

How Octopus manages releases with Argo CD

A release in Octopus is a versioned snapshot of three things:

• the deployment process (the steps it runs)
• the variables those steps use, and
• the package or image versions they reference.

Once a release is created, it's frozen. You can promote 0.0.2 through Development, then Staging, then Production, and each environment receives the same release, not whatever happens to sit at HEAD when you trigger the deployment.

With the Argo CD integration, that release model integrates into your existing GitOps setup. When you deploy a release in Octopus, you can have an Argo CD step that rewrites the image tags in your repo, commits the change with the credentials you've attached, then asks Argo CD to sync. Argo CD keeps doing the reconciliation while Octopus keeps the lifecycle.

Prerequisites

This tutorial assumes you have:

• A working Kubernetes cluster (any flavour)
• kubectl installed and pointed at the cluster
• The GitHub CLI (gh) installed and authenticated (gh auth login). We'll use it to create the demo repo in one command
• An Octopus Cloud (or self-hosted) instance, with the Argo CD Gateway installed in the cluster and an Argo CD instance already registered against Octopus. If you haven't done that yet, follow the Connecting AWS EKS Argo CD to Octopus Cloud guide first; the same flow works for any cluster.
• A Git Personal Access Token (PAT) scoped to your demo repository, Octopus will use it to push image-tag commits back

Step 1: Create the Octopus project

1. Sign in to Octopus and head to Projects > Add Project.

2. Give it a name (we'll use argo-demo), pick Kubernetes as the deployment target, and answer Yes to Are you using Argo CD?

3. Leave the lifecycle on Default Lifecycle.

   :::figure

   

   ::figcaption[Creating the argo-demo project.]

   :::

4. Hit Create Project.

Step 2: Add the Argo CD step

You'll land on the Process editor.

1. Click Add Step and filter by Containers and Orchestration > Argo CD in the left sidebar. You'll see two installed step templates:
   • Update Argo CD Application Image Tags, which updates an image tag in your manifests and commits the change for Argo CD to sync
   • Update Argo CD Application Manifests, which generate manifests from templates using Octopus variables, and commit those.

We'll use the first one for this walkthrough.

:::figure

::figcaption[The Argo CD category in the step picker. Both templates ship with Octopus once an Argo CD instance is registered.]

:::

Step 3: Set up the manifests repository

Argo CD syncs from Git, and Octopus pushes image-tag updates back to that same repo, so before either side has anything to do, we need a repository with a workload manifest in it.

We'll keep the layout small: one Deployment, one Service, and the Argo CD Application that wires the two systems together.

argocd-octopus-demo/  
├── application.yaml      \# the Argo CD Application (apply once to the cluster)  
└── manifests/  
    └── app.yaml          \# workload Argo CD reconciles from this repo

1. Create the directory locally:

   mkdir -p argocd-octopus-demo/manifests
   

   cd argocd-octopus-demo
   

2. Write the workload manifest. We're using nginxdemos/hello:plain-text, a tiny nginx that responds with a text page naming the pod that served the request. It's a single-container app with no state, perfect for proving reconciliation worked end-to-end.

   cat > manifests/app.yaml <<'EOF'
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: hello
     labels:
       app: hello
   spec:
     replicas: 2
     selector:
       matchLabels:
         app: hello
     template:
       metadata:
         labels:
           app: hello
       spec:
         containers:
           - name: hello
             image: nginxdemos/hello:plain-text
             ports:
               - containerPort: 80
             resources:
               requests:
                 cpu: 25m
                 memory: 32Mi
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: hello
   spec:
     type: ClusterIP
     selector:
       app: hello
     ports:
       - port: 80
         targetPort: 80
         protocol: TCP
   EOF
   

3. Initialize the repo, create a public GitHub repo from it, and push the first commit:

git init -b main

git add .

git commit -m "chore: init"

gh repo create <your-org>/argocd-octopus-demo \
  --public \
  --description "Argo CD + Octopus Deploy demo manifests" \
  --source=. \
  --push

With the repo live, we can write the Argo CD Application that points at it.

Step 4: Scope the Argo CD Application with annotations

The Octopus gateway pulls the full list of Argo CD Applications from your cluster, but it doesn't yet know which of them belong to this Octopus project. You can tell it with three annotations on the Argo Application:

• argo.octopus.com/project[.<source-name>]
• argo.octopus.com/environment[.<source-name>]
• argo.octopus.com/tenant[.<source-name>]

The .<source-name> suffix matches spec.source.name on the Application, which lets you scope different sources of a multi-source Application to different Octopus projects.

For our argo-demo project and Development environment, an annotated Application looks like this:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: argo-demo
  namespace: argocd
  annotations:
    argo.octopus.com/project.argo-demo-source: argo-demo
    argo.octopus.com/environment.argo-demo-source: development
spec:
  project: default
  source:
    repoURL: https://github.com/<your-org>/argocd-octopus-demo.git
    targetRevision: HEAD
    path: manifests
    name: argo-demo-source
  destination:
    server: https://kubernetes.default.svc
    namespace: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

The two annotations there tie this Application to the argo-demo Octopus project, scoped to the development environment. The suffix .argo-demo-source matches the spec.source.name field below it.

Apply it:

kubectl apply -f application.yaml

Within a few seconds, the Octopus gateway picks up the new annotations, and the Application appears in the project's deployment preview.

Step 5: Preview the deployment

Back in Octopus, in the Process tab, click on the step you just added.

:::figure

::figcaption[Octopus has discovered the argo-demo Application via the annotations]

:::

Annotations have been discovered, but flags that the source repo is missing Git credentials. Click View Applications to see what was matched.

The deployment preview opens. You should see the argo-demo application listed under the Rollback Argo CD instance, scoped to the Development environment, with the source repo URL underneath.

Step 6: Connect Git credentials

Octopus needs Git credentials so the step can push the new image-tag commit.

1. Click the Connect Git Credential dropdown next to the repo URL.

   :::figure

   

   ::figcaption[Two paths: spin up a new credential in Octopus from a PAT, or reuse one you've already attached to this space]

   :::

2. Pick Create a new Git credential (or Connect an existing Git credential if you've already added one to the space).

Once the credential is attached, the warning on the deployment preview clears and the source shows up as ready to update.

Step 7: Configure the step

Back on the step, fill in the rest:

:::figure

::figcaption[Confirm your step settings look good]

:::

The key fields:

• Step Name, leave as the default unless you have multiple Argo CD steps in one process.
• Container Images, click Add to declare which container image(s) this step should bump. You'll point it at a package/container reference so Octopus knows which image and which feed to pull tags from.
• Git Commit Settings, the commit author/email Octopus uses when it pushes the tag update.
• Commit Message, supports Octopus variables, so something like chore: bump image to #{Octopus.Action.Package[hello].PackageVersion} is a good default.

Save the step.

Step 8: Create a release

Head to the project dashboard, and you should now see the Create Release prompt.

:::figure

::figcaption[The project dashboard once the deployment process has at least one step. Hit Create Release.]

:::

On the release form, give it a version (Octopus auto-suggests 0.0.1) and click Save.

:::figure

::figcaption[Create release page]

:::

Release notes are optional but recommended for anything you'd want a changelog entry for later.

Step 9: Deploy to Development

Pick Deploy to Development on the release page. Octopus kicks off the deployment, and the task log streams the three sub-steps in real time:

1. Update Argo CD Application Image Tags, Octopus rewrites the image tag in the manifest and pushes the commit
2. Triggering Argo CD application syncs, Octopus asks Argo CD to sync the Application
3. Wait for Argo CD Application Health, Octopus polls the Application until it reports Healthy

When all three turn green, you'll see a completion line in the task log, and a Deploy to Staging… button appears at the top right, ready for the next environment in the lifecycle.

:::figure

::figcaption[Deployment completes in just over 90 seconds]

:::

You can verify the rollout landed on the cluster side too:

kubectl -n default get deploy hello -o
jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'

Step 10: Rolling back: redeploying a previous release

The deploy half of the demo is done. The other half is the one that actually earns its keep: a release that turns out to be bad, and how Octopus takes you back to the previous one without anyone opening a terminal.

To simulate it, hit Create Release again, save as 0.0.3, and deploy it to Development. Imagine the new image is broken (regression, missing env var, whatever). To roll back, head to Releases in the project's left sidebar.

:::figure

::figcaption[Every release Octopus has ever cut for this project, newest first. Each one is a versioned artifact you can revisit and redeploy. To roll back, open the last known-good release, 0.0.2]

:::

Click 0.0.2 to open it, then look at the lifecycle table. The Development row will show its previous deployment with a Redeploy… button.

:::figure

:::

Click Redeploy…, confirm, and the same three sub-steps run again.

Octopus rewrites the image tag in Git back to the one 0.0.2 used, commits it, asks Argo CD to sync, then waits until the Application reports Healthy.

When the task completes, the project dashboard shows 0.0.2 back in Development.

:::figure

::figcaption[The dashboard after the rollback shows a healthy application.]

:::

When to roll back vs when to roll forward

For this demo, rolling back was a single click. That doesn't mean rolling back is always the right call. Rollbacks have a way of failing that isn't always talked about.

They assume the rest of the system is in a state that the old version can still talk to. Most of the time it is. Sometimes it very much isn't.

Take, for instance, a database. If a bad release adds a new column and your users have been generating data through that path for an hour, rolling the application code back doesn't unwind the schema or the data. The old app meets the new schema and either ignores fields it doesn't know about (best case) or crashes on them (worst case)

Refactoring databases buys you flexibility. If schema changes go in through additive, backward-compatible migrations (add columns nullable, write to both old and new fields for a release cycle, then drop the old in a later release), rollbacks stay safe much longer.

Make rollbacks boring

If you made it this far, you've got the full loop: an Argo CD Application scoped to an Octopus project with annotations, a deploy that flows from a click down through Git into your cluster, and a redeploy that takes you back to a known-good release the same way.

Argo CD keeps reconciling, your manifests stay the source of truth, and Octopus wraps versioning, approvals, audit, and that redeploy button around them. The rollback nobody had tested becomes a button two engineers know how to press.

Beyond manual rollbacks, Octopus enables you to build out full lifecycles on your deployments, so this can all be automated conditionally.

Extend the pipeline to your real environments using the Argo CD integration docs, or if you don't have an Octopus instance yet, start a free trial and run this end-to-end on your own cluster.

Even for teams that can effortlessly deploy their application code, database changes can be more stressful. Changing a schema is a high-stakes operation, and there are many ways to do it badly.

Read on to find out why database deployments are different from application deployments, and what techniques you can use to make them worry-free.

Watch the episode

You can watch the episode below, or read on to find some of the key discussion points.

Watch Continuous Delivery Office Hours Ep.5

Why are databases different

When you change your application's code and discover an issue, it's trivial to revert to the previous version. In rare cases where a change affects your data, there may be cleanup to do.

You can use timestamps and a one-off process to fix those rare data-mess-up instances, but they hint at something fundamentally different about database updates. They have different levels and types of risk associated with them.

If you think backups will save you from a bad database deployment, you haven't yet tried it. Sure, they prevent total data loss, but between your backup and your fix, the data moved. Often by a lot.

There are techniques to apply transactions from the backup up to the point of failure, but if your change caused an issue you needed to roll back, you probably don't want to apply those transactions automatically. Welcome to the "data remediation project".

If you added a new column or table as part of your database change and you need to roll back, you have to decide what to do with any data in those tables. Do you forget it, or do you need to keep hold of it and reapply it later when you make a new attempt to extend the schema?

Modern software teams prefer fix-forward for application issues, which are reasonably easy to roll back. Databases take rollbacks to another level.

Crucial modernization steps

Imagine you met a friend for lunch and they told you they store they application's source code on a network share, rather than in version control. You'd think it was a joke, and when you realize it's not, you'd form a strong opinion about the kind of sloppy outfit they must be running.

When we meet for lunch, what are you going to tell me about your database schema and static data? Please tell me it's all in version control, not on a network share.

You should make all database changes by updating the files in version control and deploying them like you would your application code. You progress the change through environments to ensure it works, and you avoid embarrassment caused by the application failing because someone forgot to add the new column in production.

There are further choices to make, which we'll cover next, but failing to version-control your database is unforgivable.

State-based vs migration-based approaches

On to the first choice for your database project. Do you make state-based or migration-based updates?

State-based schemas describe the desired state of the database. It will list each table with its columns, indexes, and relationships. You use a model-based tool to deploy the database, which compares your current state with the desired state and applies the changes for you.

Some state-based tools convert the differences into standard database scripts, like ALTER TABLE... scripts. Others perform migrations by creating a new table with the changes and moving the data into it. This is important if you're using a technology like replication, which prevents the model migration mechanism from working.

The alternative to state-based database updates is migration-based updates, where you write your own ALTER TABLE scripts. You keep all your scripts in version control and use a tool that applies them in order and tracks when each was applied, preventing the same script from being applied twice to the same database.

The main difference between the two is procedural. You can code-review the migration scripts on demand, but you'll need to review state-based migration scripts after your tools generate the implementation plan, which can make the review task more of a large batch.

Tooling and automation

Whichever approach you use, tooling helps it work. Many of our customers use Redgate tools as part of their deployment process to manage database schema changes, and there's value in leaning on a tool written by folks who care deeply about the problem and how to solve it.

Crucially, you shouldn't make any database changes outside of tool-based automation.

Test data management

Once you've automated your database schema and static data, it's worth considering your test data. When automated or manual acceptance tests fail, it's usually because someone unwittingly messed up the data. Prior test runs often leave data in an inconsistent state, especially if a test failure halts the run.

You can resolve this issue by automating your test data setup. Not only does this make it easy to reset the data during your build and test cycle, but it also lets you provide a self-service runbook for the test team to reset the data in their test environment whenever they need to.

There's an up-front investment in this, but I can promise that it takes fewer hours than fixing your test data a few times.

Database refactoring patterns

The final thought to ponder concerns the steps you take in changing your database schema. When you're in the habit of deploying your database and application in the same release process, you start to depend on this change coordination.

You delete a column from the database and immediately deploy the application, with all references to the deleted column removed. It looks like it works smoothly, but it's a trap.

Imagine you had a critical bug in the application and had to redeploy the previous version. Now you can't, because the previous version will try to read from a column that doesn't exist. You no longer have a quick, easy back out plan, since you have to re-add the column, and you'll also need data to put in it.

This approach also prevents you from making seamless deployments, as even if you progressively roll out the application version, the old version will error out due to the database change. You may deploy in the opposite order, application first, then database. You'll discover the same problem with new columns that the latest version expects to find in the database.

You need to decouple database and application deployments, and there's a whole book on the topic, called Refactoring Databases (Ambler, Sadalage). You can start by following the expand/contract pattern, which splits updates into steps. The principle is that you don't delete a column until the production application has no reads or writes. You add a column and don't reference it in your code until you deploy to production.

This means you can run the current version and the new version of the software against the same database, which means you can progressively roll out the new version and redeploy the prior version without touching the database.

Databases are only as hard as you make them

The database is high-risk, which is why updating it can be scary. I hope you've found this post full of practical advice for making database deployments robust and stress-free.

Database deployments, like application deployments, should be a happy time. You should be celebrating the new features and enhancements you've delivered to your users, not biting your nails and worrying that something's about to go horribly wrong.

Happy deployments!

:::div{.hint}

Continuous Delivery Office Hours is a series of conversations about software delivery, with Tony Kelly, Bob Walker, and Steve Fenton.

You can find more episodes on YouTube, Apple Podcasts, and Pocket Casts.

:::

That's why the strategic partnership between Octopus Deploy and GitHub is so significant.

While many organizations view AI as a coding assistant, GitHub and Octopus are helping teams unlock something more powerful: an AI-enabled software delivery platform that connects development, deployment, operations, and most importantly governance into a single intelligent workflow.

As partners, we made steps in this direction by contributing to major innovations such as Copilot Extensions back in 2024, then Custom Agents in 2025 and now being again a launch partner for Agent Apps within GitHub.

No more context switching and more control over your agents

The launch of Agent HQ at GitHub Universe made it already possible to access coding agents from Anthropic, OpenAI, Google, Cognition and xAI directly within the GitHub interface. Because regardless of how powerful they are, more agents will not improve efficiency, but rather generate complexity and loss of control unless properly governed.

But writing code is only one part of the journey. The real challenge for enterprise teams begins after the pull request is merged:

• Understanding deployment status
• Managing complex release pipelines and troubleshooting production issues
• Coordinating multi-tenant deployments and identifying issues while operating software at scale
• Maintaining security and compliance controls

Historically, these activities have required developers to switch between multiple tools, dashboards, and operational systems. But every context switch creates friction, every disconnected workflow slows delivery or, even more importantly, delays recovery when something goes wrong.

This is precisely the problem Octopus and GitHub are solving together by bringing the Octopus Intelligence to Agent HQ . When deployment information becomes accessible through natural language, organizations can unlock knowledge that was previously only accessible to specialist teams and take action quicker when necessary, avoiding downtime.

"The future of software development won't be shaped by any single agent, but by an ecosystem of specialized ones. Our partners are defining what that looks like by embedding Agent apps into the GitHub workflows where developers build and ship." Mario Rodriguez, Chief Product Officer at GitHub

By combining the Octopus MCP Server with specialized skills, the Octopus Intelligence Agent enables a seamless, automated workflow. This integration allows the agent to interact directly with Octopus Deploy from the GitHub interface, surfacing key data and executing tasks as directed.

Here is an example use case for the Octopus Intelligence Agent, directly within GitHub Agent HQ:

• Show the deployment status of a PR, and when it was deployed to a specific tenant
• Investigate and diagnose why it has failed
• Use the appropriate runbook to fix the failure
• Redeploy to tenant

See it in action! The demo video below shows how the Octopus Intelligence agent helps diagnose and recover from failed tenant deployments without leaving GitHub. When reviewing a PR, you can check the deployment status across tenants, investigate failures, and trigger the appropriate runbook to fix the root cause - then redeploy directly to the affected tenant once resolved.

Demo video

Our strategic partnership

Octopus Deploy and GitHub share a long standing committed partnership, focused on shaping a best-of-breed enterprise CI/CD landscape.

GitHub is where developers collaborate, build, and innovate. Octopus is where organizations orchestrate complex deployments across hybrid infrastructure, including Kubernetes, cloud-native platforms, or legacy systems, with enterprise governance in mind.

And we are not stopping there. By aligning our GitHub and Microsoft partnerships, we are able to bring the next level of DevOps innovation to our customers.

Together, we create a connected software delivery experience that helps teams move faster and securely despite the increasing complexity.

Learn more about our partnership with Microsoft and GitHub:

Microsoft case study video

If you are interested in using Octopus together with GitHub, you can now register for a free account - the agentic features are included! Octopus Deploy can be purchased via the Microsoft Marketplace and is Azure benefit eligible.

Happy deployments!

What are superseded tasks?

A task is superseded when a newer task makes it redundant. For example, if three deployments of the same project and environment are queued, the first two are superseded once the third is queued, as the third deployment will leave the environment in the desired state.

As AI coding tools push commit and PR volume upwards, a growing queue of superseded deployments becomes more pronounced

Instead of letting those redundant tasks run to completion, Octopus can automatically cancel them to expedite the latest task. The result is faster deployments, shorter feedback cycles and less resource use.

How does it work?

When a new deployment task is queued, Octopus checks for other queued tasks that target the same project and environment. If it finds tasks that are waiting to start, it cancels them and marks them as superseded.

:::figure :::

A few important things to note:

• Only queued tasks that are yet to start are considered for cancellation. If a task has already started executing, Octopus won't interrupt it mid-deployment.
• Tasks with skipped steps or included/excluded targets are not considered for cancellation since it's difficult to ensure the intended changes are still flowing through the deployment pipeline
• Superseded tasks are cancelled, not deleted. You can still see them in the task log with a clear explanation of what happened and which task superseded them.
• This applies to deployments and runbook runs where the outcome of the newer task makes the older one irrelevant.

Variation - cancel running tasks

As we were developing the Argo CD integration, we realized that Octopus could potentially have many deployments paused while waiting for pull request approval for the same project and environment. Once the latest pull request merges, the user would need to cancel the previous deployments manually.

This is why we've added the option to also cancel running tasks that are waiting for external events like pull request merges and manual intervention.

How do I configure this?

These settings are active by default for all new projects, you can customize the behavior for each project's deployment process or runbook by editing their respective settings.

:::figure :::

Learn more

For the full details on how superseded tasks work, head to the official documentation.

Happy deployments!

Most practitioners will be familiar with this. A script, a Lambda, a homegrown integration that filled a real gap at the right moment. It was built for good reasons, shipped, and is still running somewhere in your stack. Knowing when it's time to let it go is the hard part.

That moment usually comes when your vendor catches up, and it happens much more often than most people realize.

Why do we build our own things?

A friend recently posted something on LinkedIn that grabbed my attention:

:::figure :img{src="/blog/img/stop-maintaining-what-your-vendor-now-ships/sam-linkedin-post.png" alt="File this under: Why did this take so long to exist?! AWS Direct Connect now publishes BGP session health metrics natively in CloudWatch. Time to kill off my Lambda that's been creating these metrics using boto3." loading="lazy"} :figcaption[Source: Sam Bentley] :::

This is a story most practitioners will recognize. You're working with a tool you depend on, but it might lack functionality for something you need to achieve today, so you build around it. A script, a wrapper, in Sam's case, a Lambda, whatever it might be. You have a real need today, the vendor doesn't have an answer yet, and may not for many months or years, so you solve it yourself. There's nothing wrong with this approach; it's pragmatic engineering, and the right call.

Vendors can't prioritize every use case that every customer might have. Their roadmap has to serve a broad set of customers, which means your specific requirement might be well understood - and something the vendor wishes to solve - but it may not be next in line on their list of priorities. So you build something that fits your context and environment, ship it, and move on to the next problem.

Platform Engineers will recognize this

This is especially common for platform teams. They're often the ones filling gaps across a whole stack of tools being used by hundreds or thousands of developers and application teams. Most Platform Engineers will resonate with building integrations, abstractions, and internal tooling so the rest of the organization doesn't have to think about the seams between tools and has a great experience.

At Octopus Deploy, we see this across our customer base. One example that comes to mind is a customer who needed to ensure a specific deployment step was always present and enabled across thousands of projects. The problem will likely be familiar — developers, eager to get past a failing step and move on with their deployment, would disable or remove it entirely. In this case, skipping the step introduced a serious risk to the organization, as the step was critical to their auditing processes.

Octopus didn't have a great built-in answer for this at the time, so the customer wrote a script that interrogated the API, identified non-compliant projects, and flagged them for manual review. It worked for the purposes it was built for, but it was reactive, needed maintenance, and relied on one or two people who understood it well enough to keep it running if something broke or needed tweaking.

We've since introduced Policies as part of Platform Hub, an enforcement engine that handles exactly the scenario the customer was looking for, baked into the product proactively.

In both cases I've shared, the build was the right call. But eventually the vendor caught up, and that changed things.

The slow drift problem

When you ship a custom solution, the problem is solved, and life moves on. But quietly over time, it becomes part of the furniture. It runs on a schedule, or it sits in a repo somewhere, and nobody questions it because it works. The person who wrote it might still be around, or they might not. Either way, the context that existed when it was built starts to fade.

Meanwhile, the vendors you depend on don't stand still. They're shipping new features, adapting to the market, and building capabilities based on the feedback their customers have been giving them. That script you wrote, or the Lambda you set up two years ago to solve a problem they didn't have an answer for? There's a decent chance you weren't the only one with that problem to solve, and the vendor may have an answer for it now. But if you're not paying attention to what they're shipping, you'd never know.

The Octopus example is a good illustration of this. The API script that checked for compliant deployment steps across thousands of projects didn't stop working, and nobody rushed to replace it. But when Policies shipped as part of Platform Hub, the situation changed. The same requirement was now handled in the product, proactively, and without the maintenance overhead. The custom solution wasn't immediately a problem; it had just become unnecessary. Unnecessary code you're still running, maintaining, and depending on is a cost easy to overlook because nothing is visibly broken.

This is precisely where it matters for platform teams. A platform team's job is to make the engineering organization more effective, and that works best when the platform stays focused on the goals it is there to achieve. Taking on a custom solution to fill a vendor gap is good technical debt. It's a conscious tradeoff, made at a specific point in time to keep the platform moving. But technical debt has a repayment point, and when a vendor ships their own solution to something you've been solving yourself, that's usually the moment it arrives. Keeping the custom solution running past the point where it is needed is when good debt turns into bad debt, leaving you paying maintenance overhead and dealing with complexity that no longer needs to exist. The best platform teams recognize that moment and act on it. Let the vendor carry the maintenance burden, absorb the edge cases, and handle the support. This keeps your platform thin, lets you remove complexity, and frees your team to focus on what only you can build.

But none of this is possible if you don't know what your vendors are shipping. The repayment moment only arrives if you recognize it, and that's the part that's easy to miss because keeping up with every tool in your stack, across every channel each vendor uses, can feel like a job in itself.

Staying connected without it becoming a job

You can't spend your life reading vendor changelogs. That would be a full-time job, and you already have one. But there's a meaningful middle ground between obsessively monitoring every release and being completely unaware, and most of the time, good vendors are trying to meet you there. At the end of the day, vendors want you to know about new features and functionality, and start using their solutions to solve the problems you have.

At Octopus, we try to surface what we're shipping across a range of channels, and the intention is that at least one fits how you already work and where you prefer to consume content. Some examples of where we do this are:

• Monthly newsletter
• Blog
• What's New feed
• In-app notifications
• Webinars
• Deploy on Friday
• Social media
• YouTube
• Community Slack
• Roadmap
• Press room
• Shipped (yearly online event)

Most vendors are making a similar effort. The bar for staying loosely connected is lower than it probably feels.

One approach I've found genuinely useful is using AI to do the monitoring for me. There are a few ways to tackle this, but I've been using skills in Claude for this use case. You can get Claude to help you build a skill, using the built-in skill builder by saying something like "Can you help me build a skill?". From there, you can provide context of what you're interested in, and provide as many resources from the vendor as you can. I specifically note that I'm usually not interested in non-product-related posts (e.g., from social media feeds), and that any duplicate items found across multiple sources should be consolidated.

Your mileage will vary depending on how the vendor provides updates and how consumable those updates are for something like a large language model. Vendors who publish structured, well-maintained feeds like a dedicated changelog page, a What's New feed, and versioned release notes tend to work much better as sources than those whose updates are scattered across social media posts and marketing announcements. But within about 30 seconds, I can say "grab me the last month's updates from Octopus Deploy", and Claude uses the skill to build a response in the format I've specified, that I can easily scroll through. Claude is also pretty good at linking to the source if I want to explore something further.

Keeping pace

When a vendor ships a solution to something you've already solved yourself, moving to it is usually the right call. They've absorbed the edge cases, they own maintenance going forward, they've researched the feature's requirements and may build something you didn't already have, and you get to decommission something. Your team can focus on building things of value and importance to your organization, and you can push complexity down to vendors, creating a thinner platform that floats on top of the capabilities they provide.

My friend Sam was paying attention to what the vendors he uses are doing, saw the AWS announcement, and was able to decommission a custom solution in favor of AWS's built-in solution. It's a small habit - staying loosely connected to what the tools you depend on are shipping - but it's what put him in a position to act when the moment arrived.

When did you last look at what your vendors have been shipping? It's probably worth 20 minutes to find out.

Happy deployments!