Matt Blodgett

Don't Mess With Delivery

2026-03-25T00:41:00.000-05:00

If your team has consistent, reliable delivery of working software to production, you’re crushing. Don’t mess with it. It's astonishing how many teams in real world software development can't do this. I've witnessed it over and over and over in my career.

If you have a backlog, sprint planning, a dependable QA function (even if manual and/or slow), and scripted deployments on-demand, you are very fortunate.

Of course, we don't want our engineers working in a feature factory, so we make sure they have input into what they're building and they know why they're building it. That's the motivation to keep delivering consistently.

So how do teams "mess with" delivery? For teams that already have consistent delivery, the way they tend to mess it up is by prioritizing speed over consistency.

I think that continuous deployment is an incredible technical capability to possess. It's amazing to have a fully automated pipeline capable of taking a code commit and moving it fully out to customers without manual intervention. This is huge for production bugs and emergency fixes.

My belief is that feature work should prioritize ease of communication over raw speed of deployment. Just because it's technically possible to deploy new features every day doesn't mean it's actually beneficial to users. I've seen firsthand the chaos that results when new features appear in front of users without people beyond the immediate team knowing about them.

Every organization wants to "go faster", but I believe once you're touching production, communication matters more than speed. If you're consistently getting new features in front of users every two weeks, you're already doing so well. Optimize something else.

If you've got a reliable, dependable, consistent pipeline of new features to real users on a reasonable sprint duration, it's not worth the communication overhead of going out-of-band. Please don't mess with delivery.

User Advocate vs. Front-End Engineer

2026-02-24T22:53:00.000-06:00

I really enjoy doing UI work: being close to the user's mind, thinking deeply about interaction design, and how small changes can massively improve someone's day. But in order to market oneself as a “Front-End Engineer” in the current landscape requires dogged attention to fashion trends, and I'm on Team Evergreen, baby.

Early in my career in web development, I loved reading Jakob Nielsen and Steve Krug, but at some point I had to accept that maintaining intimate awareness of the specifics of JavaScript Framework Du Jour was not sustainable for me. Call me "full-stack", that's fine.

Front-end is the most ripe for résumé-driven engineering. Default skepticism is always warranted for new technologies, but in front-end, it is truly a necessity to avoid lighting your company's money on fire.

I found a rather cathartic post by Marco Rogers on Hacker News recently called The Frontend Treadmill. I fully agree with Marco's take:

If you are building a product that you hope has longevity, your frontend framework is the least interesting technical decision for you to make. And all of the time you spend arguing about it is wasted energy.

I would not necessarily describe myself as a React fan, but I am very happy that it feels like we finally landed on a framework that can survive for several years, with wide adoption by startups and enterprises alike. Let's go, Lindy effect. Maybe React can be the framework equivalent of what jQuery did as a library.

Above All Else, Sustainability

2026-01-28T00:44:00.000-06:00

From the principles of the Agile Manifesto:

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Extreme measures are always doomed beyond a limited time horizon. Whether we're talking about diets or software development practices, without sustainability you have nothing.

Initiatives to increase productivity, improve quality, lower costs, or any other "good thing" simply don't matter if they're not sustainable.

A common topic in Agile circles is "sustainable pace", which usually focuses on the futility of working overtime, typically over 40 hours a week. I would argue that sustainable pace is about more than just the number of hours clocked in a work day or work week.

Is the work emotionally sustainable? Do people hate working here? Do people end the work week feeling a sense of accomplishment? Do people feel like their leaders have reasonable expectations? Are they constantly battling reality?

Sometimes the degree to which an enforcer must continuously apply pressure to get people to follow a process indicates how sustainable the process is. Powering through is not a sign of discipline, it’s a sign of delusion.

Ignorance of reality is unsustainable. Coercion is unsustainable. Surveillance is unsustainable.

Can I and the people around me keep this up forever? If not, it's time to pause and reflect. Above all else, sustainability.

Agile as Progressive Enhancement

2025-12-08T23:08:00.000-06:00

When I was coming up as a young web developer. the concept of progressive enhancement was kind of at the hipster vanguard of web development. This was in the phase of the web where JavaScript and CSS feature parity amongst browsers could not be taken for granted like it is today.

The idea of progressive enhancement was that you started by making your web application work on an essential informational level with semantic markup, then you layered on presentational and dynamic niceties with CSS and JavaScript respectively, if available to the user agent. At each layer, or each stage, you left the user with something usable and valuable that stood on its own without the other stuff.

Learning about the concept of progressive enhancement early in my career colored my interpretation of the Agile philosophy forever. The iterative nature of Agile methodologies, with "Working software [as] the primary measure of progress" is essential to how I approach making software for users to this day. In my mind's eye, I still visualize iteration as progressive enhancement. We start with the essential core of value, and ship that. If the core of the idea resonates with real users, then we iteratively layer on enhancements to the core, shipping to users at each stage.

In order to work this way, you have to accept that quality and feature completeness are not the same thing. At each layer, we're delivering a scope with high quality always, but with no ambition of completeness. We're delivering shippable increments of software as layers over a core idea of value that we've already shipped. We build out our product backlog as all the nice things we could layer over the core, as we think of them, but they are not essential. One of the amazing things about working this way is that users are delighted by it. The features just keep getting nicer before their eyes.

Legibility

2025-11-18T23:01:00.000-06:00

I recently came across a blog post called Seeing Like a Software Company by Sean Goedecke on Hacker News. The post hooked me right away by introducing to my vocabulary the term "legibility":

By “legible”, I mean work that is predictable, well-estimated, has a paper trail, and doesn’t depend on any contingent factors (like the availability of specific people). Quarterly planning, OKRs, and Jira all exist to make work legible. Illegible work is everything else: asking for and giving favors, using tacit knowledge that isn’t or can’t be written down, fitting in unscheduled changes, and drawing on interpersonal relationships.

One of the advantages that small companies have is that they can achieve greater speed compared to a large company by eschewing legibility. When you know everyone else in the company by name, or maybe you all work in the same room together, you don't need standardized processes to go about your work, in fact, they just slow you down. Work happens through what the author calls illegible backchannels:

An engineer on team A reaches out to an engineer on team B asking “hey, can you make this one-line change for me”. That engineer on team B then does it immediately, maybe creating a ticket, maybe not. Then it’s done! This works great, but it’s illegible because the company can’t expect it or plan for it - it relies on the interpersonal relationships between engineers on different teams, which are very difficult to quantify.

One of the struggles that small companies face as they grow into larger companies is that they can't get by anymore without legibility. Maybe they've grown to hundreds of employees or they're working with people distributed across multiple timezones.

The fact of growing up as a company is that the loss of speed in individual tasks is outweighed by the predictability of process. As the author writes:

The processes that slow engineers down are the same processes that make their work legible to the rest of the company. And that legibility (in dollar terms) is more valuable than being able to produce software more efficiently.

I feel like this is a hard concept to sell people on who are used to working in small companies. In the long run, going slower is actually better for the company. Writing things down and organizing the work in a more structured way reduces the chaos left in the wake of localized, marginal speed-ups.

The author concedes that illegible work is necessary in small doses even in large companies. In cases of show-stopping production bugs, for example, large companies create temporary sanctioned zones of illegibility in which they gather together a strike team of experienced people to swarm on a critical issue until it's resolved. They then return to legibility.

Legibility is a massively useful concept that I will carry with me. It explains so much about how companies function internally, in often counterintuitive ways.

The Financial Architecture of Software

2025-10-24T00:58:00.000-05:00

Conway's Law is foundational to software engineering. It says:

Organizations which design systems...are constrained to produce designs which are copies of the communication structures of these organizations.

If you've ever had any job in the software industry, you've almost certainly seen this play out in the architecture of your codebase. Group A works on Thing X and Group B works on Thing Y, so Thing X is in one [project, repository, web service] and Thing Y is in another [project, repository, web service]. You can read the organizational structure in the structure of the technology. It helps to make sense of why things are the way they are. The silos of communication are reflected in the solutions.

But I recently came across this interview on InfoQ with Ian Miell, who I've mentioned on this blog before. He has interesting things to say about the sociology of software development, and in this interview he talks about a book he's writing about the financial architecture of software:

The material aspects of the world ultimately determine the software we build and specifically the decisions we make at a grand scale about the software that we build.

I remember being frustrated and confused as a junior engineer, not understanding why certain things are prioritized in a company and others aren't. Obvious best practices from the industry at large don't get traction in certain places, what seem like universal good things like "quality" don't seem to matter, or why values spoken about at the company level seem unevenly distributed and applied.

Ian explains...

When I have a code base, I want to make sure all the tabs are correctly aligned before I do anything else that might take time, or I wanted to make sure the naming paths are consistent. But you get larger scale problems where engineers say, "No, we have to fix this now because it's a huge thing". And you say, "Well, I get it, but that's going to take us a month and in that month people are going to be looking at me as the leader of this team and saying, 'What have you delivered?' And I can try and explain to them what I've done, but it's not going to apply".

Hard work is not always rewarded. It depends on who sees the work, what their individual priorities are, and how much money they control. My advice to my younger self would be to pay close attention to what your specific boss thinks is important, and show yourself working on those things. The next level is understanding what your boss's boss thinks is important and by what criteria they're judging your boss; then you'll really understand what kind of work is rewarded.

As Ian describes...

Who owns the budget is a fascinating question. ...I start the book with a story about a very successful engineer, I call her Jan. She becomes a leader of a small group of engineers and she builds a platform in her spare time with those engineers, like a Kubernetes platform. And she does it within a central IT function and she builds it and she takes it, she shows it to her manager and the manager's like, "Well, this doesn't help me hit my goals for the year. This doesn't help me get a promotion, this doesn't help me get a pay rise". And she's nonplussed like. "I'm trying to deliver faster, cheaper, better. That's what the company is, we're supposed to be agile. I'm trying to help the whole business".
And it might sound naive, but a lot of people think this way. I certainly did. Where you're thinking, well, I'm doing what's good for the company, but the system is not structured in that way. It's not designed to be run that way because people are complicated and their interactions are complicated, so you silo things and you have different budgets and so on, and the end result is that you are slaving away at the central IT function, trying to make the business better as a whole, but it's not accounted for.

Understanding where the money comes from sheds light on the technology decisions, where quality matters a lot and where it's almost an afterthought, and why different teams are more scrutinized than others. In the end, follow the money.

Slow Is Smooth, Smooth Is Fast

2025-09-23T23:54:00.000-05:00

If you want to move fast, don’t make speed your goal. Make smoothness your goal. Fast naturally follows smooth.

Every software organization wants to be fast. And fast has a certain look to it. Buzzing Slack channels, video calls, people huddled around a screen or whiteboard. It's very easy to conflate "high touch" work practices with speed.

As I mentioned in my previous post, one of the huge benefits of sprint-based development is that sprints act as a delivery protector. A bulwark against constant churn.

The principles of the Agile Manifesto talk repeatedly about change, continuous X, face-to-face conversation. Some teams and leaders really latch onto these concepts. Let's change the requirements every day; let's discuss every decision face-to-face.

One of my favorite principles is:

Working software is the primary measure of progress.

All the churn that looks like "Agility" is canceled out if you don't deliver on a regular enough cadence that your customers notice. They don't see anything else, they don't measure you by anything else.

The passé methodology of Waterfall became passé because it didn't respond to change well enough. People were wed to process instead of reality. Why spend a year building something that no one actually wants, right?

But I firmly believe there is such a thing as being too responsive to change. There is such a thing as too much communication. Remember, your customers only see delivery. They only see what pops out the other end.

There is a benefit to following a well-defined process from idea to delivery. Process is perhaps a dirty word in Agile circles, but I think in some ways being anti-process is being anti-delivery. Every time we circumvent a process, we create communication overhead. Each team member impacted must be notified and followed up with to make sure they know about the circumvention.

Any change is only good if the ripple effects created by the change are worth suffering. I reject the interpretation that Agile implies that more change is better. We have to always balance change with delivery. In fact, we have to balance anything that takes time and effort for a software team with delivery.

Sprints Balance Consistent Delivery With Optimal Value

2025-08-26T23:26:00.000-05:00

Value, value, getting value to users. Work on the most valuable thing, always. You could probably summarize the Agile philosophy as "deliver value to end users faster." What's the next most valuable thing we could work on, get it done, and get it out to production.

With a sprint cadence, we commit to a strategy for two weeks, we work as a team through the items in our sprint backlog, and we retrospect at the end to talk about how we did. There's always the chance that some circumstance around the business could change mid-sprint that may change the team's idea about what is most valuable to work on in the moment. Since sprints balance consistent delivery with optimal value, we keep our heads down and finish the chunk of work we agreed to deliver at the start of the sprint.

This requires leaders and engineers to get used to telling people outside the team, "That's a great idea! We'll put it in the product backlog. We can start working on implementing this idea in less than 2 weeks from today!"

As arbitrary as the sprint cadence can seem, I see the sprint as a delivery protector--a built-in reason to say not right now. The items we're working on today may only be 80% of optimal value, but we're going to finish them dammit. In no more than two weeks we can shift our strategy 180 degrees if necessary, but not today.

Sprints Need a Cooldown

2025-07-28T23:02:00.000-05:00

Teams vary in the amount of handwringing their leaders do about sprints that aren't perfectly rightsized. What if we bit off too much work at the start of the sprint and can't finish it all by the end? What if we overestimated the work and didn't fit in as many tickets as we could have?

I've worked with teams where having work incomplete on the last day of the sprint was no big deal, it happened regularly, and no one was too concerned about it. And I've worked with teams where the team committing to a certain scope of work was sacred, and tickets carrying over was a grave error that we fretted over and vowed not to repeat.

The teams that prefer the overstuffed sprint are trying to maximize velocity and reduce downtime of the team members. Teams that treat the sprint timebox as sacred are trying to maximize predictability and often striving for a kind of "standardization" across teams. Some teams will even wonder why they're bothering with these stupid sprint things and opt out for a methodology like Kanban where people just grab work items when they finish the previous one, and toss out the timebox.

I have always believed strongly that whether you're using Scrum or Kanban, and working in discrete timeboxes or not, that team members need a regular cooldown. Just as an athlete cannot sprint indefinitely and must rest, I believe the same is true of software engineering teams.

As you might imagine, this puts me in the camp of "fill the sprint conservatively so that we don't go over." This, of course, means that many sprints will end without the theoretical maximum amount of trackable work items packed into them. And to that I say: Good. That's the point.

I believe that implicit in the social contract of teams is this: If you want people to sprint, then they must be allowed to cool down. That's the deal. Continuous sprinting means burnout.

Quoting a previous post of mine, here are a few things engineers need to do that are not represented by tickets in a sprint backlog, and are perfect for filling the cooldown period at the end of a sprint, at their discretion:

Training
Preparing for a presentation
Proof of concept / demo of an intriguing tool, framework, technology
Updating documentation that's been bugging you
Spikes into performance improvements
Cleaning out your inbox
For companies that do some sort of periodic "goal setting" for each employee, let people work on goals from their list

If we set a strong sprint goal, bite off a chunk of work that the team is confident of completing, and we get it all done without rushing a day or two before the end of our sprint, that's a good thing. The team feels a sense of accomplishment, the business gets a valuable increment of working software that fulfills a real need, and the individuals get some time to cool down and shift their attention to some low-intensity stuff that nevertheless needs doing before the next sprint starts again.

For me, the cooldown at the end of the sprint is what sustainable pace is all about. We go hard toward an important goal, and if our planning and teamwork are on point, then we know we'll have that down-shift at the end.

There's a Fine Line Between Increasing Productivity and De-skilling

2025-06-23T23:15:00.000-05:00

In the software engineering profession, there's a concept called "resumeware". It's software that was made with the engineers involved in the design specifically making choices about technology stacks in order to have marketable experience to put on their resume in order to improve their future job prospects, regardless of whether they're a reasonable choice for the system at hand. You can kind of tell when you're looking at resumeware because it looks like the architects threw in every fashionable programming language, library, framework, and architectural pattern into an incoherent hodgepodge.

You could say this is a cynical take on the employer-employee relationship, and perhaps, taking advantage of one's aura of expertise. And that could be true. There's a balance to be had in any job between giving your employer a good return on your salary and the desire of an at-will employee to increase their transferable skills. I think every employer knows this--in fact the good ones use this as a recruitment advantage.

The flip side of this coin is de-skilling, the centuries old drive by employers to reduce the skill needed to produce a product, as part of the endless push to reduce labor costs. Today generative AI is hailed as the ultimate de-skilling force. Employees across all kinds of industries (including software engineering) are feeling the heat from their bosses to increase their use of GenAI in their everyday work. I think that history shows plainly that the march of capital toward greater efficiency is unstoppable. Smashing looms is not a sustainable defense.

When I look into my crystal ball, I see a "Great Sort" coming to the software industry. Just as the middle class of well...life...continues to shrink, I can see the same thing coming to the skilled profession of the software engineer. The question is: Do you want to be sorted to the bottom or the top? Do you want your work to become more skilled or less skilled?

I really do feel for junior software engineers coming out of college in 2025, the time of your career when you're the least skilled, and trying to find a great job while you're trying to get your foot in the door in the age of GenAI. The least skilled are the most vulnerable.

For those among us fortunate enough to be past our junior years to the "experienced" stage of our careers already, what does the future hold? My personal perspective is that I, as I always have, fully embrace the idea of reducing the tedious aspects of my daily work. Programming can be bloody annoying a lot of the time. Assembly language is not a skill I want to have because C# exists, and it's way less annoying to use. There is no way in hell that I'm going to write code in Notepad with a command-line compiler (like I did in college) when IntelliSense and Visual Studio exist. Remember trying to track down the cause of weird error messages in your program before Stack Overflow existed? I do! And it fucking sucked.

I will never take the stance that I want to stay inefficient. I do not want to do my work in a more tedious, annoying way so that I can bill more hours to do the work. But at the same time, I will not de-skill myself for anyone. My general feeling about the big, scary "AI" future of software engineering is that I'm happy to use any tool that makes my job less annoying. My real skill is in producing great software. I am not attached to the tools that I use to do it today or tomorrow. If a statistical word predictor trained on the entirety of Stack Overflow, and fully integrated into my code editor, saves me having to read endless Stack Overflow posts in dozens of tabs in my web browser to produce the same end result, then bring it on.

My personal rubric for productivity-enhancing tools is this: Is my work getting less annoying or am I getting dumber? I refuse to get dumber. Getting dumber is the feeling of sorting to the bottom. If a statistical word predictor can do my job better than me, then I need to learn to do something harder. I will not try to compete at the level of the tool. I will be sorted up, not down.

Just as any employer has the natural right to decrease their cost to make a product, I retain the right to increase my transferable skills. There's a thin line between increasing productivity and de-skilling. I'm all for increasing productivity, but I will not de-skill myself.

Calibrating Your Care

2025-05-27T23:29:00.000-05:00

I've been a subscriber to the r/ExperiencedDevs subreddit for a few months, and I've noticed a common pattern.

Someone will post a question to the group desperately seeking advice about some stressful situation that they are clearly very worked up about at their job. The author's specific concern or complaint could be just about anything, but the general feeling is that from their point of view everything seems to be on fire around them and none of their peers or managers are doing anything about it.

Being around social media for many years trains one to expect posts like these to be opportunities for commenters to rage about the injustice on display and to pile on with their own indignation about incompetent managers and clueless coworkers.

Instead, many of the highest voted comments to the post will be something to the effect of:

With all due respect, I think you care too much about this.

Or...

You seem to be taking this personally, and there's no need to do that.

I was definitely surprised at first by responses like this being highly upvoted, but I just kept seeing them from different people on different posts, with very little pushback from other commenters.

Early in my career, I took things at work very personally. I saw it as a badge of honor to care more than the people around you. I should be mad if things are not being done the right way.

The unpalatable truth, in many cases, is that this thing that seems of grave importance to you...actually doesn't matter as much as you think it does. And the clueless coworkers and managers around you actually understand something important that you don't. In fact, you might be the clueless one.

It's easy to get caught in the spotlight effect, where you assume your thoughts and feelings about a situation are the right ones. And it's easy to believe that your view of what's important is shared by everyone.

If your ground level belief is that everything is on fire and no one cares, you’re probably right. But you have to keep in mind that no one else sees the world through your eyes. You may be reading the room inaccurately. Your boss may have more context around the project than your daily experience.

One could take the advice that "You care too much" as being told "You should be dead inside." I would say, just be prepared to lower the stakes in your mind. Be humble about your own role in the world of people around you. Get some mental distance from the situation and try to read the room.

Everybody gets to decide what they care about. Calibrate your care.

Don't Let an LLM Write Your Unit Tests

2025-04-22T21:59:00.000-05:00

Writing unit tests is one of those tasks that software engineers often see as tedious. In fact, I've heard from fellow engineers that unit test generation is one of their favorite uses of LLMs. Let the AI write them! I've always been deeply uncomfortable with this use of LLMs, even though I'm all for reducing the tedious aspects of work in general.

Unit tests are not boilerplate code. If you've read a lot of bad unit tests, then you may get that impression, though. If your only goal in writing unit tests is “Coverage number goes up!” then by all means let an LLM write your unit tests.

But I would argue that writing great unit tests means thinking carefully about what is valuable about the system under test. An LLM is great at generating reams of code in an instant, but it will never know anything about the value of the product your codebase represents.

If our goal instead is “Let’s ensure that our product is retaining its value as we add more code to our codebase.” then a human who understands the value of the product should write the tests.

Unit tests have yet another critical function, which is to provide runnable documentation about the system for other engineers to read. Want to know how some part of the codebase is supposed to work? Go read the unit tests; they represent the valuable features of the software that an engineer wanted to point out.

Code is written once, but read many times. And test coverage is just a number. If you want tests that matter, write them yourself.

Copilot Is Outsourcing 2.0

2025-03-26T01:06:00.000-05:00

Decision-makers can remain irrational longer than you can remain solvent. (Or, in this context, remain employed.)
- Will Larson, "Career Advice in 2025"

"AI" coding tools like Copilot went from a mere curiosity to full-blown silver bullet territory in a few short years. I'm all for reducing the accidental complexity Fred Brooks wrote about in his classic essay on software engineering, but what’s driving me a little bit crazy lately is talk amongst people (mostly non-engineers) around the industry that Copilot and the like are magical productivity machines.

I'm not the only seasoned engineer to notice that this present moment feels eerily similar to the outsourcing craze of the 2000s. The dream of every non-technical executive who had a bunch of software engineers on their payroll was to lay them off and replace them with people in faraway countries with a lower standard of living who they could pay less to do the same job. Well...it didn't exactly work out that way. It turns out that software engineers are not, in fact, fungible code-typing machines. Outsourcing jobs to cheaper parts of the world is very much still a thing, but it took several years of reality colliding with the dream to bring expectations down to a reasonable level.

In 2025, the executive's dream is back, and bigger than ever. What's better than paying humans in other parts of the planet less to do the same job? We can get AI™ to do the job for even less!

The outsized hype around AI coding tools is premised on the idea that software engineers are primarily code-writers. If we could get machines to type the code, then we can save a ton of money, right? The reality in my experience is that typing code faster is not a bottleneck for any software engineer that you would want working on your product. Writing code is a small percentage of what a software engineer actually does.

I remember a friend of mine from college asking me many years ago, when learning that I was studying Computer Science, if I was worried about destroying my hands from typing so much. He apparently imagined that the career I was training for was similar to a court stenographer. In the software industry, we used to refer to engineers who produced huge volumes of repetitive code quickly as "code monkeys". This matches the outside perception of someone furiously hacking away at a keyboard, with speed of keystrokes making the overall impression.

The thing is, a “code monkey” is replaceable by AI. If there's anything we've learned over the last 200 years or so, it's that machines can do repetitive, mindless tasks faster and more cheaply than humans. If the work of a software engineer really looked like: 1. Do a Google search. 2. Copy and paste code from your web browser into your code editor. 3. Hit a button to commit to your team's code repository; then heck yeah, a machine can do those things way faster than a human.

I don't fear being replaced by an AI that can copy and paste code from the internet faster than I can. I do worry about non-technical executives either firing me or not hiring me based on marketing, or a belief that this is what software engineers primarily do to create software that is valuable to an actual business.

Just like the executive's dream of outsourcing faded to a modest incremental reduction in the cost of developing software, I believe that so to will the dream of the AI engineer. I don't need to ask Fred Brooks what he would have thought (RIP).

Engineering-led Research Tickets

2025-02-25T23:28:00.000-06:00

One of my optimistic beliefs about software engineers is that we're full of good ideas about how things around us could be improved. We're smart people, problem solvers. We read tech news, we're friends with engineers at other companies, we bring diverse career experiences to each job--in other words, we're at least vaguely aware of better ways of doing things that are happening in other places.

If we want to go beyond just talking about better ways of doing things, we need to empower engineers to capture their ideas for improvement in the backlog right next to ideas from the product management side. It's critical, of course, that items in the backlog generated by engineers are refined, estimated, and actually included in sprints on a regular basis.

And we want to go beyond the all-too-easy step of adding a one-line ticket at the end of our backlog that says “do cool thing X”. Since no one has the foggiest idea how we’ll actually do that cool thing, the backlog item just sits there and no one ever puts it into a sprint because it’s too nebulous and scary and we don’t know how long it will take or if it’s even possible in a sprint-worth’s amount of time.

Make that “do cool thing X” a research ticket. Time-box the effort. We can put this into a future sprint because at the end of the time box (fitting within one sprint) we will have a documented outcome that gets us closer to the goal than we were before, and we’ll have a much better idea of the actual steps to get there with "real" tickets representing the work.

The output of these research tickets can be a comment saying we shouldn’t do this and here’s why. Or the output could be a series of other tickets the engineer creates with a ticket for each step of the process toward accomplishing the ultimate goal...or a nice intermediate goal.

Even if we end up looking at the list of steps for follow-on work and think, “Geez! That’s a lot of damn work to get the outcome we want!” and we decide it’s not worth the effort, that's still a great outcome.

Software Engineer Career Levels at Companies You've Heard Of

2025-01-28T23:41:00.000-06:00

I found a site called Progression.fyi that curates a list of "career frameworks," or in other words, the level-progression system of titles that a bunch of tech companies use for their software engineers. It's pretty interesting to peruse.

Several of these companies define what the often nebulous title of Staff Engineer means to them, which I find particularly compelling.

Here are a few companies you've probably heard of, with links to their pages where they define their career progressions for software engineers:

Heisenberg Effect in Software

2024-12-17T22:24:00.000-06:00

One of the greatest services that software engineers provide is that we help people understand their own thought processes. We're transcribers of human thought, in a way. We have to understand a human process so intricately that we can write down instructions in another language that the dumbest, most literal being on earth (a computer) can follow.

We hold a mirror up to the user: We built this system in your image--does this look like you? Is the image distorted? If so, how? In the Agile philosophy, we hold this mirror up as early and often as we can.

Another metaphor I like comes from an old interview with Andy Hunt and Dave Thomas (of "Pragmatic Programmer" fame), where they described a Heisenberg effect in the process of building software for people:

Software [has] a Heisenberg effect, where delivering the software changes the user's perception of the requirements.

Users need to see our interpretation of what they told us. That helps them understand themselves better, and then, we can understand them better.

Are You a Stable or Volatile?

2024-11-19T23:53:00.000-06:00

I've written before on this blog about the different personality traits of software engineers, and how different traits balance each other on a team.

Some of the spectra I came up with were:

Dreamers ↔ Pragmatists
Big Picture ↔ Detail-Oriented
Move Fast & Break Things ↔ Slow & Methodical
Optimists ↔ Pessimists
Answerers ↔ Questioners

Recently I read a comment on Hacker News (which I can't find now) that mentioned a blog post from 2012 on Rands In Repose, a blog I've been familiar with for years, but somehow this post had escaped me.

The author, Michael Lopp, introduces the Stables and Volatiles divide, which feels to me like a meta-category that encompasses the ones I listed above.

You can probably guess just from the names, but Stables tend to like having a well-defined plan, are more risk-sensitive, predictable, and reliable. Volatiles are disruptive, like to blaze their own trail, dislike process, and have a strong bias for shipping something (even if it's ugly and unstable).

Lopp says:

Stables will feel like they’re endlessly babysitting and cleaning up Volatiles’ messes, while Volatiles will feel like the Stables’ lack of creativity and risk acceptance is holding back the company and innovation as a whole. Their perspectives, while divergent, are essential to a healthy business.

I believe a healthy company that wants to continue to grow and invent needs to equally invest in both their Stables and their Volatiles.

I think the Stable / Volatile concept feels very true to my experience, and it's one I'll keep with me. Assuming that everyone on a team is equally intelligent, competent, and well-intentioned, you need people of both the Stable and Volatile persuasions. Even though they annoy each other, a project staffed by only one or the other type is doomed.

Who Writes the Backlog?

2024-10-22T22:35:00.000-05:00

The Product Owner has the final say in backlog prioritization. But who writes the backlog? Is it always the product owner?

My experience with Agile has been that, in the real world, the backlog contains two types of items:

The "user stories" that discuss functionality from the perspective of an end user
Technical stories--including things like technical debt, maintenance, package upgrades, etc. Things that don't represent user goals that the product owner would be focusing on.

Both types of items are important and need to be worked on.

You can get into trouble in a couple ways:

Every item in the backlog is forced into a "user story" template ("As a...", "I want to...", "So that...").
The product owner is writing backlog items that are of the "technical story" variety since they think they need to write every story in the backlog.

If you have a product owner who has consistent time to write Jira tickets (or whatever issue tracker you have), you're pretty lucky. My experience with actually-existing-Agile is that the Product Owner is typically someone in middle management who is juggling multiple jobs, "product owner" of this product being one more job, and certainly does not have the time to dedicate to backlog management.

Occasionally you get an overactive product owner who tries to capture every task the development team is working on as backlog items, including purely technical tasks that are better written up by engineers or engineering managers/leads.

My opinion is that anyone on the team (engineers, QA, managers, product owner) should be empowered to write backlog items. When the nature of the backlog item is technical, let the technical people write it. When the backlog item is a "user story", the product owner should write the item, or at least delegate the requirements to whoever writes it (probably not an engineer).

Backlogs are just another tool in the tool chest of software engineering. As long as the people doing the work are clear on what the goal of the item is, and the acceptance criteria for its completion, then anyone can write them in whatever format they want.

Legacy Systems Age in Reverse

2024-09-24T23:22:00.001-05:00

I feel like every job I've ever had in software there has been some old legacy system hanging around that everyone denigrated and perennially spoke about replacing with a newer, shinier system. That replacement was always coming any day now. By the time I'd moved on to a new job, that legacy system was still running, quietly doing some important function for the business, having somehow survived the demise everyone predicted.

The first release of jQuery was in 2006. Within a few years--and every year after--web developers would talk continuously about how outmoded jQuery was, and that no self-respecting developer would still be using it when so many newer, better JavaScript libraries had come along to replace it. Well, guess what. In 2024 (18 years later), jQuery is still in active development, and according to the official jQuery blog, 90% of all websites use jQuery currently. As someone who's been working in the web development space since roughly the dawn of jQuery, I can say that it is still extremely rare that I see a codebase that doesn't use jQuery somewhere, whether as a direct dependency, a transitive dependency, or as part of some 3rd-party tool integrated into the product. jQuery will outlast the human species.

Software systems exhibit the Lindy effect:

The Lindy effect (also known as Lindy's Law) is a theorized phenomenon by which the future life expectancy of some non-perishable things, like a technology or an idea, is proportional to their current age. Thus, the Lindy effect proposes the longer a period something has survived to exist or be used in the present, the longer its remaining life expectancy. Longevity implies a resistance to change, obsolescence, or competition, and greater odds of continued existence into the future. Where the Lindy effect applies, mortality rate decreases with time.

They actually age in reverse. Every year they exist doubles their additional life expectancy. That old system that everyone thinks will be replaced any day now is gaining strength before your eyes, becoming every day less likely to be replaced.

New technologies are the least likely to survive another day, another year. Just as a business that's existed for 100 years is more likely to survive to its 101st year than a 2-year-old business is likely to see its 3rd year.

The implications of the Lindy effect to any of us working in "technology" are so widespread that it's hard to overstate. We're in an industry obsessed with the new, but the new things are constantly passing away as the old geezers are running laps around them. Any new programming language you're excited about right now will be dead long before C is. VS Code will kick the bucket before Vim. Be kind to those elderly technologies around you, because they'll be here long after you're gone.

Scrum Tensions: Code Review

2024-08-27T23:08:00.000-05:00

There are tensions in Scrum anywhere you have intra-sprint cycles that must resolve by the end of the sprint. In my last post I wrote about manual quality assurance and the tension that causes in a Scrum setting. Another one of these intra-sprint cycles that most Scrums teams include in their process is code review.

You could, in a way, consider code review to be another form of "QA". In both cases we're initiating a cycle-within-a-sprint with a loop of approval, rejection, re-work, and acceptance.

In the sense that Scrum teams strive to get every sprint backlog item to "done" by the end of the sprint, back-and-forth cycles kill sprints. But they're also essential. Therein lies the tension.

Most Scrum teams conduct estimation sessions prior to starting a sprint. The estimates that a team assigns to product backlog items are typically based on a gut feel of how long it will take or how complicated it will be to "complete" a backlog item. In most teams I've worked with, that estimate is implied to mean how long it will take a software engineer on the team to submit a code review (via a pull request or something similar), i.e., when the engineer thinks their part is "done". Some teams also build into the estimate a duration or complexity estimate based on the work necessary by the QA people.

What estimates don't ever capture, in my experience, is how much effort/time/complexity is required to complete the code review or QA cycles on a backlog item. How can we know ahead of time how many pieces of feedback are going to be left on a pull request? How many of those will necessitate re-work on the item before the end of the sprint? How long will the re-work take? What if the re-work is still not up to snuff? Etc., etc., etc. Each code review starts a cycle of unknown duration.

And all the while everyone who cares about the completion of the sprint is tap-tap-tapping their fingers, nervously wondering if these intra-sprint cycles are going to complete on time.

Even though people mean well, what happens in practice is that thorough code review is tacitly disincentivized. Most product owners are not engineers, and although they understand intuitively that code review, like any form of quality assurance, is necessary, it also adds delay to the immediate "done-ness" of work.

Engineers also know that code reviews are important, but there is always pressure to get to them faster. When bugs crop up later, or the codebase slowly accumulates technical debt, no one in management is going to track down the specific pull request that introduced the problem, read the list of names in the approvers list and assign blame to those individuals. A swift click on the "Approve" button satisfies everyone in the moment.

What do we do about this tension? We can't have code review without introducing an unpredictable amount of delay to each sprint backlog item. You can never fully resolve this tension, in my opinion. But there are some ways to ease the tension.

Be Your Own Reviewer

I'm going to put this one on the engineers. And, no, I'm not suggesting that an engineer who submits a pull request should be clicking "Approve" on their own pull request. What I am suggesting is that before an engineer submits a pull request, it should only be after they have examined their own code to the degree that a reviewer would.

I can't tell you how many times over my career that I've reviewed a pull request where it was clear that the submitter had not even looked at what they were submitting for review. You're not ready to submit a pull request until you've looked line-by-line at the diff you're about to submit and pre-corrected every issue you can find. We're not talking about architectural judgment calls here, we're talking about misspelling method names, pushing blank files, including huge sections of commented-out code from various abandoned local experiments.

It should be rare that a reviewer needs to point out an obvious mistake in a review. I personally feel embarrassed when a reviewer points out something that I know I could have found myself. And I feel doubly embarrassed if it's a mistake they've pointed out multiple times.

Automate, Automate, Automate

Take advantage of every opportunity to automate the tasks associated with code review. We have linters, pre-commit hooks in version control systems, IDE integrations. There are ways to automate basically any tedious, repetitive aspects of code review. If reviewers are repeatedly finding the same kinds of mistakes, it's time to automate the checks for those mistakes, so submitters can't submit them in the first place.

Back-and-forth intra-sprint cycles are what kill sprints. Code review is a cycle, but what can we do to get down to one iteration per cycle as often as possible?

The tension between any kind of quality assurance (including code review) and sprint-based methodologies is impossible to erase, but we have techniques to make it less tense.

Scrum Tensions: Manual QA

2024-07-29T21:24:00.001-05:00

After working for many years in the software industry, almost always using a methodology resembling Scrum, there are certain tensions that I've come to believe have no good solution--at least--I've never seen them solved elegantly.

One of those tensions comes from manual quality assurance. When sprint backlog items need to be manually tested by dedicated QA people, there is simply no clean way to do it in my experience.

Here are some of the issues that I've seen repeatedly:

QA does not have time to test items where the development work is completed late in the sprint
QA people are sitting idle for the first few days of the sprint with no completed development work to test yet
If a bug is found, there is no time in the current sprint to fix it

Here are some attempted remedies that I've seen:

Let's build some slack into the sprint for the developers so they can complete their development work with X days to spare at the end to allow time for QA
Let's have the developers "work ahead" of the QA people, as in, the development work that we say is "done" within one sprint is actually tested in the next sprint

And here's where those remedies fall down:

Inevitably, there is pressure to work more items into successive sprints, and development work starts creeping over the false "deadline"
We end each sprint not knowing what is actually "done", and any bugs that are found have probably already been merged into whatever common branch the developers work from

QA people are in a really tough position as they are always sort of passively pressured to not "hold up" the sprint. They know that everyone wants to see a clean sprint where every item is signed off on by QA by the last day. And the double whammy is that they get heat when bugs are found in production.

This is one of those essential tensions in Scrum-based development that I've come to accept is not resolvable. I have never seen manual QA fit neatly into Scrum.

As I see it, there are two options to break the tension:

Do manual QA, and abandon sprint-based methodologies (try Kanban, for example)
Use a sprint-based methodology, and avoid manual QA (some teams fully automate testing)

In my experience, organizations do neither of the above, and the tension continues.

Continuity of Leadership

2024-06-27T23:05:00.000-05:00

There's a Kafkaesque situation that develops in companies that cannot retain senior employees. A team feels a sense of urgency about shipping software, but no one can tell them what to build.

Engineers are responsible for making the products that sustain the company, but there's no one around who can say what the products should do exactly.

It's like a movie where the protagonist wakes up every morning with no knowledge of what happened the day before, and tries to piece together his identity from artifacts scattered about.

Big organizational initiatives fail repeatedly because there's no one around from the beginning to the end of the initiative. People working on the initiative find that they can't even remember why the initiative was necessary. The leader who championed the initiative isn't around to take credit for its completion, so there's no one working on the initiative at any given time who cares about its completion.

Cross-team relationships barely exist because whoever is leading one team at the moment doesn't know the leaders of the other teams and has no history with them.

Leaders are continually "backfilling" for other leaders that left, being asked questions that they can't answer.

Morale stays at a low simmer. People show up to work each Monday morning knowing they won’t be productive. Another week where they won’t know if they did good work or not, because no one can define the goal.

An organization without continuity of leadership is an organization with amnesia.

Daring to Care

2024-05-30T19:30:00.000-05:00

The most effective technical leader I ever worked with had a track record for coming onto a project and whipping it into shape. His ideas were not groundbreaking. He was not a genius engineer. He was a smart guy, but not necessarily the smartest guy in the room. He wasn't an expert politician, or a charmer. His superpower was that he simply cared more than anyone else around him about the project's success, and he would not back down when implementing improvements. When he noticed an area for improvement, he just did whatever it took to fix it. He would calmly but with absolute persistence run through his argument with whoever he needed to convince that it was the right thing to do. It didn't matter how many people he had to convince, how high up they were, how difficult it was to convince them. He just wouldn't stop if he knew he was right. No improvement was too small or too big. I remember him saying to me once that he didn't understand why people around him would acknowledge obvious problems, but not fix them themselves. What I didn't say, but was thinking silently, was "because no one else here cares as much as you do."

There's risk that comes with caring about something. If I take the initiative to fix the slow build process, then I'm now responsible for it. If I break something, everyone's going to look at me. I might have to talk to the Infrastructure team. Jeez, those guys take so long to get back to you. I'll have to open tickets. I might have to bother people I don't know well to make my task their priority. The build process works now, right? Yeah, it takes longer than it probably needs to, but it works. Do I really care enough to take all this on?

The individuals who rise to the top aren't always the smartest, the most creative, the most charming. They just give a damn. They care more than the people around them. When someone truly cares, they will find no shortage of problems to solve around them. They will find solutions. They will push through objections. They will argue with people. They will take on responsibility for things they don't strictly need to, things no one asked them to take responsibility for.

The open question is: How do you get someone to care? Why do some people care so much more about a project than those around them? Those people are worth their weight in gold.

Movin' Tickets

2024-04-26T12:05:00.000-05:00

Recently I was re-reading Joel Spolsky's classic blog post The Joel Test: 12 Steps to Better Code. I hadn't read that post in many years. Although a lot of the advice in that post seems almost quaint now, as many of the practices it encourages are ubiquitous and taken for granted in 2024 (Joel wrote that post in 2000), there is one passage that seems just as fresh as ever to me...

...project managers had been so insistent on keeping to the “schedule” that programmers simply rushed through the coding process, writing extremely bad code, because the bug fixing phase was not a part of the formal schedule. There was no attempt to keep the bug-count down. Quite the opposite. The story goes that one programmer, who had to write the code to calculate the height of a line of text, simply wrote “return 12;” and waited for the bug report to come in about how his function is not always correct. The schedule was merely a checklist of features waiting to be turned into bugs.

One of my frustrations with Scrum, or with many teams who say they are "doing Scrum" is the obsession with the Sprint. Teams develop a short-term mindset, where all things begin and end within a two-week period.

We have some sort of "Board" where all the tickets (or PBIs, or cards, or whatever you call them) are shown in one of several different columns each representing a "status" of the ticket. A ticket starts in the far-left column on the first day of the sprint, and by the last day of the sprint, it must be in the last column. That's how we know we had a "good sprint".

Obviously the tickets on the board are just an abstraction representing work. But it's easy to get obsessed with this abstraction. Instead of making our users' lives easier and adding valuable features to the product they use, we're just moving tickets across a virtual board, sprint after sprint.

I always think it's fascinating to hear the language people use to talk about a team's work. In standup, people will say they plan to "have that ticket moved over" today. The team's manager might talk about how "good the board looks" today. In a retrospective meeting at the end of a sprint, the team might talk positively about how quickly tickets were "moving across the board" during that sprint.

The people on the team actually doing the work know they're doing well when they've moved a ticket from one column to a column to the right of that column. This is what they optimize for: efficient ticket-moving.

The necessary work of software engineering that doesn't have a ticket on the board feels downward pressure. A thorough code review for one ticket takes ticket-moving time away from the reviewer. If there are issues to be corrected, then the ticket being reviewed is stalled in its own rightward journey.

QA people on the team are in a difficult position of doing their quality assurance on tickets that are just to the left of the ticket's final destination--the place we all want it to be.

The whole team is incentivized to make sure all the tickets on the board are in the right-most column on the final day of the sprint. As in Joel's anecdote above, bugs found later merely become new tickets to move from left-to-right in a future sprint. Long-term concerns like sound architecture don't have a place on the board.

When a team judges its effectiveness based on the movement of virtual tickets from one status to another, it can lose sight of the big picture. Who is ultimately benefiting from these ticket movements? Why are we moving them exactly? Where do they come from?

I think it's important that teams talk about their work, at least occasionally, without mentioning tickets. What are we accomplishing at a higher level? What are our users saying about our work? How is the business that pays our salaries benefiting from our work?

Surely we're not just movin' tickets.

Vision

2024-03-26T23:16:00.000-05:00

Vision is so important in software development. Without the engineers understanding the overall vision, they can't resolve ambiguity in their daily work without consulting someone who holds the vision.

If the engineers don't understand why they're doing any of these things, then they can't fill in the gaps logically. They can't suggest improvements, improvise, or have confidence that they're moving the organization closer to the vision. Teammates talk past each other. One person has more of the vision than another, but doesn't know that. Misunderstandings are common. The track being laid from each end doesn't meet up in the middle.

Every little bit of vision transmission compounds in value. The decisions we make today form the foundation for work that comes later. A misunderstanding in vision today requires re-work tomorrow, a week from now, a month from now.

One of the things that can get left behind in the just-in-time fashion of Agile sprints is that the team can get lost in the weeds. We have to remember that we're building toward a significant milestone of some kind for the business, not just a random sequence of tasks.

It's difficult when backlog items are being entered by one person or a small group of people separate from the engineers and QAs who will be actually building and testing the stuff. They get queued up and drip-fed every two weeks to the broader team. But often there's no shared context transmitted to the whole team about what broad goal we're doing all these tasks for.

Sprint goals are tricky to set. But if a team can't ever seem to define a clear sprint goal, that's almost certainly a sign that the team is lacking a vision.

People like to make fun of heavyweight methodologies like SAFe, but doing a Program Increment Planning event--although tedious--sure does get everyone on the same page with a shared vision for the next few months of work.

Most of us are not working on the Manhattan Project--the end goal should not be obscured from the individuals making the parts. In fact, the end goal should be so clear to everyone that any person on the team could describe it clearly in their own words.