About STP / 877.257.9531
Log In Join Now
STP Community Blog





Tuesday July 6th 7am

Salami Slicing - Re:Visited

Testing Software Test and QA Management Quality Assurance Performance Process

Last week I wrote about Salami Slicing -- the viscous circle task of "making money" by offering less product for more money while squeezing your employees and supply chain for every possible penny.

To some extent, that's good business.  After all, part of Adam Smith's argument for capitalism was that competing organizations would find cheaper ways of doing business, passing a portion of the savings on to customers.  So by shopping around for the best deal from your suppliers, you are encouraging them to innovate.

If you look hard enough, you can always find a better deal.  The problem comes in when you below rock bottom prices, to prices that just don't make any sense any more.  Or, as John Ruskin is claimed to have said:

It's unwise to pay too much, but it's worse to pay too little. When you pay too much, you lose a little money — that is all.

When you pay too little, you sometimes lose everything, because the thing you bought was incapable of doing the thing it was bought to do. The common law of business balance prohibits paying a little and getting a lot — it can't be done.

If you deal with the lowest bidder, it is well to add something for the risk you run, and if you do that you will have enough to pay for something better.

Another way to state this law is "you get what you pay for", and you can see it at play every day in the real world.  For example, I recent bought a pair of dollar store sunglasses for a family trip to Itasca State Park in Minneosta.

I usually don't wear sunglasses, but it was thirteen hour trip each way, and hey, they were a dollar.  

Yes, one buck.  They didn't look terrible and the road wasn't a beauty contest.

Of course, one of the glasses fell out of the frame after the first half of the trip, lasting a total of fourteen hours.  It wasn't just cheap and short-lived -- no -- it was so cheap and so short-lived that I had to drive back with my eyes looking into the sun all morning.

The problem was that I payed too little.  Even going the next level up, getting something for five or six dollars at a grocery store, likely would have lasted me ten to twenty times as long.  Above that, and, if I took care of it, we could see one hundred times as long for only twenty times the price.

Now let's talk about the organizations I see doing outsourced testing.  

Question 1: of the organizations doing outsourced testing (or, in some cases, even in house) -- how many of them are managing the test organization as it they were me going to the state park?

Answer 1: Far too many

So me ask question 2 -- What is the testing equivalent of broken sunglasses?

My guess is that not only is the software is broken and management doesn't know it and has bad information, but more than likely the software is also late and delayed because the outsourcer had problems setting up and staffing the test lab.

It gets worse.

To explain how it gets worse, let's start a typical salami-slicing scenario. Say, for example, you have a  software project that needs to be tested.  For the sake of dicussion, assuming it were possible, let's say you break the project into one hundred minor "featurelets" that ca be tested in isolation as the sum of the project. (This is just to save estimation -- more about that later.)  We'll call this statement of work the 'demand', and it's not cheap.

Counting salary, benefits and other things on the back of a napkin, your loaded cost for an employee is about $500 per feature, maybe $750 if you hire a short-term contractor from that local 'Agile' shop.  To save money, you outsource the work to a location with cheaper labor, say $250 per feature, for a total of $25,000 in savings off that regular cost, more in comparison to a consulting shop.  

Sounds good so far right?

Well ... let's get back that Ruskin quote.  The projected savings assume that the outsourcer can actually do the work.  What if he can't?  What if he doesn't understand the requirements, or the environment setup, or the intended use?  What if he calls you up and asks for guidance or to place someone on-site?  What if he turns in the work for the first twenty features and you realize it technically fulfills the letter of the contract, but you find the result unacceptable?

This creates a new type of demand, the cost of asking for the same thing a second time, perhaps with more clear language.  This kind of new demand is something that the british psycologist John Seddon calls 'failure demand.'  That is to say, when you are salami slicing, you might decrease the cost of each individual piece of work, but the number of pieces of work consistently increases.  And it increases by a lot.

To put that a different way, we'll use Mr. Seddon's language:

"When you focus on decreasing costs, costs go up."

Back to our test example, that means bugs will slip through to production, you'll have to do a patch release (or two) and need retesting -- plus the time to manage the project will go up.  Outsourcing in this way becomes a sort of false economy.

This is a real situation that occurs with many companies; in 2008 at a major test conference, one presenter claimed that she to be successful with test offshoring they needed to write test cases at a roughly 3rd grade education level.  That's not an intended as an insult, more as a fact of life -- when you try to communicate an idea to someone who is thousands of miles away by paper, information gets losts -- and you have no mechanism to recognize this and adjust.  So to be successful, you have to convey simple ideas in a very step-by-step fashion -- ideas so simply that both you and the other party understand the words without confusion.

Even this, however, has a problem, because you are telling the person what to look for.  Yet at the end of every test case is this hidden check that few people write down: "... and nothing else strange or unexpected happened."

It turns out that describing that check in a simple, step-by-step fashion is surprisingly hard.  In order to execute it well, you have to like, well, you see ... you kinda have to know the application.

Tell us how you really feel, Matt

Let me be clear: I'm not against outsourcing.  My cohort in graduate school wrote our master's capstone on outsourcing, and there are a lot of cases where it makes sense.  Here are a few cases:

1) When you need to use a commodity.  Commodities are products that you can purchase that are essentially identical, where the only thing about them that differs is price. Paper, Gasoline, Oil and Corn are all commodities.  You can tell a commodity because it has a "futures market."  In most cases, they are cheap to buy but expensive to  develop yourself, and multiple companies compete by price to sell to you. In that case, it makes sense to purchase your supplies from someone else.  For example, try to find a local restaurant that makes it's own napkins, silverware or china -- it just doesn't make sense.    More practically, electricity, bandwidth, and, to some extent, databases, compilers, and operating systems lean in this direction.  Outside of Microsoft and Apple, most of us buy these products without a second thought.

2) For expertise you don't care to develop.  In this case you might start a long-term partnership.  The classic example of this is the big company hiring an advertising agency, but it happens in software as well.   I know of several web development and programming shops that partner with companies to build software, but the partnership is focused more around expertise and priorities than it is around hourly rate.  In some cases, these companies have worked together for two, three, five or ten years successfully.  One thing I see in common in all these engagements is that they were not primarily motivated by cost.

3) For temporary staff.  Most companies have up and down periods in a rough cycle; hopefully the trend line is up, but you will always have a time period where the current big project is winding down and the next big project hasn't started.  With straight employees, it's very hard to properly staff.  So you might bring in contractors, consultants, or hire a firm to offload some of the work during the ramp up phase of the big project.  The benefit here is not cost savings per transaction; it's quick ramp up and down.  In fact, you'll generally pay a premium to hire contractors; the hire bill rate is given on purpose in consideration of less job security.

4) When your company lacks expertise in an area and wants to develop it.    In this case, you are hiring an outsourcer yes to do a project, but also to train and equip your team to do the work themselves.  The Agile Coach/Consultant is probably the best software example of this, but I know folks who do this kind of work in software testing.

If you'd like the whole story, you can read the old Capstone paper.  It's not terrible.

In the mean time, let me re-iterate:  Organizations that manage by costs alone worry me.  Because you can always save a dime here or there by taking a short cut.   As Ruskin tells us, when all you care about is cost, you run the risk that what you deliver isn't capable of doing what you ask it to do.

Somewhere in North America, there is a drum, beating for 'higher efficiency', and what it means is cheaper transaction cost.  

I don't care about cheaper transaction cost.  I care about value, and, if any cost, lower total cost.

Now if we wanted to change our organizations to focus on value -- what would that look like?






Friday July 2nd 6am

Sorry, no TWiSt this week

Testing Test and QA Software Interviews
So last week I started the "This Week in Software Testing" podcast.  With a name like that, you'd expect it to be, well ... weekly, right?

Then the screen went out on my macbook.  No, really, it's an apple warranty issue.  They are replacing it free.  No worries.

But until I get it back, I'm stuck on this old dell with no good recording software.  So no TWiST this week.

I can, however, recommend some good alternatives.  My friend, Željko Filipin (A walking internationalization test), is involved in TestingPodCast.com, a site that compiles and lists podcasts about testing.  I think the tag line they have for the site is just great:

Audio podcasts on software testing. All of them.

Of course, because they index all of them, the results are a mixed bag.  At least most of them aren't boring.  So you might listen to a podcast that you disagree with in order to understand the other side's position -- and, maybe, to deconstruct it.

If you listen on your computer, maybe a lunch, a video might be good, in which case I'd recommand Simon Stewart's latest "Test Engineering at Google" talk.

In the talk Simon covers what Google calls SETS - Software Engineers in Test - what I've been calling on this blog SDETs.  

I find Simon's perspective fascinating.  According to Stewart, the SET is actually a programmer who spends most of his day, well ... programming.  Not necessarily writing programs to directly test, though, the SET spends his time on test infrastructure, writing frameworks and tools to help the end developers at Google do their own test automation.  Another big piece of his job is education and training on how to use those tools.

The radio of dev to SET at Google is something like seven to one.  So developers are expected to do more testing and test automation themselves. Instead of testers, this picture is one of SETs as accelerators.

To my knowledge, google also employs a third or possibly fourth role of "Test Anayst" or "QA", most of which are contractors and do what you and I would call "humans in a seat, using the software" testing.  They seem to be reluctant to talk about that.

Finally, my friend Alan Page at Microsoft has a blog post where he claims that while SDETs at Microsoft need to have developer skills, that does not mean that they have a vision where 100% of all tests are run by a computer unattended by a human being.  Alan sees it as a continuum, with 100% on one side and 0% on another, and most good testing is somewhere in between.  So instead of test automation "Yes or no", we need to be asking where and how much.

The truth is that Alan Page and I agree on a whole lot - it's just the things we disagree on are the most fun to debate. 

So, sorry, no TWiST this week.  But between Željko's podcast list, InfoQ's videos, and Alan's writing -- somehow, I don't think you'll starve.

More to come.  Serious this time! :-)






Wednesday June 30th 4pm

Special Offer Extended

Membership STP Community News ST&QA Magazine

ST&QA Print Subscribers Get a Week Extension on Special Offer…

In the printed edition of Software Test & Quality Assurance magazine (ST&QA), page 9 included a special offer to readers. We have received a great response! The offer was scheduled to end July 1st but since many printed editions only landed on desks last week, we decided to extend the offer for an additional 7 days. The offer code will continue to work until July 8th.

So for all of our loyal readers I hope you take advantage of the offer. This offer is NOT included in the digital copy of the magazine.

We will continue to offer special deals in our print magazine. To guarantee a printed copy simply join as a Pro Member…







Wednesday June 30th 10am

Doing More with Less

Software Test and QA Testing Management
Okay folks, it's 2010.  For those of you millennial's who entered the work force after 2001, you've spent your entire career living in something like a recession.

Oh, I know, the definition of a recession is two quarters of negative change in Gross Domestic Product, or GDP.  So it hasn't literally been one long recession -- but it's felt like one.  For those of us in the United States, we've seen slowly rising unemployment, a tech bubble burst, a housing bubble burst, a decline in manufacturing jobs combined with a decline in call center and related "business process" jobs as they are moved to lower cost countries.

Those transfers mean that in the case when GDP has actually increased or flatlined, the profits have not gone to Joe worker, or to hiring more of them. Instead companies are sitting on cash, passing dividends to shareholders, or partners.  The Wall Street Journal recently published a story about the jobless "productivity boom", mostly based on hustle and brains, asking it if is sustainable.

At the same time, we've had a merge and acquisition boom.  One of the benefits of these mergers is the company has the same sales, but now redundant HR, IT, and Accounting departments, and can save money by consolidating those departments.  When the consolidation happens, they don't need twice the staff, so some staff members are redundant.  Hence the phrase "made redundant", which in business is sort of a code word for "layoffs."

So my guess is that most North American, and likely some European, readers have recently heard some lines about "tightening our belts" and "doing more with less."

The first line can be done; you can cut the training budget, not buy new computers and not upgrade to the latest-and-greatest version of your test tool or compiler.  Once you've done that, however, you've got a bit of a problem.  Sure, you can cut out the daily bagels and snacks and save the company hundreds a week, but cutting out the monthly anniversary cake is going to save you more like a hundred a year.  And beyond that?  The next few levels of cuts doesn't save much money and directly hurts productivity.

Note this first strategy does not actually mean doing more with less.  Instead it means doing less with less - but trying to do the less to other people, not to the company itself.  Another term for this approach is "salami slicing."  Salami Slicing is named after the anonymous salami sandwich dealer, likely in New York city, that had a profitability problem.  Instead of looking into a better location, better advertisements, or taste-testing different recipes and attempting to raise prices, he chose the easy way out -- slice the salami thinner, and you can get more slices per stick.

For our sake I'll call it salami slicing anytime the business strategy is to squeeze everybody else. The typical ways to do this are to pay the employees less, cut the health-care benefits, increase the co-pays, increase costs to customers while aggressively negotiating with suppliers.  Montgomery Wards and Circuit City both pursued this strategy immediately before going out of business.

While watching your costs is important, if that's all your doing, it's almost certain your customers and the most talented employees - the ones who can find other jobs - are going to walk as well.  Actually, you might say that aggressive Salami Slicing is an indicator that your companies management has run out of ideas.

Which brings me to that second phrase "Doing More with Less."

I am even more skeptical of that phrase, because when I've heard it, it is usually empty. It is a label used in place of an actual plan to do more with less.  Oh sure, we can talk about productivity tips and all-pairs and getting testers involved up front, and try to make something valuable out of it, but when I hear that phrase, I tred very carefully.

Which tends to inevitably lead us to the Cost Of Testing - how to decrease it - and the value of testing - how to increase it.  Long-time readers know that I've been working on a collection of interviews with testers, possibly to become a book, but it hasn't gotten much progress.

At the same time, several of my colleagues, led by Govind Kulkarni, have started to collaborate on a possible contributed book, themed around decreasing the cost of testing -- with real, concrete ideas and solutions.  I've been working with them as a contributor and editor, and it's just getting to the point that we are beginning to talk about the book project in public.

Me, I'm cautious about saying "I'm working on a book", especially when the contract isn't signed.  At this point, I think it's fair to say that I am working on a book proposal, and am asking the community for feedback.

Would you be interested in a book on changing the cost/value proposition of software testing?  If yes, what would you like to see in such a book -- and what things would you be disappointed if they did not appear?

I'd really like to hear from you on this.  Thanks!






Monday June 28th 1pm

If you don’t vote, please don’t complain! Everyone is welcome to participate in the debate…

Membership Leadership STP Community News Software Test Professionals Conference Test and QA Awards

Thank you to all of you that have been nominating your choice for the STP Luminary Award. As we approach the close of this portion of the award process, I thought I would outline the next steps in what we hope to be a summer of great debate, lobbying, and education about the top 3 people that have been chosen by you.

As I said, nominations close this Wednesday. At the close of business on Wednesday, we will start tallying the results of the nomination process to determine our top three nominees. Those nominees will be input into our online voting survey to be released on July 15th. Although you may not see your nominee on the list, we hope you will take a look at the top three vote getters and get involved in understanding why others have chosen one of these three luminaries.

On July 15th we will also include on the website, under the Luminary Award Crew, a little bit of information about each nominee. Some of that information will come from the comments of the people that nominated them for the award. Some information will come from the research we here at STP will do about the nominees, and then we will open it up to you. We will be opening up the Luminary discussion to all Basic Members (complimentary Membership) to comment and lobby for your choice to receive the award. We will not field the questions of “this is bogus, I don’t see (put nominee name here).  He/She should have been on the list.” We can only put forth the people you nominated. If you have not nominated someone you have 2 days left to do so! Once the nomination period is closed it is closed.  (Click here to nominate)

The voting will start July 15th and continue through September 2nd. You have plenty of time to vote and if you know who your candidate is “hands down” then vote immediately. But if your candidate is not in the top three, I hope you will take the time to learn why others chose these candidates, and make an informed vote for the one you would like to see win the award. I expect that the supporters of each of these candidates will have an active role in lobbying for their candidate but we will see. This debate could make the “dog days of summer” a little more tolerable and interesting!

I only ask that the debate be respectful of the nominees. I expect we will see plenty of differences in these candidate’s qualifications, and the discussion will revolve around the positive aspect of each of the nominee’s contributions to our industry. This debate should really be a great window into the qualities, values and accomplishments that we believe make for great leaders in our industry.    

The final stage of the process will be the actual award presented at the 2010 Software Test Professionals Conference & Expo at the Mirage in Las Vegas. I hope you engage in the debate and the voting so that we have the opportunity to present the winner with a community award we can all agree went through a fair, robust, and open process…

Good Luck to all of our nominees!


Friend SoftwareTestPro on Facebook
Follow @SoftwareTestPro on Twitter
Create or Join a Crew

Tweets you care about

  • tonegeek (Ryan ) YEEEEEEEEAH. Episode 30 is done, this is the one that im testing the mic on AND using new editing software! IT'S ALL GOOD!
  • HL7 (HL7) HL7 News: Green Hat Introduces Testing Innovation for Software AG: Green Hat ... http://bit.ly/aFIP6g
  • HL7Jobs (HL7 Jobs) HL7 News: Green Hat Introduces Testing Innovation for Software AG: Green Hat ... http://bit.ly/aFIP6g
  • kservos (Tania) (c'd) needs to be done with this software testing bc I can't just read specifically what the tester has done to get said results >.<
  • HL7Tools (HL7 News & Tools) HL7 News: Green Hat Introduces Testing Innovation for Software AG: Green Hat ... http://bit.ly/aFIP6g
  • netdeveloper1 (dotnet developer) manual software testing - http://www.outsourcingdotnetdevelopment.com/manual-testing.html
  • youfollowme2 (Abad Perez) RT @zippycart: Sam’s Club Testing Online Loan Program: July 8, 2010 By the ZippyCart Ecommerce Software Reviews Content T... http://bit.ly/cmcCrl
  • zippycart (Scott | Nick | Amy) Sam’s Club Testing Online Loan Program: July 8, 2010 By the ZippyCart Ecommerce Software Reviews Content T... http://bit.ly/cmcCrl
  • TheWorldNews (The World News) Testing Suites Show Firefox 4 Not Quite Caught Up in Speed: We ran our own Mac-based performance tests last week w... http://bit.ly/9wBB3f
  • Biebs2NFLDagain (Hailey Bieber ;)) this is a pic I made of @justinbieber while testing out a new software :P Like it? I make icons and BGs too :D http://twitpic.com/23i1jo
  • wattbike (wattbike) @billyfishWORC We are just doing the final testing of that at the moment, will update here when the version of the software is live
  • tech_career (IT Jobs, India) Excellent opening for AAA Testing Professionals for..., Bangalore, 4 - 6 Year Exp,Software Test Engineer Telecom... , http://bit.ly/aTtGs1
  • tech_career (IT Jobs, India) Test Engineer, Hyderabad, 2 - 3 Year Exp,Software Test Engineer , Testing, automation,sdlc,stlc http://bit.ly/c4kEz8
  • WB2ColoradoRSS (WB2ColoradoRSS) Now: Testing Suites Show Firefox 4 Not Quite Caught Up in Speed http://ow.ly/183ar6
  • TrainingCareers (Training Careers) Training Job @ http://training-central.com Software Testing, Training, Technical Support - NY-Jamestown, Libera, Inc. is hiring software