I’m not making this up!  Maslow went as far as listing belonging as one of the primary psychological needs in his famous pyramid, on top of Safety but less important than Esteem.

This means that for you it is (obviously) extremely important to make sure no one is running behind you with a loaded gun; but right after that it is more important for you to feel that your group likes you than it is to know they appreciate you as a professional tester.

If you haven’t understood how this is making you a bad tester, then let me explain a little further…

We are paid to criticize our peers

Our main task as testers is to criticize the work of our peers and to provide visibility into the status of our projects.

Criticising in itself is not a bad thing!

The definition of the word is actually “to consider the merits and demerits of (something) and judge accordingly“.  For example think about the movie critic who writes a positive review of a movie, or the food critic who recommends a restaurant to his readers.

For us as testers this criticism is translated, in the simplest ways, to:
- Test the product developed by our programmers and set our tests to pass or fail.
- Report bugs when something was not done in the way it should have been done.
- Raise flags when we understand the team is not working in a way that will allow the project to be released on time, on scope or with the proper level of quality.
& finally
- Write reports that give an overview of the status of our project:  Good or Bad, and why?

In short we are charged with the job of investigating the work being done by our peers, and reporting to management whether this work is good or bad for the project.

But I don’t want to get my friends into trouble…

When the work is good then we obviously have no problem telling this to our bosses.

But when the work is bad: when tests keep failing all the time, when bugs are raised in large numbers, when the application is not on the state it should have been by this stage of the project…  Then we need to come to our management with bad news that will most certainly get some of our “friends” into trouble.

It is normal to feel remorse for escalating bad news to management, specially when we know that our friends are not completely to blame for all the delays and issues happening in the development process.  So what can you do?

Maybe we need to be “team-players” and help our friends?

How about we stop reporting all the bugs in the Bug Tracking System?  Instead we can send “off-the-record” emails with issues found to our developers, and this way the statistics don’t look so bad for them.

But then what will happen when one of these bugs escapes from the system and is found by a customer?  At that time we will be asked how come we didn’t find this critical bug during the testing process???

Or maybe we can delay some of our tests because we know that the system is not ready and most of them will fail.

Then what will happen when the project manager comes and says that we are the one to blame for the delay of the whole release because we were not able to run our tests on time???

All of a sudden, because we became “team players” and didn’t do our jobs correctly, we became the focus for all the issues in the project!

The best path is to be transparent and professional in our work

It is understandable that you don’t want to get any of your “friends” into trouble, these are the same guys you are having lunch with every day and the same guys with whom you play poker or basketball one night a week.

The best way to help your team mates is by using the same tools that may harm them in order to improve the way they do their work.

For example:

Give them constant visibility into the statistics reflected by your bug findings.  Have a process and a mechanism in place that helps them to see the “story” told by the bugs being found and their trends.  Help them to use this information for their own benefit and to have the ways of proactively informing management of any issues before you get to them with the “bad news”.

Make sure they are constantly aware of your testing approach and schedule.  If you are going to be doing extensive tests on the new functionality give your team the time to have the feature ready on time, share with them the tests you will run so that they are not “surprised” by the bugs that you find.  Remember that you work together so you don’t need to “hide” your scenarios from them in order to find more bugs.

Share your results with the development before you send them management, this will give them time to add their own comments and so help you provide the best possible picture of the situation.

In two words: Working Together

It took me some years to realize this, but as testers we don’t work against our developers, we work for them!

Part of our job (among other things) is to help them write beter software, and to do this as quickly and effectively as possible.

We need to make sure they understand that we want to help, and at the same time to educate them in the ways in which they can help us to do this job better (but that I will leave how to educate them to a different post).

So, the next time you feel that your friendship is getting in the way of your work, remind yourself that you are there to help your friends in their work and not to cause them any harm.

Think about the PROFESSIONAL things you can do in order to assist them in their work.

The reason I call these engineers “interesting” is because they constantly brought to the team all sorts of new stuff (technologies, methodologies, articles, tools, etc) that ended up making our work more effective and even more enjoyable.

This is one of the things I like most about PractiTest (the company I’m working in today).  The team is aligned with the philosophy that experimenting with new stuff is not only fun; it is also the best way to improve our product, to fine-tune our methodology, and even to increase the effectiveness of our company.

Of course there are costs to this!

It starts with the fact that instead of working on a task that is part of your formal project plan (e.g. testing or developing some feature), you take this time to deliberately work on something that is not related to the product or to any of it’s milestones.  This is not something easy to do in a culture where we are “running against the marketing clock” all the time!

You also need to take into account that many times (even half the times!) you will start experimenting with something that ends up providing no real value.  And so in fact you are throwing away valuable time that you could have been investing in something more traditionally “useful”.

Sometimes you need to strike-out twice before hitting a grand slam!

It’s been a while since I watch a baseball game, but I remember someone once saying that “when you hit a grand slam in the eighth inning no one remembers that you stroke out twice before during the game”.

I think the comment should be even harsher than that, for example “in order to hit a grand slam you will need to strike-out a couple of times before during the game”.

Or in engineering therms: in order to find the things that will help us improve the work of the team you will need to waste time checking some other stuff that will not provide any real value.

Make it part of your professional philosophy to “waste time” exploring new things

And so my bottom line is that you should make it part of your professional culture to “waste time” and to take chances exploring new technologies, methodologies, tools and approaches that may improve the value of your work.

Don’t expect people to learn and look for things only during their free time (nights and weekends).

Take scheduled time aside, even from hard pressed projects, to learn and to explore.

Not only will this help you to keep your team motivated (giving people time to learn new things as part of their work is one of the best ways to keep the right employees motivated), but it will also provide you with the Grand Slams that will keep pushing your team forward.

Does your company encourage you to “waste time” exploring?

Please leave your comments telling us how and what have you achieved with this time!

I remember meeting Scott in a testing conference some years ago.

Scott was the guy running around the conference, making sure everything was working perfectly and personally verifying that all the attendees where gaining the biggest value from the event.

From the very beginning it was clear that Scott was a pragmatic and value-oriented kind of person.

This impression of realistic pragmatism was also palpable in the way he talked and discussed about testing-related topics. Always approaching testing from a value-adding perspective.

Looking back, I think that Scott helped me to fine-tune my testing approach of always placing “the customer” in the centre of my testing practices.  But the revolutionary part of this approach resides in the correct definition of who “the customer” really is in each of your testing projects.

Surprisingly enough, in most projects this figure (the customer) is not the end-user of your product but the “internal customer” of the information you are providing with your tests and bugs; your peers and management team looking for the information necessary to make the best possible decisions about their projects.

I am really happy I asked Scott to answer my five testing questions, as I think he gave us some very straight and insightful answers.

To quote a couple of phrases that really stuck to me:

“Counter to what many testers want to believe, testing is not about finding bugs, is not about quality, is not about the user, is not even about the software – at least not directly.”

or

“…if you feel like your organization sees what you do as easily replaceable, it’s time for you to raise your game and start adding value that isn’t seen as easily replaceable.”

and even

“Businesses don’t *really* care about “low-defect” software. What they care about is software that makes or saves them the most money, as quickly and cheaply as possible.”

Now go and read the whole set of answers!

About Scott

Scott Barber is a renowned tester, speaker and writer on testing related topic; specializing in the area of System Performance Testing.

Other than providing consulting services as part of PerfTestPlus, he is constantly speaking in conferences and gatherings about a number of varied testing topics.  He has co-authored or contributed to 4 testing books (Performance Testing Guidance for Web APplications, Beautiful Testing, How to Reduce the Cost of Testing, and Web Load Testing for Dummies), and has composed literally hundreds of articles and papers on the topic.

In the past Scott served for 4 years as the Executive Director of the Association of Software Testing.

And now, let’s dive into Scott’s answers…

5 Testing Questions

1) What is the role of testing in today’s “typical” development process or organization?

I like that you ask what the role of testING is, as opposed to the role of testERS. I think the distinction between those two things is both important and frequently overlooked.

Today, I don’t think there is *a* “typical” process or organization, but I do think it’s fair to say that there are 3 “typical” roles that testing plays. These roles appear in a variety of combinations and levels of focus in different processes or organizations.

· Help Developers write “good” code: detect and resolve issues *before* story acceptance or tagging a build as done.

· Find bugs: most frequently done one sprint or build behind development and includes the defect reports, triage meetings, etc. typical of iterative approaches.

· Ensure compliance with legal/regulatory standards: Generally limited to “Release Candidate” builds, *shouldn’t* find bugs, but always does; thus jeopardizing the release date.

I think these three roles are actually just different aspects of the same overall mission of testing within a for-profit business: to help the organization make or save money. Counter to what many testers want to believe, testing is not about finding bugs, is not about quality, is not about the user, is not even about the software – at least not directly. Testing is all about the bottom line on the accounting spreadsheet. Bugs, quality, the user and software are simply things businesses take into account when trying to turn the highest profit they can get away with.

Businesses don’t *really* care about “low-defect” software. What they care about is software that makes or saves them the most money, as quickly and cheaply as possible. Once the business perceives that it will cost more to fix a defect than it will cost to ship that defect, the software is going to ship.

In reality, that makes testING a supporting role of business risk management, not actually a role related to improving or assuring the quality of the software product.

2) Which are the most important traits a tester should have (or should develop) in order to succeed today?

Seek to understand what makes businesses successful. Learn to think like a business executive (at least sometimes) when you are testing. Understand business risk management and the reality that as a tester, you are a cost center, not a profit center. No one (in their right mind) wants to have to pay for testing – sure they want the information, but they’d rather not have to pay for it, so you’ve got to make sure that information is valuable in *their* eyes.

Of course, good business sense is useless to a tester if they are a lousy tester. I think to be a really good tester today one needs to be far more social, collaborative, cross-functional and participatory than was common when I started testing well over a decade ago. Of course, all of those traits we commonly associate with testers like curiosity, technical savvy, oral & written communication and industry expertise still apply.

3) There have been a number of publications, talks and twitter threads lately about the possibility that testing is in fact a “Dying Profession”.

Do you agree with this statement?

In the sense that testing as a profession is in the middle of a dramatic transformation, yes, I do agree. In the sense that testing as a part of software development is somehow less important or necessary, no, I do not agree.

I think there are certain types of testing that can be accomplished perfectly well by people whose primary job is *not* testing. I think there are other types of testing that absolutely demand the skills, knowledge and specialization that you only expect to find in someone who dedicates their career primarily to testing. I think organizations are starting to figure out which is which, and that as they do they will assign testing tasks where it makes the most sense as opposed to automatically assigning them to the people with “tester” in their title.

What does that mean for testers? It means that if you feel like your organization sees what you do as easily replaceable, it’s time for you to raise your game and start adding value that isn’t seen as easily replaceable.

4) Looking back at your successful professional career, can you point to 2 or 3 milestones (or people) that made you the testing professional you are today?

I have to give my father a lot of credit for teaching me how to think for myself, how to solve problems, how articulate concepts through analogy. As a kid, I was sometimes annoyed with how he wouldn’t just answer my questions, and seemed to turn everything into what I now know to be called “teachable moments”, but in retrospect, I am grateful. The skills and qualities I developed as a result have been absolutely invaluable in my career and my life.

Mike Kopecky deserves credit for recruiting me into testing when I didn’t even know that software testing was a career. At a time when I was looking for a new job, he recognized that my personality coupled with an Bachelor’s degree in Civil Engineering, a Master’s in IT, 4 years of experience as an active duty military officer, 18 months as an Information Engineer/Analyst, and year as a configuration manager/DBA/System Administrator added up to “future tester.” When I called and asked him to help me update my resume, he said “Don’t worry about that, comes work with me, we need a Performance Engineer.” I replied “What’s that?” He responded “Don’t worry, you’ll like it.” And that was all it took, I’ve been in testing ever since.

What finally solidified my career path was Ross Collard inviting me to co-found WOPR. That experience, along with the all of the great people it introduced me to who I have learned a great deal from along the way, are what allowed me to grow from a “very good employee performance tester” to the industry influencing thought leader I am today.

5) What would be your most important piece of advice to a beginner tester today?

I’d offer 2 pieces of advice to beginning testers today:

· If you don’t love what you’re doing, do something else (or at least try doing it for a different organization). Testing is not something you’re going to be happy doing for the next 10, 20, 30 years if you don’t love it today.

· Don’t ever let yourself believe that there is one way, one term, one process, one tool, one report, or one anything else that is either “universally best” OR that all testers agree on. There are only two things that seem to be even close to universally true when it comes to testing – things are constantly changing, and if you put 3 testers in a room with a testing term or topic to discuss, no more than 2 of them will ever agree at the same time (except maybe that they’ve had enough disagreeing and want to do something else now).

Thanks to Scott

Once again I wanted to thanks Scott Barber for his answers and for taking the time to explain his testing approach and real-life wisdom.

I am sure many testers will get a number of ideas on how to approach their testing in a concretely different way today, and generate more value to their organizations.

We do this around the Jewish holiday of Passover, that is set to coincide every year with Spring in the Northern Hemisphere.

But from talking to friends who are not Jewish and don’t live in Israel I understood this is more than just a religious-driven tradition.  I even found an entry on Wikipedia about it under Spring Cleaning.

We are better at getting things than getting rid of them

A Universal truth about humans is that we are good at collecting stuff.

We have no problem buying a new t-shirt even if we already have 35 in our closet.  Or getting a stack of 50 blank DVDs to burn, even if we have not yet finished the stack of 15 DVDs we bought a year ago, just because there was a sale in our computer hardware store.

We like getting stuff, it’s in our nature…

On the other hand, we are really bad at getting rid of it.

When we realize the pants we took out of the closet are not our size anymore , or that the t-shirt you wanted to wear has been washed so many times it became see-through, we don’t immediately throw them away or given them out to charity.

We place them in our closet in the area of “things that I don’t think I’m gonna wear soon” and forget about them.  Until one day this area of things you are not gonna wear takes up half your shelves!

But don’t worry, you are not alone!  Most of us are just like you.

Nor does this only happen with our clothes/hardware/kitchenware/kids-toys/etc.  This happens with everything, including with our testware.

The art of collecting useless testware

The same thing I just described about t-shirt, DVDs and pants happens also in testware.

When you are looking for a test case to check a feature being updated in your application, you will look for a similar tests for approximately 45 seconds before deciding that you “might as well just write a new test for it”  (without realizing there are 2 tests that could have been slightly modified to cover this change).

Or, when part of your application is completely re-written and many of your tests become useless, you don’t delete them right away because you may “need to release a patch further along the road”.  But 2 years later, when there are no patches in sight and none will be created, you still keep these tests because “they are not harming anyone there”, and plainly you don’t have the time to review them and delete them right now…

If you look into your testware right now, and if you have not done any cleaning lately, it is reasonable to assume that between 10% to 15% of your tests are not relevant anymore, and another 15% to 20% of them need to be updated to match the current state and functionality of your AUT.

But this is not hurting anyone, right?
WRONG!

So you have a lot of useless tests, so what?!

It is not like you are paying more to electric company for them!  You are also not wasting any extra computers to store them!  And as they said countless times lately, storage space is so cheap nowadays that you should not even think about it anymore.

If so, what’s the big deal with having useless tests in my testing repository???

The big deal is that you are not looking at the problem from the correct angle.  The waste is not in computers, or electricity or storage…  these were the expensive resources 10 or 15 years ago.  Today the most expensive resources are the human resources – your team – and by having many useless tests in your repository you are wasting their time!

You are wasting the time it takes them to find the tests they need to run.  You are making it harder for them to understand if they need to create new tests or if they can re-use some the tests that are already in your Test Management System.

And maybe most importantly, by having a mix of good tests and bad tests in your repository you are increasing the chances of a tester that will run a wrong test and miss part of the issues that a good test would have detected quickly.

So what can you do about it?
3 steps to help you clean your testing repository quickly.

OK, so let’s assume you cannot drop what you are doing right now and take 3 days of all your team to go over your testing repository and clean it up.  There are still simple things you can do to start improving your process and make it easier for your team to clean up your tests in the future.

Step 1 – Organize your testing repository

The first step is the one that may provide the biggest immediate impact.

Most times the main problem with your tests is that your testers are not able to find them.

Once this starts happening then they either write duplicate tests (creating more useless stuff in your repository) or choose to run tests informally from their heads (maybe missing part of the things they should be checking).

To stop this from happening you need to organize your tests in a way that will help your testers find them easier and faster.

At the risk of being a little pretentious or immodest, we provide a pretty nice solution for this in PractiTest in the form of our Hierarchical Filter Trees, that let you organize and catalogue your tests in multiple ways at the same time (automatically organizing everything based on your tests’ fields).

One of the coolest things of this feature is that it allows you to “place” tests that check more than one area of your product under each of these areas (e.g. a test that checks your Website and your Reports Console and your User Management Module at the same time) instead of having to choose only one area (or folder) for each of your tests.  But enough of PractiTest for now…

If you are using any other test tool or even Excel to manage your testing you should try to create the most intuitive and simple classification in your test tree, use a convention that will let you manage test cases and the rest of your test ware.

When choosing a convention, the most intuitive one is usually based on your application’s structure.  Start from the products or components, and then go down 2 or 3 levels to Sub-Components, Features and even Screens.

Your convention also needs to take into account cases where one of your tests covers more than one product or components (like the case of end-to-end testing scenario we talked about previously).  You may choose to look for the “main component” for each test and set it under this categorization, and if this is not clear enough to have a number of “central components” where all the integration tests should reside in.

Conventions will never be perfect, and there will always be cases that fall out of it.  But it is better than every tester placing tests where he thinks they should be…

Step 2 – Mark your tests while you are running them

The second step aims at making it easier for you to do your Spring Cleaning once you have the time for it.

Add a new field or parameter to your tests that each of your testers can set when they are either running or reviewing their existing tests as part of their work, and ask them to them to set in this field one of the following 4 values:

Ready – if the test is OK and it can be run as is.

To Review – if you think something can be update in the test to make it better.  In this case you may also want to add a comment with why you set it to it.

To Repaire – when there is something that needs to be changed. In this case too, write down what you think should be fixed.

Obsolete – when the test should simply be discarded.

By having these fields you will be able to review them a lot easier.  You will also be able to see what tests are “abandoned”, by looking for tests that after a couple of cycles still have no value filled on this field.  Abandoned tests are usually useless and should also be discarded.

Step 3 – Set time aside in your schedule before your next “big project”

The last step is the trivial one.  You need to set time aside to clean your tests.

The best time (and often enough the only time) you will have is in the short span between projects.

If you plan this ahead and make sure that no one else has already assigned a task for you and your team, you may be able to review all your tests.  In case you are not able to go over all of them at least start from the most critical areas of your product.

Make it a habit for your testers to work correctly

Once you have a clean repository it will be easier for you as test lead or manager to ask your tester to work in a cleaner and more organized way.

Ask your testers not to write unneeded tests, and to always place tests in their correct are within the test tree.

No team will be perfect and with time your tests will again run into a small or larger level of chaos, but at least when you start from a clean page it takes it a little longer to reach this state.

It was about 4 years ago, as I started doing testing consulting, I began running into work opportunities where companies were looking for help in implementing Agile Testing as part of Agile Development efforts.  I knew about Agile mostly from what I read on posts or discussions in the web, but I had no real Agile experience.  And so I began by reading as much as I could on the topic.

I found a lot of people commenting about how testing was lagging behind on the Agile school of though, and others going as far as saying that in the Agile world testing was bound to be extinct as a profession (sounds familiar even today, right?).

But the first and most complete piece of information that actually explained how to work and provide real value as a tester in an Agile development team came from a book released that year (2009) writen by Lisa Crispin and Janete Gregory simply and accurately called “Agile Testing“.

This book was extremely helpful to me as a testing consultant and a test manager in a number of Agile teams.  It  explained how Agile testing is not very different from the testing we’ve been doing all along, but how the holistic approach to testing in the complete development team needs to be “a little” different in order to succeed.  And how these “little differences” need to be pushed and lead by the testers in the team.

The interesting fact about their “testing book” is that I started seeing it and being cited by many development leads and R&D managers, not only when they talked to their testers but also when they communicated with their whole development team.

So in short Lisa, I wanted to personally thank you (and Janet) for the help you gave me while starting my path in the world of Agile testing.  And also to thank you on behalf of all the readers, tester and programmers, who today understand the concept of Agile testing better because of your work.

About Lisa

To quote her personal site “Lisa Crispin is an agile testing coach and practitioner”.  You can find her talking and contributing in many Agile Testing conference and forums.

She has worked in IT for a number of years not only as a tester (QA Manager, Director, etc) but also as a programmer, analyst, support engineer, etc.  Maybe helping to support the theory that the best QA engineers actually come from a well rounded technical background and experience, helping them to provide value to the whole organization (read more on her answer to the first question bellow).

A thing that I always find interesting when reading Lisa’s posts or her twitter feed, is her work with donkeys – I guess people need to have non-technical occupations in order to continuously feed their technical minds

And without much additional introduction, let’s move to Lisa’s very interesting answers to my 5 testing questions…

Five Testing Questions

1) What is the role of testing in today’s “typical” development process or organization?

Hmmm, it’s hard for me to say what is “typical”. I still see many dysfunctional development organizations around, where no value is placed on quality, and teams labor under ever-increasing burdens of technical debt. Often, in those types of companies, testers are undervalued, and often, the people hired as testers lack skills and don’t have any passion for testing and quality, which reinforces the view that testers are some kind of second-class team member.

The role of testing should be, IMO, an integral part of development, along with coding and other activities needed to create a quality software product. As Elisabeth Hendrickson says, testing isn’t a “phase”. The successful teams I’ve been on – whether waterfall or agile – start each new feature by writing tests. They continue to test, code, test some more in small increments and iterations until they have something of value to release. There are definitely testing activities that still need to happen once coding is thought to be complete, such as exploratory testing, so that we learn what else may still be needed in the feature or set of features.

I know many testers nowadays who play an integral part in helping business stakeholders decide on the features that provide the necessary value, get examples to illustrate desired behavior, and turn those examples into tests that help guide development. These testers collaborate with all roles on the development team to deliver valuable features frequently, while keeping technical debt to a manageable level so the team maintains a steady cadence.

2) Which are the most important traits a tester should have (or should develop) in order to succeed today?

Attitude is everything. Curiosity, a passion for learning, willingness to take on any task, desire to collaborate with team members and customers in other roles, courage to experiment. The skills needed to make all that happen can be learned.

Of course, the ability to learn requires that the company nurtures a learning culture and ensures “slack” time for learning, experimenting, innovating, failing, improving.

3) There have been a number of publications, talks and twitter threads lately about the possibility that testing is in fact a “Dying Profession”.  Do you agree with this statement?

Janet Gregory and I recently refuted the “Testing is Dead” myth in our keynote at Agile Testing Days. Development teams should examine their environment and their team cultures.  Some teams may not need a professional “tester”. I have met teams whose developers had enough testing expertise and were able to produce quality software without any team members in a testing role. However, these tend to be teams working on highly technical solutions, such as message handling systems or other types of embedded software, whose customers are other development teams. What I’ve seen though, is that these teams tend to be working on more technical solutions.

As soon as a system has business users and the domain knowledge is critical, the “no testers needed” model tends to fall apart. Teams need someone who can focus on the bigger picture, and ask questions to reveal hidden assumptions. It’s hard for a programmers to keep up with the technical knowledge needed to program really well, and focus on one tiny part of the system that they’re currently coding, and be thinking about the bigger picture as well.  In my experience, testers think of questions that don’t occur to programmers. For example, a business stakeholder may assume the development team will provide adequate security for an application, while the programmers focus on the functionality that the customers want and don’t even think about security.

The testing profession is changing fast, but I’m confident that we’ll always find ways to contribute significant value.

4) Looking back at your successful professional career, can you point to 2 or 3 milestones (or people) that made you the testing professional you are today?

My second manager, back when I was a baby programmer, taught me something important about leadership that has guided my career. He said leadership means helping others understand the value that you and your team contribute. We need to make our contributions visible. Lots of people, especially women, feel this is “tooting your own horn” or bragging. But it’s especially important in a culture where testing, and testers, have often been under-valued.  Throughout my career, I’ve helped managers and other teams understand the risks involved in a given project, and what we can do to not only mitigate those risks, but deliver a really great product that delights our customers.

Another big milestone for me was learning about Extreme Programming. In the 90s I worked on a great waterfall team that had complete automated test coverage from the unit level on up, continuous integration, automated deployment, and took time to do enough exploratory testing. The waterfall methodology was fine, because we worked on a database product that only needed to be released once a year. When I moved to a web startup, suddenly that waterfall process didn’t work anymore. What was worse, everyone was in such a rush to get software out the door that they ignored good practices and just hacked out solutions

In 2000, one of my friends went off to form their own startup, and they gave me a copy of Kent Beck’s Extreme Programming Explained.  Reading it was an epiphany. It was all focused on quality, and collaborating with the customers! I had to try it. I begged my friends to hire me as a tester, and never looked back. The Whole Team Approach to quality is exactly right for me.

5) What would be your most important piece of advice to a beginner tester today?

Learn. Be courageous. Ask for help. Try experiments. Ask your teammates to pair with you, whether they’re other testers, programmers, customers, analysts, database experts, whatever. Pairing is the fastest way to learn.

Take advantage of all the free resources today: ebooks, blogs, webinars, videos from conferences, online courses. Join your local testing user group, or if you don’t have one, start one! Or, start a testing Community of Practice within your own company. We need a place to share experiences and ideas. Start up a Lean Coffee group. Join Weekend Testing sessions to learn from testers all around the globe. Join the Software Testing Club to tap into a similar global testing community.

Learn by doing. Blog about your experiences. Present at a local user group or a conference. Go to a code retreat. Work through a book such as Everyday Scripting with Ruby by Brian Marick to learn a scripting language. I’m often asked if programming skills are required to be a tester on an agile team. You don’t need to be a programmer, but you should be what Janet Gregory calls “technically aware”.  At the same time, you must also invest time to learn your company’s business domain, so you can help your customers solve their problems. You need to be able to communicate with everyone on your development team, and with your business stakeholders.

That sounds like more than one thing, but I think the focus should be: “Learn!”

Thanks to Lisa!

Other than the personal thanks I gave her at the beginning of this post for her contribution to my agile testing career, I wanted to thank her for the insightful and complete answers she provided to my five questions above.

I found specially true her advice to “Learn. Be courageous. Ask for help. Try experiments…”  These are the things that will really make a difference for most testers in the long run (as opposed to finding one or ten additional bugs in our current testing project).

Hand in hand with her self-description of being a “testing coach”, I think that many testers will find in these answer tons of useful knowledge and ideas on how to continue improving and expanding our test careers.

If any of you has feedback for Lisa and her answers feel free to leave them as comments and I will make sure they reach her.

More to come…

I am already working on the next Five Testing Questions interview, but if any of you has ideas on additional testing experts and personalities you’d like me to interview feel free to let me know by sending me an email or leaving a comment.

If you google the therm “Military Intelligence”, among the first results you will find the following Creed of the (US) Military Intelligence Corps:

I am a Soldier first, but an intelligence professional second to none.
With pride in my heritage, but focused on the future,
Performing the first task of an Army:
To find, know, and never lose the enemy.
With a sense of urgency and of tenacity, professional and physical fitness,
and above all, INTEGRITY, for in truth lies victory.
Always at silent war, while ready for a shooting war,
The silent warrior of the ARMY team.

I couldn’t find the source to quote it, but this sounds close enough to what I know about Intelligence Officers from some friends with vast military experience.

Why am I writing this about “soldiers”?

The answer is simple.

A short while ago Jerry Weinberg commented to one of my posts (republished in Tea Time with Testers) that he was looking forward to my explanation on why I think the work of testers is in many ways similar to the job of Military Intelligence Officers.

I believe he was referring to the following quote from my post “To Protect and Serve“:
“… In many cases we are serving as (Military) Intelligence Officers to our Organizations, helping to make the most complex and challenging strategic and tactical decisions.”

Some of you may already know that Jerry is high in my list of admired testers, so I would not even dream of not answering his questions.  So I decided it was time to take upon this topic with a post of its own.

Back to basics: what is the role of the QA Tester?

In the past I wrote my definition of QA (or Testing) Intelligence as follows:

To provide concrete, relevant and timely information
captured from multiple data sources and using many disciplines 
to help our stakeholders make their tactical and strategic decisions.

This definition is composed of 3 parts:
(1) We provide the right information
(2) We gather the information from various sources
(3) The aim of this information is to help make the correct decisions

Mental Exercise:

Before we move forward I would like you to perform the following exercise.  I promise it won’t hurt, and it will take you less than 4 minutes to complete it.

PART 1:

Read the definition of QA Intelligence above and think about your current testing team.

- Does the definition help to define the objectives of your team?
- In a high level, does it help you to accurately prioritize your work and to explain to others in your team and outside of it what it is that you are achieving?

PART 2:

Now, think about a Military Intelligence Officer working alongside the Top Generals of an army, and once again read the definition of QA Intelligence but think about the Intelligence Officer’s work.

- Does the definition help to define the objectives of the Intelligence Officer?
- Does it help to prioritize his work and explain to other members of his team what are his responsibilities and tasks?

Conclusion:

In many ways the objective of the QA Tester and of the Intelligence Officer are very similar.

Each of us in his or her own contexts, are tasked with providing information that will help our superiors and the rest of the team/unit to do their work better and to make the correct decisions.

We are in the business of Information

Just like the intelligence officer is not tasked with fighting the enemy in hand-to-hand combat, as testers we are not tasked with writing the code that will be delivered to the end users.  Our job is to provide information support to the people who are making the strategic and tactical decision as well as those in the fighting and coding lines.

This doesn’t mean that we don’t have an intricate and highly technical job.

Many times our jobs are even more technical than “only doing the coding”, because we need to think about and simulate strange but realistic scenarios where customers will be using our product in unforeseen ways, and that way seek out the issues that may be hiding under these extreme conditions.

Have you ever seen testers using “counter-bug-intelligence” tactics exemplified by the phrase: “if I was a bug, where would I be hiding…”?  I know I have!

Now seriously, one of the most complex jobs of QA Engineers is to make sure we are providing the correct information.  By correct I don’t (only) mean the right data from our test runs, but the actual information derived from processing all our data points and putting together an image that is both accurate and informative.

Not only that, but we need to work with incomplete information, making assumptions and explaining them as risks of things that may or may not happen.  Does it sound like guessing where the enemy is hiding and how they will behave?

In the end of the day our stakeholders don’t have the time to go over all the results and assumptions.  They expect us to do this processing for them.  All they want to receive are the synthesized pros and cons, described as simply as possible, to help them make the right decision quickly.

We gather information from multiple sources of raw data

Another thing that connects between testers and military intelligence officers is that we work with a large number of data sources and types.

Just as we need to run functional tests, API tests, Load tests, etc., intelligence officers need to gather data from satellites, personal observations, spies, etc.

For us is not only about bugs and tests but also about statistics of usage and different types of user behaviour as well as technological changes and different types of platforms and conditions where our products may be used.  In a similar way, for them is not only about the enemy and their weapons, but about the general population in specific areas and the political, economical and even ethnical connections between different factions of a war.

Both us and them need to provide concrete, concise and timely readings of all this information, presenting the current status of affairs and an appraisal of the future based on a limited number of assumptions.

Integrity and truth

Finally, I couldn’t help but notice something that was written in the Creed and connect it to words mentioned both by Jerry Weinberg and James Bach in their answers to my 5 Testing Questions.

The creed says:
“…and above all, INTEGRITY, for in truth lies victory.”

When I read this, I realized it was similar to what both Jerry and James answered to some of my questions.

Like when Jerry answered that the most important piece of advice for a tester would be to:
“Never lie…”

And when James wrote that among another number of traits a tester should always have:
“…a strong sense of ethics.”

What do you think?

Are there other similarities between QA Testers and Military Intelligence officers?
Do find another profession where with which we share many of the same traits and challenges?

Please let us know by leaving your comments!

A couple of weeks ago I published a post where I asked Jerry Weinberg 5 testing questions that I had been pondering over for the last months.  The questions were about the stuff that makes us better testers, as well as about the future of the testing profession.

After getting tons of feedback and noticing the large amount of attention around this post, I realized it would be even better to ask these same questions to additional testing experts to get their insights and advice on these topics.

With this in mind I made a list of testers and QA experts I follow and respect, and I decided to contact them one by one.

I was extremely happy when one of my favourites testers, James Bach – the original buccaneer tester – told me he would be happy to reply to my questions so that I can share his answers with you here.

About James

To me James Bach is one of the biggest names in testing.

If you have not seen or heard him up to now just google his name and check out a couple of the life video presentations that will jump at the top of your search.

I met James for the first time in one of the Start Conferences over 10 year ago.  I remember he sat on a room and challenged us (the audience) to name any application, and he would find bugs on it even if he was not familiar with it up front.  To jump to the chase: he not only found bugs on the app, but he crashed it TWICE.

This experience was not only the first time I met him in person, but also the first time I perceived the strength of Exploratory Testing (a name that has become synonymous to JB for many of us).  Together with Cem Kaner, James is the father and one of the most active contributors to Exploratory Testing and the Context Driven School of Testing as a whole.

He is a genial speaker and one of the most inspirational testers I’ve met.  One episode that jumps to mind is of an afternoon I spent with him where he introduced me to “the dice game” and through this exercise thought me one of the biggest tools I now have to train testers on the importance of rapid learning, data analysis and self-driven-understanding as a tool for testing problem solving.

There are too many additional things I could write about him, but in the interest of time and self learning I will let you find these interesting facts by yourselves (hint: you will want to start from his site – satisfice).

Five Testing Questions

1) What is the role of testing in today’s “typical” development process or organization?

I can tell you about the role of what I would call testing. It means questioning the product (through the mechanism of operating and observing it) so as to enable informed decision-making by the people who matter.

Testing is kind of like insurance. No matter what you do to prevent bugs, you still have to plan as if they might be there. So, we’d better go look and see.

2) Which are the most important traits a tester should have (or should develop) in order to succeed today?

A zeal for rapid learning is one, and I agree with Jerry Weinberg that courage is a vital quality.

I’ll add one more: a strong sense of ethics.

3) There have been a number of publications, talks and twitter threads lately about the possibility that testing is in fact a “Dying Profession”.  Do you agree with this statement?

No. That’s a dumb idea. It’s propagated by people who– SURPRISE– aren’t testers and don’t study testing.

However, I would say that Factory School testing (dim-witted script following) is something that I HOPE will die. And soon.

4) Looking back at your successful professional career, can you point to 2 or 3 milestones (or people) that made you the testing professional you are today?

0. Conversing with my father.
1. Dropping out of high school.
2. Reading books by Douglas Hofstadter and Paul K. Feyerabend.
3. Taking the Problem Solving Leadership course from Jerry Weinberg.
4. Reading the works of Jerry Weinberg.
5. Working with Cem Kaner.
6. Working with my brother Jonathan.
7. Working with Michael Bolton.

5) What would be your most important piece of advice to a beginner tester today?

1. Practice practice practice.
2. Build your colleague network.
3. Everything important about testing happens inside your head, not out on your desk.

Thanks to James!

I wanted to thanks James for taking the time and for sharing his insightful views on these topics.

I’m sure that both young and experienced testers will gain greatly from these answers.

Share your feedback

I’d like to invite you all to share your feedback.

- Do you have something to say to James?
- Is there a specific tester you think I should have on my list of testers and QA experts to ask these questions?
- Would you change part of these questions?

Let me know, leave a comment!

There are questions that I am constantly asking myself regarding testing and QA.  What value do we provide to our Organizations?  How can we be better testers?  How will the future of testing look like? etc.

If you’ve read some of my posts you will certainly be no stranger to these questions or to my point of view on them.

Some time ago I thought that it would be interesting to ask other testers what they thought about these questions.  And then I thought that it would be even better if I could ask not “any other tester”, but the one tester that is in my opinion one of the top contributors to the testing world (or the software development world for that matter), Mr. Jerry Wienberg.

The amazing fact is that when I asked Jerry if he would be willing to answer my questions he answered that he’d be happy to.

So here are the five testing questions I asked Jerry Weinberg and his insightful answers to them.

About Jerry

I am not sure Jerry Weinberg really needs to be introduced, but just in case you have been living in a cave and have never heard of him, I think that the best way to start an introduction of Jerry is to say that he is a special kind of person, a “people changing person”. Jerry has the incredible gift of, in his own words, “make you aware of the things you were not aware of” and thus change the way you look at things and the way you approach tasks.

On a personal note I had the pleasure of meeting Jerry once some four years ago at a CAST conference in Colorado Springs. From my first interaction with him I was blown away at how, with his down-to-earth approach and his always candid comments and tales, he could motivate and invigorate any person he was talking to at that moment, regardless if this persons was a world-renowned QA expert or a Jr. tester.

As part of the protocol I will list a couple of numbers and facts about Jerry:
- He was awarded the first ever Software Test Luminary Award back in 2010.
- He has written over 80 books and hundreds or articles.
- Between 1959 and 1963 he was the Manager of Operating Systems Developing in the Project Mercury.
- Since the 1970′s he’s been a consultant and is world known for his leadership workshops.

If you want to learn more about Jerry and how he has influenced the people around him you should read the book “The Gift of Time”, a collection of essays in honor of Jerry’s 75th birthday, edited by Fiona Charles and with contributions by James Bach, Michael Bolton, Esther Derby, Johanna Rothman, Jonathan Kohl, and other stars.

Finally, to get a quick taste of Jerry, and a more complete introduction on his work I invite you to check this short video in YouTube of his Luminary Award Acceptance Speech.

Five Testing Questions

1) What is the role of testing in today’s “typical” development process or organization?

It’s hard to answer this, because I don’t see a “typical” development process or organization. So, I’ll answer for the “best” development organizations I work with–maybe 5-10% of all development organizations (excluding the orgs that develop the hundeds of thousands of game apps).

In those best organizations, testing is involved in the development process from the very conception of a project. They show the other project teams what might be hard (impossible) to test and help develop test approaches in advance of any building of code. All through the process, they inform the project leads where the risks are lurking. That’s it, at a high level.

At the other end, those hundreds of thousands of games and trivial apps, I don’t have much contact except for the results of their work. Based on my experience as a user, my best guess is that these orgs have no testing at all, at least nothing effective.

2) Which are the most important traits a tester should have (or should develop) in order to succeed today?

Courage, communication skill, self esteem.

3) There have been a number of publications, talks and twitter threads lately about the possibility that testing is in fact a “Dying Profession”.  Do you agree with this statement?

That’s just a dumb slogan.
Testing has barely been born yet. As IT matures, so will testing. Without testing, IT will never mature.

4) Looking back at your successful professional career, can you point to 2 or 3 milestones (or people) that made you the testing professional you are today?

a. Working with Bernie Dimsdale (and thus indirectly with his mentor, John von Neumann).

b. Learning to be fluent in a second language.

c. Being a participant-observer with literally dozens of IT projects.

5) What would be your most important piece of advice to a beginner tester today?

Never lie, and never pay attention to those who say stupid things like “Testing is dying.”

Thanks to Jerry!

I wanted to thank Jerry for taking the time to answer these questions and for giving us more tools (both direct and indirect) to become better testers and increase the value we provide to our organizations.

You can read more about Jerry, his writings and all his extensive work, from his site - http://www.geraldmweinberg.com

For me, the choice would be -

To Protect and Serve

And not because we are “testing policemen” patrolling the “functional streets” of our AUTs (Applications Under Test, for the non-testers among us) looking for “criminal bugs” and placing them in “bug-tracking-jail” like petty thieves – although the mental picture is kind of funny

But because our responsibility as part of our team and organization is to serve our internal and external customers (e.g. the external end-users; and also the internal developers, product managers, executives, etc) and to protect them from making the wrong decisions about the product we are developing and the process used to develop it.

Protecting Who and From What Exactly…?

Boiling it to the minimum, I believe that as testers we are here to:

(1) Make sure that our teams are developing the right product – with the right features, answering the real needs of our users, without any unwanted issues, etc.

And at the same time:

(2) That our teams are developing the product right - following the process we decided to follow, without wasting unnecessary time, working in a socially and economically efficient way, etc.

We are here to provide visibility into the product and the process, to reduce the time to market uncertainty and to answer the $25,000 question or whether we are ready to release or not – in the past I’ve called this QA Intelligence.

And if all this wasn’t enough, in many organizations we are also been asked to lead the task of performing Risk Analysis and Management throughout the end-to-end development lifecycle.

Risk Analysis?

Simply put, as a tester you should make sure that if there are any risks that may affect the project from been completed on scope, on time and on budget, then these risks need to be identified, tracked, if possible avoided, and if not then they should be handled correctly.

I will write more about risk management in the context of QA & testing in a future post.  But for now I want to add it as another one of the tasks we are doing in order to protect and serve our customers.

Bottom Line… Are We Policemen?

It is true there used to be a stigma of seeing the tester as the bug-policeman some 10 to 15 years ago, and I believe there are some development teams where this might still be the case (let’s call these guys cave-men-developers, since they seem to be still leaving in the stone-age of software development).

But Reality seems to evolve, and so in today’s development practices, and as we become better professional testers and QA specialist, more and more of the actual tests and bug detection activities are carried on by developers and others members of our organizations.

And we as testers are taking on a different responsibility of guiding and steering the process in the right direction.  In many advanced of cases we are serving as (Military) Intelligence Officers to our Organizations, helping to make the most complex and challenging strategic and tactical decisions.

What do you think?

What motto would you choose for us testers, and why???

A simple approach

I think that a lot of my approach is directly derived from my Agile way of looking at work.

I don’t intend to sound like an expert on making things simpler (my wife would even say that I always make simple things complicated!), but I like to think that my approach to managing testing projects comes from experience and boils down to working as straightforward and simply as possible.

Basically I follow a small number of principles to help me organize the work of my team, making sense of the mess caused in today’s working environment characterised by the chaotic “right-here right-now” syndrome.

But enough chatter… Here are my 5 simple tips to help you keep your testing simple:

1. Divide and conquer

There are almost no real complex tasks, as long as you are willing to look for ways to break them into smaller and simpler components.

Many times I meet QA managers who explain to me how they manage their tests using a small number of (very long) excel sheets or wiki pages.  When I ask them why do they work this way they explain to me that they started with small documents that grew over time…

One of the first pieces of advice I give these managers is to divide and conquer.  By breaking down their very long and complex testing procedures into smaller and more modular test cases, they can gain flexibility and achieve faster and more accurate coverage.

But this advice is not only good for the size of test cases.  If you look into any testing tasks and break it down into smaller testing tasks then you will be able to manage your team more efficiently and provide better visibility to your internal customers.

Let me use a specially hard example that people always use to explain why some tasks cannot be run as modular activities – Load Testing.

If instead of scheduling all the load testing activities towards the end of the test plan (based on the premise that you should tests only once your whole end-to-end system can be tested for the performance and correct response time) you schedule smaller and more focused load testing sessions on specific components during the development process, you will be able to find issues faster and without the need to run complex load scenarios and even more complex analysis operations to find the actual bottlenecks.

My experience shows that by running more focused load sessions you will spend less time testing overall and reach better results than by working based on more traditional – once all is in place – scenarios.

The same goes for any testing activity, break your complex testing operations into smaller tasks and try to run them as soon as possible.  This will help you to find the issues fast and help your developers solve them even faster.

2. Keep your eye on the “important” ball/s at all times

Managing any services organization is always about juggling tasks (and yes, testing is a service).  You have many things that need to be done and only a small number of resources to do them.  No matter how much you try, you will not be able to do everything, specially not at the same time, and never with the level of coverage/thoroughness/attention you intended at first.

Once you understand this it becomes trivial that at any given time you can only focus your attention on a limited number of tasks.  So it is better to have a good definition of what are the important tasks that you want to focus on, and make sure all your team is in the same page with you.

A practical idea is to make sure that during your weekly QA meeting (if you don’t have a weekly meeting with your team then stop reading this post and set up a recurring meeting NOW, it will be the most valuable thing you do today!) have a whiteboard handy and each week make sure you write down the important tasks or things that should be on everyone’s radar.

Give your team the chance to comment on these tasks (or any they think should be there) and make sure you are not missing something important that they know and you don’t.

Leave the list in a place that everyone can see – not an email!

You can even start each week’s session by going over the list from the previous week and catching up on the events that happened around these tasks.

BTW, regarding the “juggling balls” analogy, an important part of it is to realize that you will not be able to keep all the balls in the air.  At one time or another you will reach your limit and some of the other balls just bounce…  It’s a fact of life and of project management.

3. Categorize tasks based on Importance, Complexity and Last Start Date

Directly related to the previous point, is the need to be able to categorize and prioritize your tasks.  This is the best way (the only way?) to know what is important and what tasks will be left to fall.

After working with many parameters, wighted algorithms, etc.  I know use a simple method where each one of my tasks (I am talking about the smaller testing tasks, and not of the very big tasks your Project Manager is always talking about) has 3  individual and unrelated parameters:

I) Importance – What is the business importance or the priority your company gives to this task?  Would it be risky if you don’t run this test?  -  High / Medium / Low.

II) Testing Complexity – What is the complexity level of the tests to be run?  This parameters incorporates the difficulty to test as well as whether this test can be done by any tester/developer or only by an experienced tester.

III) Last Test Start Date – An important parameter that will reflect when is the latest you can run this test and still be relevant for your project.

These 3 parameters won’t give you a calculated index or any way of automatically deducing what tasks should you be doing or when.  I personally don’t believe in these algorithms mainly because reality is constantly shifting in our type of projects, but they will help you to sort and filter your task backlog and make sure you always know what to focus at what time.

Remember that as a Manager, the task of managing cannot be delegated to any algorithm

4. Learn to say NO

Wow, this is a hard one!  Maybe the hardest one in the list.
You need to learn how to say to No, and also when to inform your customers that you cannot do something that you previously said you would do.

As I stated before, the reality of our projects is that they are in constant change (I remember someone once saying that the only stable part of our projects is change itself!), and as so you will need to be flexible and also to teach your internal customers that this is the reality of the game.

The math is simple: If you pull the blanket on one side of the bed then someone will be left out on the other side…

Stop committing to tasks you know up front you cannot perform, and start updating your users as soon as you understand you will not be able to perform a task that you previously committed to do.

Timely information is the key to making the correct decisions and compromises.

5. Give full visibility into your work

Hand in hand with prioritizing and saying NO comes the practice of providing visibility to all your work and tasks, as well as to the prioritization and the changes affecting your work.

Some managers think that if they “show all their cards” they loose part of their power, their flexibility and the ability to make thrir own decisions.  This could not be farther from the truth!

By working in full transparency with your customers and peers you will show how professional your work really is.  You will also allow them to provide inputs that will help you make better decisions for your team and for the success of your project and your company.

Do you have any other ideas or tips?

Transparency and sharing is the key to growing and improving.

If you have any other simplifying ideas share them with us!