Software Test Management | Testuff » blog

Absolute Beginners

admin — Tue, 24 Jan 2012 14:18:32 +0000

When we first look at a new thing, we are bound for that first impression. This first impression is sometimes the barrier, stopping us from starting a conversation with someone, choosing something to eat from the counter and of course buying a product. “No second chance for first impression” is a well known saying. For a SaaS business it might be a major differentiator between success and failure.

Most of our potential customers are anonymous for us, and their purchase decision process is often the same as with all SaaS online sellers:

The customer needs a service/tool.
Searching the web (usually Googling for key words matching the services they need).
Evaluating several of the found alternatives, to see which one best fits their needs.
A purchase decision for one of the options evaluated is then made.

This is of course the short version of the process, which can be a longer, more complicated one, in some organizations (involving people from different departments, internal demos, questions sent to the vendor, quote requests and other).

I will not discuss the process of how to be one of these user-found-me-on-the-web alternatives. The online activities of advertising, the SEO, social media, and website quality are all aimed to bring your product/service/tool to the right place to be included in the evaluation process of potential customers. I personally don’t believe anyone has found the Holy Grail of how to do it right (yet?). I will also not discuss here what makes a good first impression, or what and how to make your tool look better compared to the alternatives.

The purpose of the post is to list several techniques of how to get a second chance for first impression… that’s right. It’s possible.

What we want to do is learn from first time users, how do they see the application on that first impression, what are their thoughts and what can they say after the first few minutes/then hours of using it. Improving this first impression experience will give us a second chance… with the next new potential customers, on their first impression of our service.

“Real” User simulation

Asking a friend to try your product, or paying a company to get you a focus-group, have both the same meaning of bringing new users and learning from their reaction, real-time actions while working with the product for the first time, their questions while doing so and their direct feedback.
You can learn by watching them, asking them questions, and even use professional tools like “eye movement tracking”. If you are under budget restrictions (aren’t we all..), you can do it all online, and send a survey to users, hoping that results will be meaningful. The downside with all of this effort, and this type of users, is that they are not your actual real users, and much of their input might be irrelevant to you or to your real potential user/customer.

Asking for feedback from users

Asking our current users how to improve the process, one they already saw, is another option. These are users who voted already for you, they are usually happy to help and give feedback, and they already know a bit about your app. The problem here is that these users are not exactly relevant to your target audience for that first impression we are looking to improve. You are looking for that fresh new look at your product, that first-time when feelings and impression play the main part, not the logic, expertise or knowledge. Your current users/customers are the experienced users. More than that, they probably have, at this stage, a positive view about your product (after all they became customers). But you are looking for those with either no set mind or even those with a negative position. Another disadvantage of this effort is the survey content, the questions asked. Our experience shows that users will always give you ideas and feedback according to your questions. They will rarely bring up a new approach to the table even if asked for it, and it will be difficult for them to go back to their initial impression and get the data you need from them, about what worked back then, that made them stay and continue the evaluation.

High priority for “how-to” support emails

Another place to get good, solid information is an easy one to implement. Support emails are full with information and knowledge, businesses will usually pay to have. Why not use it if it’s already there?
Whenever a user sends a question or request, it should raise several follow-up questions by you:

Why do they need our assistance, maybe our product is not clear/intuitive enough?
What should we improve? Is there anything missing?
And for me, the most important thing, is asking the user what were they trying to do and why.

In many cases the user might ask a simple “how-to” question. You can answer the question, explaining how to do it and ‘close’ the case. That’s wrong. Consider asking them what were they trying to do, and more important – why. What is their process, work-flow, and how do they use your application for that matter. You will find that you can improve many areas in your solution, change how things work and make it useful for the (real) users.
While this method seems not strictly related to our first impression discussion, it can be so. By continuously improving you product you will have a better product and obviously a more user friendly one, making a better impression even in the first look.

Using freelance testers

A recent practice I learned from using freelance testers (using online resources such as oDesk and others) is that they are a great source for that first-look-impression feedback. You can hire many testers for a relatively low budget, and ask them to use (test) any part of your product. Since these are experienced testers, they will send you great, valuable feedback which will not only help you with the testing but will provide ideas and insights to get better first-time impression. Direct them right, sending the exact expectations, and get that feedback right back.
Although not what we defined earlier as ‘real’ users, they do have the QA skills and will know to tell you about the usability of your product, some possible improvement directions, perhaps a few bugs found on the way :-)

All of the above mentioned options, to get that so-important feedback on how to improve the first time impression are valid, good to use, with their advantages and disadvantages. At Testuff we found that giving high attention to a customer’s email, wisely asking them those “what” and “why” questions, lead us to improvements that most of our users liked. And using freelancers testers gave us that useful fresh look of our product, and led us many to good new enhancements ideas.

2012 – SaaS Test Management Solution Year

yoav — Tue, 20 Dec 2011 14:04:36 +0000

As we come to the end of our most successful year, it is summary days. We are going into our 5th full year, and this is already the fourth summary we write. Time flies.

In each of the previous years, we wrote a short summary about how the year was, and what did we do that year. We talked about our plans for the next year. Since this is holidays season, tradition and all in the air, we’ll continue ours (tradition) as well…

As we’ve mentioned last year, December 31st is no different than January 1st for us same as it is, we’re sure, for you. Work continues, old and new apps to test, existing and new features to try, old and new colleagues to work with. However, in the spirit of the time, new year and all, we sit back for a few minutes, stop the daily routine and think about what happened here and what do we wish would happen….

And so, keeping the same format of last years here it is:

What did we do

We have managed, yet again, 12 new versions, as planned. Not obvious, not easy, but thanks to our developers and testers we have made it. These versions were rich with useful features. Just a few as example:

Share reports
Synchronize Defect Status directly from your tracker
Defect Template
Integration with Clarizen, Youtrack, ChiliProject, activeCollab, Basecamp
Improved integration with all bug trackers: custom fields support and more
New API
Audit Trail
New Reports
Test editor Spell Checker
Template Tags
Server framework upgraded from CherryPy+Sql Alchemy to Django
Added internal bug tracking functionality to Tesuff for those who don’t use one
Added Customization for a few objects

And there are so many more. If you are a user, you know them and have witnessed and enjoyed the monthly progress. If you are not a user…. well, time you became one.

What else? It’s a repeat of 2009, 2010:

2011 was another year of growth for us. Many new customers from many countries, have joined our community.
We worked hard to keep on the good service and smooth operations. From the feedback we get it looks we manged not bad.
Our support team had kept their average response time of less than 3 hours to any incoming email.
We’ve continued to improve our back-end infrastructure, added data centers in several countries and prepared ourselves for another year of growth.

What didn’t we do

We also failed on a few things. There are one or two tricky bugs which we are hunting for some time now, so far with no luck. There’s progress, and we know we will catch them, it is only frustrating while it lasts.

What will we do

We can promise that:

We will continue to be #1 SaaS Test Management Solution.
There will be 12 new monthly significant versions.
Customers will still be the #1 source for what should be included on these versions.
We will introduce new services.

What we won’t do

As always, we won’t be annoying any customer or registered user with too many emails.
We won’t stop enjoying what we do :-)

Happy Holiday Season To All !

Making the Difference

admin — Tue, 06 Dec 2011 14:59:34 +0000

“Happy families are all alike; every unhappy family is unhappy in its own way” Leo Tolstoy

“Happy” Testing Teams

What makes one testing team better than the other? Is it their manager? The methodology they use? The quality of the testers – experience and education included? Or perhaps the combination of all and more?
You would usually want to think that a “Yes” is in place here. But is it really? To an extent it surly is, however if we compare a large number of testing teams, we should – statistically – expect to get the same quality of testers (as testers and as people, whatever that is…), same manager, a few same methodologies used by most teams. You get the point.
On average teams, the bulk of them will fall under the same ‘total score’ for quality. Forget about the exceptions at the top and bottom of the list. Odds are you are not in either :-)

The fact you have the best team players, or the biggest budget, or the best support from above and around you doesn’t necessarily say you will do well. Just see what happened to some sports team who bought, for unreasonable amounts of money, the best players they could get, just to find out a few months later, how big the mistake was.

Don’t fool yourself. We hear those QA managers, each with his ‘answer-to-all-problems’: if they just gave me that automation tool and agreed to implement it, if they just gave me more resources, if I just had a few more days/weeks/months, if only the project manager listened to me…. stop and think. What if this happened? would that make the big difference?

Why would one team perform better than the other? Some groups are found to be more productive, get better results of their testing work, and in a shorter time.

“Unhappy” Testing Teams

Testers have a long list of reasons they give for poor delivery:

The schedule was too short
The budget was too small
The team was inexperienced
The developers didn’t corporate, and communication with them was impossible
The tested application was not ready even for testing

and the list goes on…

You know it :-)

For each project, and each cycle one or two can be picked of the list and used to explain. So, those “happy” groups from before, how do they get the job done under the same conditions? We agreed, as the assumption for this discussion, that the average grade (or quality points) of a typical testing group is the same as the other group. So, how come one is happy and the other isn’t?

Drive, Consistency and Work Ethics

Whichever team you have, whatever methodology you decide to work with, tools you use, budget allocated or scheduled defined – if you set correct 3 things, you are bound the get the best results possible for the situation:

Drive

“Those people blessed with the most talent don’t necessarily outperform everyone else. It’s the people with follow-through who excel.” Mary Kay Ash

Make sure the team is working as a group, understanding the end goals, knowing how to get there. Make sure they have the Drive otherwise they will just an ‘OK’ job, not a good or excellent job. If your testers don’t love testing, replace them.
Give them cool tools to use, appreciate what they do, make sure they are part of the project and not just a side-kick of it.

Consistency

“In any team sport, the best teams have consistency and chemistry” Roger Staubach

Choose how you want to work and go with it. Don’t run after every passing new fashion, with lots of buzz-words explaining them. We think that common-sense is your best friend, but any methodology you choose, stick with it. Same goes for much of how you run your projects. This doesn’t say you can’t improve, change, or be creative. There’s no contradiction between the two.

Work Ethics

“Striving for success without hard work is like trying to harvest where you haven’t planted.” David Bly

You can have the best plan, a good team, you chose your route and you know what and how to do to get to the results and goals you set yourself and your team. If you and your team don’t work hard to get there, day in and day out it doesn’t worth anything. Your team must come to work, not just get to the office…

We’ll discuss in one of our future posts, a totaly different angle of happy testing groups, and how to make them happy.

Those little things that make our work easier

admin — Thu, 10 Nov 2011 20:49:08 +0000

Every application we use has those little things, that make the user’s work so much easier. It can be a keyboard shortcut, a hidden option, or a quick way to make the most out of a feature.
We once had a post written, by a former team member of our support, explaining how to change the background color on a PowerPoint presentation in a specific scenario where it wasn’t obvious – the feedback was amazing. Emails, comments and loads of website visitors who were looking for this over the net and got to this post…..
Those little things.

For over a year now we have been sending Testuff usage tips on our newsletter. Two of them every month.
We’ve decided to sum them up all here, after getting questions from users asking “where can I get the full tips list?”. Truth is that we don’t (or didn’t) have such a list, as the tips were written each month, following our own ideas or our support team mentioning questions that showed a few less known Testuff options and features.
Usually users easily find the various options and features Testuf offers, as these are all visible on-usage and intuitive in their location in the app. Still, some help is always a good idea :-)

So, here it is. Let us know if you find it helpful or if you have a few tips of your own for us to add. We may add the list to our Online-Guide soon.

Testuff Tips:

Copy & paste does miracles in Testuff. Use it for a test or a suite, or even a group of them (ctrl and shift work). It works between branches and projects as well.
Soft-Link is a great tool for better structuring and organizing of your testing.
Bug tracker setup allows you to connect each Testuff project to a matching tracker project.
A test in the lab will be automatically re-assigned to the tester who runs it, regardless of any other tester’s list of tests. You can use it, for example, when not sure about assignments when creating the lab.
Watch for the icon in Testuff. Click on it to get more information about a test.
When reporting a defect, you can drag & drop files from a folder to the entry fields of the bug reporter file attachment section.
Click on a column’s title to sort the table by this column.
A test can be edited while running it. Use the “edit test” link on top of the runner.
To re-organize your suites, you can Drag & Drop a suite into other suites.
To pass more than one test at the same time, mark a group of tests in the Lab, then right-click them and select “Mark as passed”.
Ask your team member to run a test directly from the Tests view by a simple right-click–>assign option.
Use the “Forgot your password?” link on the login screen, to create a new personal password if yours is lost.
Click the bulb icon on the top left side of the Test Runner to get the history information of the test you are running.
In Testuff home screen, bottom right side, you can see the version you are working with. It is highly recommended to always work with the latest version.
Follow us on Twitter. Sometimes we tweet interesting stuff there…
Twistuff can be great group tool, for messages and mutual follow up. It can be a manager let-me-know system as well.
Testuff is built to be intuitive for you to use. This means you should try to do all those things you got used to, such as highlight multiple items with shift or ctrl, use copy & paste for tests/suites, use the mouse right click to discover the many options on each screen, and on each item.
Subscribe to our data centers status page. it can come up handy one day.
Double-click a search result item to go directly to it.
Select your spellchecker preferred dictionary. Go to Settings–>Languages. Can’t find what you need there? Let us know and we’ll add it.
Test Runner window is “always on top” by default, however you can control this, in the Settings–>Look and Feel tab.
Table columns can be customized – sort the table with a click on a column’s title, adjust the width and use hide/show option to select the columns to display.
Use Export test/s to Excel to quickly edit them, and import back using overwrite import option.
Use Template Tags for defect custom field value reporting.
Keyboard Shortcuts work in the test editor. Use them to speed up your work.
When assigning tests in the lab, select more than one Run-Configuration to create a run for each with one click.
To change your monthly subscriptions (licenses) number go to your Settings window in Testuff and click on the Change link next to the number of licenses.

Is Automation Overrated?

yoav — Tue, 11 Oct 2011 17:06:56 +0000

No doubt that automation testing is sexy and impressive – it seems to be a great cure for the annoying process of more laborious testing. Imagine that you can write the crappiest, laziest piece of software and a magic automation tool painlessly finds any and all defects and helps you fix them.

Sadly, this is not possible. But you probably already knew that. Nothing compares to a good human tester with extensive experience, advanced skills, and basic intuition. However, the human approach is not without its drawbacks either. Human testers are more expensive, and that’s only if you can actually find and train them. Keeping them working at the same company for a long time is even harder still.

But this post is not about the tester – it is about automation testing, when to use it, and when to avoid it. After all, automation testing is not always a bad thing. You don’t have to pay for training, and you never run the risk of sick days or the possibility that your automation tool will look for greener pastures. I would just caution anyone who embraces automation as a silver bullet – a panacea to fix all ills and completely replace actual human testers.

However let’s cover when automation is a true benefit. I can think of two distinct scenarios – “Not GUI” and “Boring Parameters.”

Not GUI
This is the classic area in which automation thrives. Developing server side components, protocols, and devices doesn’t require an actual person to send raw commands to the device and test for functionality. Of course, developers naturally do this during the development stage, but to require a physical tester on-site whose sole purpose is to type “sudo blah blah blah” is a phenomenal waste of time if you have an automation tool that can perform the same duties. Automation testers never get bored, rarely make mistakes, and can share their results instantly.

Boring Parameters
This scenario applies to testing in which algorithms generate different results based on different parameters. A good example is the complex algorithm used to calculate your monthly cellular bill (minus all the sneaky fees that invariably require a “human” element). This type of testing can involve more parameters than NASA uses in spaceship design (270,000 according to Bruce Willis in Armageddon). In these cases, automation tools are better equipped to handle the vast number of boring and monotonous inputs than a typical human tester can comfortably manage.

The approach that many automation tools use is something I call Sanity GUI. Such tools provide you with a great framework for recording your application GUI, allowing you to run it and generate scripts that the tool can replay ad infinitum.
The rationale behind this approach is simple – you can do a “real” run in your application, and the tool will record it and allow you to replay this run at any time in the future to ensure that everything still works properly. Moreover, the automation tool will never tire or complain, managers cannot “fault” you for not doing the tests, and the tool itself is actually quite enjoyable to use.

I have no argument against this general rational. On the surface it makes perfect sense. However, in my own testing experiences across a broad range of applications and environments, I have discovered that “Sanity GUI” brings certain disadvantages. In effect, when using automation tools in the above example, I often end up spending more time and money. In nearly all cases, recording is a breeze, script repeating works like magic, result comparison is a snap, and my managers love the neatly generated reports. And on top of all this, the technology is getting better and better, bringing improvements on nearly all fronts. So what is the problem exactly – what is there not to like?

Simple. The problems typically start on Day 2.

After recording the scripts, it becomes increasingly cumbersome (i.e. expensive and time-consuming) to keep them updated. Each time the GUI is modified, the scripts routinely fail. Moreover, there is usually no direct benefit from running the script through the automation tool since one typically tests the application from the GUI. Very rare is the automation script that will discover something that I hadn’t already discovered on my own through normal application testing.

But for me, the worst thing about automation testing is the de-emphasis it places on living, breathing testers. Not only do I waste time using a tool that finds defects I already know about, but automated tools also find “false” defects due to the constantly updated GUI. Every minute spent sorting through obvious defects or false positives is a minute taken away from real testers who run analyses using the actual application to find “real” defects.

When all is said and done, hours of automated testing may need to be followed up with “hands on” human testing, which begs the question – “why automate in these specific cases at all?” It’s like paying people to clean your yard with the full knowledge that you’ll have to go over their work when they’re done and essentially start from scratch as if they hadn’t done any work at all. This is both a waste of money and time.

To Automate or not to Automate?
As I mentioned before, automation is not a bad thing – in most of the industrialized world, automation is a buzzword that connotes efficiency, reduced costs, and speed. But when deciding whether or not to use automation, we should remember that “automation” itself is not a goal. Rather, it is the efficiency, speed, or reduced cost that automation can potentially bring when designing or producing a superior product. In other words, the product itself is the ultimate aim, and one should decide what role automation can play in moving this aim closer within reach.

In our industry, testing (and not automation) is the ultimate goal. In times when automation can yield superior tests with lower costs, better results, and fewer backside problems, then such tools are clearly beneficial. However, if using automation tools results in more clean up, more human testers (after the fact), and an inferior or costlier product, I caution you to exercise good judgment. Real testers might represent higher upfront costs, but if the end result is a product that makes users happier (and your company more profitable), then clearly automation does not represent the best course of action.

Test management and bug tracker – integrated VS integration

yoav — Tue, 20 Sep 2011 20:37:34 +0000

Within the software testing world, there exist a range of different tools, such as automation tools, bug tracking, test management and others. While it is possible to select each tool ‘a la carte’, some of the major releases come as all-or-nothing application lifecycle management (ALM) suites that combine all the potential features into one convenient package.

There are many advantages to the ALM approach. When delivering a complete product, ALM suites could have been be ideal, theoretically, since they combine project management, software development, and test management into one system, allowing for better integration between the different parts. Administration also tends to be easier, as are information sharing and overall project development. However, there are also drawbacks to the all-or-nothing approach. Imagine carrying a Swiss Army Knife with you at all times when all you ever really need is the toothpick. Many users don’t need all the extra bells and whistles, and they remain reluctant to pay for them. Such items not only carry higher costs at the time of purchase, but they can also consume more hours in the form of training, installing, and troubleshooting.

Whether to opt for the “total package” solution or the “a la carte” solution is ultimately up to the user. While we do offer integration with a range of automation tools, our focus is on test management, bug tracking, and how best to combine the two. We adopt this focus for a number of reasons, chief of which is the feedback we’ve regularly received from our target market – software testers who don’t typically need the added features that come with standard ALM suites.

Having settled on test management and bug tracking, there still exist a range of options when selecting tools – should you buy prepackage solutions (and limit expandability) or should you go a la carte (and face potential interoperability issues)?

Again, there are no easy answers to this since each user’s needs are slightly different. Our approach is to offer a pre-packaged solution that works seamlessly with whatever components a user may want to add further down the line. Even still, we initially faced a chicken-and-egg dilemma when deciding in which area to begin our focus – the bug tracking or the test management.

Ultimately we chose to concentrate on bug tracking first, building our test management suite around it from the ground up. The rationale behind this was simple – we discovered that for the average company, adding “test management” solutions was typically Stage 2 of a product’s evolution. While designing and testing new software, developers usually add the bug tracker component prior to handing the package over to those in charge of test management.

This of course makes perfect sense. To accurately test a product, you must have solid infrastructure for finding, tracking, and duplicating errors. Companies that fail to understand this usually find out the hard way that optimal testing is next to impossible if they don’t have the proper support tools and methodology in place before running their first trials. Because developers come first sequentially in a product’s release, we designed our own suite to reflect this tradition.

Another danger that many companies face is having great support tools (i.e. bug tracking) that don’t mesh well with test management tools further along the chain. Interoperability can be a major issue, forcing some testers to abandon and replace many of the tools originally recommended by developers. This can create redundancies (at best) and non-functioning components (at worst).

In serving the ultimate end-user, neither result is ideal, and so we’ve designed our own suite to be as flexible as possible by keeping integration as simple as possible. We’ve accomplished this by limiting our own internal bugging functionality and ensuring optimal integration with as many bug trackers as our technology allows. Currently, our suite integrates with as many as 20 different commercially available bug trackers, with more slated for future releases. Our attitude regarding this is simple – there are countless bug trackers out there, each serving very different needs. Rather than compete with all of them at once, we’ve opted to work with each one individually instead. In this way, users never have to abandon whatever bug tracking preferences they’ve already made prior to using our products.

That being said, we have noticed a trend in recent years as clients increasingly request more comprehensive solutions to the limited bug tracking that currently comes preinstalled in our test management suites. In effect, they want the convenience that prepackaged, all-in-one ALM solutions offer and the flexibility and variability that a la carte solutions typically bring.

This is a unique challenge since the two approaches have historically been mutually exclusive. However, as technology continues to evolve, we constantly discover new ways of fusing the benefits of both the menu approach and the a la carte mentality. While keeping integration and expandability central to our core design, we plan to offer more functional internal bug tracking pre-installed with our standalone products.

Our plan to expand internal bug tracking while enhancing the integration we already support is in line with our general philosophy regarding comprehensive test management. Even with monthly releases and regular feedback from customers, it is not possible for any test management company to perpetually remain ahead of the curve – user needs change faster than our own developers can constantly anticipate. Thus, we plan to weave both flexibility and expandability into our products to ensure that customers never need abandon their current environments or protocols as their requirements change over time. As with all Testuff products, our goal is to fit around our users’ current and future needs rather than forcing users to fit around ours.

New Website Launched

admin — Mon, 19 Sep 2011 13:09:11 +0000

For us it is an exciting day.
Our new website is ‘live’, after we’ve completed the long and hard work on it. We are proud of it already …
We hope that our site visitors will enjoy the new look, new improved navigation, and will find it easier to learn all about our products. If you ever worked to launch a new website for yourself, or your company, then you must know how much work there’s on the details, how much attention it needs and how important it is.
Well, for us this was the case.
.

.
In our design process, we were conscious about our two types of website visitors – new users, trying to learn about Testuff, making the initial decision to give it a try, and then our customers, getting back to the site for more information, mainly visiting the online-guide.

The redesigned site reflects our desire to build a good relationship with a new audience right from the starting point. The site designers kept it fun looking and simple with a user friendly approach. Exactly as Testuff test management is.

Send us your feedback. Good or bad. If you’re a customer, or had experience with how we work, then you should know we take this feedback seriously, working to improve by it. We are also conducting A/B testing, to see what works best for you and for us.

We thank all of those who worked to make it happen

Virtual Conference and Virtual Expo on Software Testing

admin — Thu, 08 Sep 2011 15:09:48 +0000

EuroStar 2011 is a major software testing conference (Manchester, England November 21-24). We were considering going there, but you know how it is – the travel, the time, the expense….
Then we got a chance to participate, even present Testuff, without going anywhere. Exactly as we like it – online.
.
Staying Virtual
Online is something we like. As a SaaS vendor we live on the net, our customers come from there, and we service them using it. So, if we get a chance for a virtual conference, how can we resist it?
Apparently, EuroStar guys realized that not everyone can come to the actual conference. They came up with a one-day virtual conference (September 13th) in which visitors can review virtual vendors booths, watch videos, chat online and attend lectures on software testing. All online, from the comfort of their desk wherever they are in the world.
.
We’re there too. We have a booth, and we’ll wait for you and your colleagues there.
.
Special Conference Discount
Our test management solution customers who will be visiting EuroStar 2011 real event, can enjoy a special 10% registration discount from us.
Our customer? Contact us, and we’ll send you the code.
.
Not a customer yet? It’s time to be one :-)

Who Cares What James Bach Thinks?

admin — Wed, 31 Aug 2011 13:37:05 +0000

When we first launched Testuff over 4 years ago, it certainly wasn’t with James Bach in mind. Although Mr. Bach had already established a substantial following thanks to his many contributions to the world of software testing, our initial foray into the industry happened quite independently of him. Our founding, our growth, our entire vision was never directly influenced by his input or feedback. And yet, anyone familiar with our suite of testing software would likely be convinced that Mr. Bach had played a direct (or at least inspirational) role in our development over these last years.
.
Vendors I like
For those of you who are not familiar with Mr. Bach’s work, he’s somewhat of a guru within the testing world, having penned many notable articles and books over the years (not to mention contests, guest spots, seminars, and the like). While no single article can fully capture his life’s work, a recent post of his on what he likes and doesn’t like about test tool vendors provides a unique glimpse into his philosophy regarding what you the client should expect of a software testing company. Granted, you likely have your own list of criteria, developed based on your current needs and wants. But Mr. Bach’s brief list offers a solid baseline – a foundation that coincidentally reflects our own attitude regarding software testing.

In an August 2011 post entitled, Avoiding My Curse On Tool Vendors, Mr. Bach outlines those attributes he expects in successful testing designers, including:

Testing for an audience…. based on feedback from clients
Power through products…. based on flexibility and expandability
Genuine love of testing….. I mean to really live and breathe the product

Now, if you ask most companies how closely they subscribe to the above, they’ll probably tell you that they’re 100% on target.

Admittedly, we’re no different in this regard.
.
What we say
We purposely design for our audience, actively soliciting feedback from our clients to ensure that the finished product does exactly what they want, when they want it. We design all testing platforms from the ground up to provide optimal flexibility and unmatched customization. And we love the work that we do – I mean, really really love it. Testing is an art, science, and passion for us, and we enjoy the challenges of overcoming seemingly insurmountable obstacles on a daily basis. We’re a little nerdy when it comes to testing, and we’re quite unapologetic.

But like I said, most any company that you ask will tell you likewise. This is why we prefer to rely on testimonials – feedback collected from living, breathing clients who have actually used our software firsthand.

We’ve compiled a number of testimonials over the years, and the comments have been extremely favorable. This constant feedback not only lets us know that we continue to live up to the high standards we’ve set for ourselves, but they also reinforce our belief that we’ve been on the right track all along, if Mr. Bach’s August 2011 piece is any indication of what software testing companies should aspire to become.
.
What they say
Below are three recent testimonials that highlight Testuff journey to date. Read them for yourself to understand our approach to software testing and how we serve the needs of our growing community. Thereafter, we invite you to download a trial version of Testuff for your company to see if we live up to your expectations:
.
.
Testuff has quickly become my favorite test management tool. In my 13 years of QA experience Testuff is a unique gem. No other tool that I have used previously has allowed my team to create and organize test cases so quickly. We are literally saving days on each project since adopting Testuff. The layout is extremely intuitive, and I love how quickly I can track test status for each project, each tester and Automation.

But I think you will find that the most impressive thing about this tool is the company. From your first interaction with support to requesting a new feature, you get the impression that this company listens carefully and responds to its user base.
Robert Shaw, Frontline Placement Technologies
.
Testuff looks to be a Godsend in its ease of use and incredibly handy features. It took me absolutely no time to start putting together test suites and assigning tasks. That’s amazing! Many of the tools I’ve seen out there are just obtuse as Hell, and people just accept that fact because they need the functionality, and they can list that software under their “skills” on their resume. Testuff has the functionality and it’s very straight forward.
William Parker, Kaplan Professional Education
.
I’d been scrabbling around for years looking for a simple yet powerful way to manage my checks and tests. Spreadsheets here, Open Source product there, free product here and text files all over the place. I was beginning to wonder whether I would find a tool that worked for me. Then I tried Testuff and was hooked. It’s very powerful, yet intuitive and simple to use. It fits around my process rather than me having to change my process to fit the tool.

Believe me, I’ve changed my process several times and Testuff has always been a trusty companion. The video recording, screen shots and editing tests on the fly features really seal the deal as it allows me to organize and manage my exploratory testing in one place, all complete with reports and recorded sessions. To top it all off the support is outstanding; fast, friendly and incredibly committed to the product. Great Stuff.
Rob Lambert, New Voice Media

Feature Request Story

admin — Tue, 26 Jul 2011 13:08:14 +0000

In a perfect world, we (the software providers) should have enough time to develop and complete the perfect product – one that it is intuitive, easy to use, and full of the most in-demand features – before selling it at a reasonable price. I am reminded of a quote from Armageddon when one of the characters offers, “You know we’re sitting on 4 million pounds of fuel, one nuclear weapon, and a thing that has 270,000 moving parts built by the lowest bidder. Makes you feel good, doesn’t it?”

Thankfully, software is not a nuclear weapon and nothing catastrophic will happen if it has some defects. However, the underlying principle remains the same – when you have too many features without enough time to fully test or develop them, quality defects are inevitably bound to emerge as you rush your product to market.

The purpose of this post is to outline the process of bringing a feature request from idea to implementation. The catalyst behind this post is a feature request (codenamed #32) that has followed us since May of 2007.

Where Feature Requests Come from
Believe it or not, we do have a roadmap! This roadmap guides us in our decision-making as we scan the horizon and envision how our clients might use whatever features or products we provide. In this vision, we blend a mix of different directives, including service add-ons, feature enhancements, infrastructure updates, marketing activities, and other crucial elements we know will help us design a superior end product. From all these items, we develop an internal list of features that we call, not-surprisingly, “our features,” but we also actively solicit feedback from our clients to ensure that our long-term vision meshes with the reality of those who ultimately rely on our products. In fact, it is through this external focus that we derive the bulk of our improvement ideas.

We follow two simple rules when incorporating feedback requests from clients:

Does the enhancement make sense?
Will the enhancement improve the GUI?

In truth, these two simple “rules” are actually not that simple. There exist some product enhancements that could be of tremendous benefit to a small subset of users while harming others. It is not easy to reject a perfectly reasonable request, but we find ourselves in this very position sometimes when we realize that to successfully implement a particular enhancement could render the GUI less intuitive or more difficult to navigate. Because of our orthodox approach to interface design, we strive to keep things simple, avoiding the endless series of options and laundry lists that plague many other products (tools -> options -> etc).

Our uncompromising stance on potentially detrimental improvements means that we occasionally lose clients, but we are willing to pay this price. On one occasion, clients requested local installation capabilities for our product, but we regrettably had to decline even though such a feature is mandatory for many users.

However, even “rejected” ideas never truly die. By communicating with our clients directly and trying to understand their needs and processes, we often discover feature enhancement opportunities that neither party would ever realize. Having an open dialogue is crucial for continual improvement within our industry.

Documenting and Managing Feature Requests
We currently use Trac to manage our features. After one of our team members sets the ticket attributes, the rest of us can modify the values, making them more accurate over time. We then use these updated attributes to help us prioritize the request:

Description – including how and where the feature fits in with the product.
Customer email – allowing for easier feedback and follow-up.
Size – development time, including time to implementation. In a later post, I may explain our “size” unit in greater detail, but in a nutshell, size is a critical component in productivity.
Testability – essentially a simple guide that explains how testers can validate the feature’s usefulness.
Milestone – usually set to TBD, but we occasionally assign specific goals based on individual client requests.

Monthly Plan – Monthly Releases
We deliver monthly releases of our product, with most updates scheduled two months in advance. Our approach is quite simple – first we add those features that we promised to our customers, then we add secondary features from our roadmap, and lastly we incorporate any TBD features until the estimated “size” of the proposed update approaches the limits of what our developers can handle.

This process allows us to respond to urgent requests very quickly – usually within 30 days. And because we have built a smart auto-update process, we can release individual enhancements and features directly to specific clients based on their current needs.

Feature #32 Case Study
At Testuff we have recorded, until June 2011, over 30 releases counting over 730 new features requests. Feature #32 is one of the first to be suggested, a few years ago.

It was a fairly straightforward enhancement that involved adding spellchecking capabilities to our editor. This request emerged internally during our planning board as we mapped out what we believed our customers would eventually request further down the road. However, we were surprised to discover that almost none of our clients needed this feature when we first began developing it. Even still, lack of interest was not the reason behind the feature’s delayed release.

Rather, we were initially unable to find a workable way to implement spellchecking in python wx.richctrl with our editor. Delay after delay pushed the timeline back, until we eventually realized that, absent any specific requests from an individual client (or group of clients), it made more sense to pause development of this feature temporarily.

More recently, however, we did begin to receive requests from our customers. Although we faced the same technical challenges as before, by pulling in additional outside help, we developed a viable workaround that allowed us to properly implement a spellchecker.
How we got this additional help is an interesting story by itself… we decided to give oDesk (and its likes) a chance and posted request for a similar job. The number of good ideas we got this way was surprising. This led us to start using this outsourcing channel, and so far, we are happy with the results.

Our successful release of Feature #32, in the last version, essentially highlights our approach to software development and testing platforms. We try to anticipate the needs of our clients while simultaneously soliciting feedback on additional improvements. In this way, we continue to deliver high quality products and services that address the individual needs of our growing user base.