Break to make it better

All's Well That Starts Well

noreply@blogger.com (Anand Tambey) — Mon, 06 Jul 2015 10:44:00 +0000

"All's Well That Ends Well"

Once upon a time, a famous writer Shakespeare had coined this phrase and a play with the same name. The full meaning of above quote continues around the play, with a happy ending too.

All's Well That Ends Well. If it is not well, then it is not the end.

Thank you Mr. Shakespeare for giving hope and optimism to all broken hearts. Still, this quote and feelings touch every heart across the world.

With due respect to him, I would like to coin and unearth a definition, which is unseen, not thought before, wistful and yet forms a new quote.

"All's Well That Starts Well"

(Then it ends well too, how? You would see in the next posts)

Dear readers, please stay with me for the next chain of posts, to get more of it and to get tips for your success in Testing & Test Automation Process.

PS – This original thought and a framework for optimized automation was presented at a conference by me in 2008. © All rights reserved.

Threat – T Factor in Estimation

noreply@blogger.com (Anand Tambey) — Sat, 31 Mar 2012 18:09:00 +0000

From Wikipedia while taking our car driving Example:-

A threat is a possible danger that might exploit some vulnerability/flaw and thus cause possible harm or loss. A threat can be either "intentional" (i.e., intelligent; e.g., an individual criminal stopping car for robbery or crowd piled up for some road show) or "accidental" (e.g., the possibility of a car part malfunctioning, or the possibility of an "act of God" such as an earthquake, a snowstorm, or a tornado) or otherwise a circumstance, capability, action, or event.

Risk is the potential that a given threat will exploit vulnerabilities of a system and thereby causes harm or loss. It is measured in terms of a combination of the probability of an event and its consequence i.e. impact.

Risk = Probability (of threat exploiting a vulnerability) * Impact (the extent of loss or harm caused)

Before even starting from home, you may need to think of all possible Threats/Vulnerabilities/flaws beforehand and covering it up like a prompt check-up of essential working condition of car/yourself, getting information/ preparing about any warning signs in traffic/weather/route from TV News/Web and so on. Finally allowing a buffer (Contingency) of extra time to accommodated uncertain events, changes and actions.

Coming towards estimation process, assumptions (some of them may convert to risks) and risk-mitigation plan forms an important part of an estimation model. As said earlier we could try as much as to put it in countable/objective manner like average risk score of all possible risks. This would help us to determine our action plans (mitigations) as well as time allocated for buffer or contingency, if risk materializes.

Do you agree?

Click Here to View All Related Posts About Estimation, in Descending Order.

Track – T Factor in Estimation

noreply@blogger.com (Anand Tambey) — Sat, 31 Mar 2012 18:09:00 +0000

As a saying goes, what you can measure you can manage, what you can track you can manage.

As we seen in drill/rehearsal phase of activities of smaller set of work, tracking and effort monitoring is vital, to reach at accurate estimates for whole work.

Also estimation results in an initial schedule of activities and as a good practice schedule must be tracked (Estimated Vs Actual Efforts), delays must be communicated and estimates/schedule needs to be revised religiously. As I’d talked about ease-of-use and ease-of-understanding earlier, estimation model should be flexible and easy to revise in less time.

Getting back to our car driving Example, you might decide few milestones like roundabout 1, turn 2 or any landmark, where you can track and calculate actual time taken Vs time estimated, to know whether you can reach on time. There might be risks, threats and delays, which needs to be communicated early/immediately to your client (wife), with re-estimated time to reach your destination.

So all and all, without efficient tracking, your estimation and plan may be just worth of file/paper, on which it is written.

The tracking of efforts and deviation recorded will help the current project as well as constitute the learning’s for other projects and accuracy of future estimates.

Do you agree?

Click Here to View All Related Posts About Estimation, in Descending Order.

Tweak – T Factor in Estimation

noreply@blogger.com (Anand Tambey) — Sat, 31 Mar 2012 18:08:00 +0000

As per Wikipedia, Tweaking refers to fine-tuning or adjusting a complex system. Tweaks are any small modifications or adjustments intended to improve system.

Coming back to the estimation, this is a forecasting game of three important factors and essentially made of Numbers, Numbers and Numbers.

Now tell me, whether all measurements could be quantitative or countable or objective? No, qualitative or perception based or subjective measures will always be there to complete the whole picture.

For an example, there is a score, calculated based on questionnaire response, to identify probability of autism disease disorders passing towards next generations. Most importantly the country of living may also matter, as in UK, typically 3 out of 100 kids born, diagnosed with Autism, for other countries it may be less and vary. You could see this as an adjustment factor after calculating overall score.

It is quite difficult or impossible to depict estimation in fully objective manner i.e. deriving from just counts; you may factorize the complexity in numbers for simple, medium and complex. (For example a test case with X steps, Y steps or more than Z steps, is simple, medium or complex respectively).However those numbers would be only applicable to your context/project only and purely subjective measure.

If you observe, even Function Point Estimation method involves an Adjustment Factor (AF) to be multiplied, subjective to system under consideration. Yes, it is necessary to have such tweaking factor, subject to individual systems/context. However it was always better to have more objective measures or countable factors e.g. files, transactions and input/output fields as in function point method. This will make estimation easy-to-use and judgment-free. You would agree that if more than 2 people are counting unadjusted function points for similar system, would be quite similar as counting method is objective and independent of personal judgment/bias. However the overall score needs to be multiplied by adjustment factor or to be tweaked.

In brief, for an estimation model to be effective it should be judgment/bias free, as much as possible. In other words, initial estimation (without adjustment/tweak) numbers should be same even if calculated by 2 or more estimators. However as we cannot remove subjectivity fully, hence need to define tweaking factor(s) for example clarity of requirements on the scale of 1 to 10, risk/contingencies factor and likewise.

Getting back to our Example of car driving, you can add a tweaking factor like 1.1 to 1.3 multiplied by normal time taken, based on risks like traffic, weather and any unexpected events for worst case estimate. If you met best case or most likely time, it is good and otherwise you had actually kept a buffer for uncertain events too.

Believe me, the old saying still holds true – Under commit and over deliver!!!

Do you agree?

Click Here to View All Related Posts About Estimation, in Descending Order.

Tactics – T Factor in Estimation

noreply@blogger.com (Anand Tambey) — Sat, 31 Mar 2012 18:08:00 +0000

When you hear the word “Tactics”, what comes first to your mind – Military Tactics? Yes, your project is nevertheless a war on multiple fronts, you bet!!!

Not that all military tactics would be useful for any IT/Testing project, but this one, which relates very well to estimation.

Battle Drill/Rehearsals - Before getting my point, you might be grinning wide, imagining yourself in body armor with holding pistols/rifles or even holding a sword. Smile, yeah, you need to prepare very well before you provide confident estimation, which is nevertheless a battle to be won before actual war.

As I’d mentioned in Ah Estimation - So Anti Time!!!, without much information in hand, the estimation should be presented with low confidence rating. However information only available with you is sometimes in documented form or the static information. So how could we instill the confidence in our estimations? Is it possible only with static data points? The answer is No.

Work Drill/Rehearsals - Here the drills and rehearsals come into picture, which is essentially doing the actual work with small set of scoped work and recording/tracking/monitoring the actual efforts closely. Now that you have this dynamic data points for a “clever” byte of work, you may extrapolate the factors & results to get estimation for full work.

Clever Byte Of Work - Remember the byte of work chosen for drill/rehearsal must be minimal work yet clever i.e. practically it should cover possible variations/ranges as well as end to end flow of activities.

Remembering our car driving Example , you might take 1 sample trip towards the destination, noting down the impacting factors like traffic, number of stops and time taken. You might choose a different route (variation) for going and coming back to gauge the time taken. More samples, variations and experience of route will help in near accurate estimation of travel time and ensuring success i.e. reaching safe and well within time.

It is also necessary; these experiences are recorded by you or in your presence, for best results. That is why people doing actual work e.g. testers/developers, needs to be involved in estimation process and to count on their opinions.

Past Experiences of others – History itself is a Mystery - If you opt for following advice/experience of a friend, it is quite possible he is less/more expert driver and routes/destinations/surrounding circumstances/context was different than yours. So always use experiences of others (history of past projects/industry) with caution. There is a good paper written by Kiran Marri about “Using Historical Data Rightly in Test Estimation”

Do you agree?

Click Here to View All Related Posts About Estimation, in Descending Order.

Talk – T Factor in Estimation

noreply@blogger.com (Anand Tambey) — Fri, 24 Jun 2011 02:07:00 +0000

“Talk it out!!!” you might have heard this many times whenever you’re bothered with issues. Keeping mum, hiding or sitting on it would not help.

Considering our Example, if you had done a mistake at any exit and do not know which direction to take on next round-about. You will just be moving around twice or thrice. You probably understand it is your mistake but you do not want to tell it as you will lose face. Ultimately you will have delays and unwanted routes, just because you did not speak up and not accepted mistake. Accepting the mistake and taking help of someone early, is way ahead better than sitting on it. You will become more credible as this honest acceptance will lead to build the long lasting trust.

Talk it out issues well in time or even ahead of time with your team and client. [Before making it worse] You would be amazed how well support systems would emerge and help the project to recover smoothly, if you speak early. Otherwise everything else would result in fire-fighting mode and become counter-productive.

So your estimation model should accommodate these unexpected issues, turn-around time and way to communicate it. Ensure issues, delays and changes communicated well ahead in time via revised estimates and schedule for quick agreement of all stakeholders.

Do you agree?

Click Here to View All Related Posts About Estimation, in Descending Order.

Trust - The T Factor in Estimation

noreply@blogger.com (Anand Tambey) — Fri, 24 Jun 2011 01:50:00 +0000

Wiktionary:- trust is Confidence in or reliance on some person or quality.

Trust is the important link between two entities to synchronize and achieve same goal together. You’ll agree that any relationship cannot survive without trust.

Considering our Example if your wife does not trust your driving capabilities and you. [Or you may not have complete trust in your wife] Any cause contributing to delay or direction changes would not be acceptable and understood. It will lead to unending conflicts/criticism and finally impact your driving while not reaching the goal successfully.

To make project successful, client team and delivery team should trust each other(Within themselves too internally). Without trust usually client and delivery team would not share much of information to each other. Instead both will keep watch on numbers to criticize/pushback rather than underlying issues to be in attention and resolved quickly. It would lead to typical Theory X of Management and counter-productive to project.

One way we can build the trust is maintaining transparency i.e. shared model of processes, information,honesty, accountability and responsibility. Remember making project a success is everyone’s responsibility.

Right now we will focus on estimation process and so we need to share underlying information about estimates to all stakeholders.

However you will ask what if client is overwhelmed by receiving too much of technical/process information in estimates and he will tend to take interest in final numbers only. Similarly delivery team had to maintain too much of information, besides delivery . It can cause delay in updating/communicating revised estimations/schedule.

This fact points towards most important factors in any estimation model, Easy To Use and Easy To Understand.

This essentially means presenting estimation in such a simple format so that all the stakeholders should understand, where the estimates coming from, in the language of them i.e. what are the factors, if changed in due course, would impact final numbers and schedule would be revised.

Be diligent in including all possible factors and information. Ensure any new factor or information would be easy to accommodate in the model. Frequent changes or revision should just take minutes not days or weeks. All and all model should evolve over the period easily and continuously improve.

Do you agree?

Click Here to View All Related Posts About Estimation, in Descending Order.

Estimation – An Example to Remember

noreply@blogger.com (Anand Tambey) — Fri, 24 Jun 2011 01:33:00 +0000

Before we explore nuisances of estimation and what are the factors important. I would like to introduce a small and simple example from real life which I will use throughout to explain the factors. You can very well relate it to actual project situations too.

Consider you are staying at Washington D.C. and your mother-in-law stays at Ashburn VA. Now you will be driving towards in-laws house with your wife and 2 kids of age 5 years and 1 year. It is compelling situation for you to reach there in 1.5 Hours, though based on directions provided by Google maps it would take 48 minutes to 1 hour in traffic. Of course your client a.k.a. wife will ensure time :-) to be taken seriously.

Driving across towns to reach a specific goal can be modeled using a flowchart like this one.

To reach this relatively long-term goal, you might engage in short-term goals such as planning the route, making specific choices along the way, paying attention to road signs as well as other drivers, allowing for unexpected developments such as bridge repairs or rainstorms or icy surfaces. We go through similar situations in our projects too.

It is a constant process of adjustment; if a stoplight is red, one must stop; if gas is needed, the tank must be re-filled. The ideal situation is when a human is making intelligent adjustments based on reality while keeping the longer-term goal in view.

If a person can drive his automobile past changing circumstances without causing a crash, and get to the destination in a reasonable amount of time, then he or she made the trip successfully.

[Source: - Mentally healthy mind - An exploration of how to flourish as a human being by Thomas Sulcer - Licensed under Attribution 3.0 Unported (CC BY 3.0)]

T-Factor in Estimation

noreply@blogger.com (Anand Tambey) — Thu, 31 Mar 2011 15:23:00 +0000

In continuation of my earlier post - Ah Estimation - So Anti Time !!!. Make sure you read it before exploring next few posts on estimation.

Now I will be touching upon more detailed thoughts around estimation, in general and then will move to specifics towards testing and test automation estimation.

"There is No One Size Fits All" - Same thing goes with estimated size & effort too.

However to do the estimation in slightly better(No best way though) way, T-Factors are really important.

Tact,Talk,Threat,Track,Trust,Tweak - In alphabetical order.We will explore it one-by-one in next few posts.

Ah Estimation - So Anti Time !!!

noreply@blogger.com (Anand Tambey) — Tue, 08 Mar 2011 01:01:00 +0000

Ah Estimation - So Anti Time !!!

When I see and jumble letters in "Estimation", I found it "So Anti Time". It is just opposite of actual time taken. How True...

Are these lines are familiar with you and your project?

I love the deadlines, especially whooping sound they make while passing...
No project has ever finished on time, within budget,to requirement - yours won't be the first to.
A badly planned project will take three times longer than expected - a well planned project only twice as long as expected.
The project would not have been started if the truth had been told about the cost and timescale.
Quantitative project management is for predicting cost and schedule overruns well in advance.
Any project can be estimated correctly (Once it is complete...)

[From 77 Project Management Proverbs www.oliverlehmann.com ]

So you are smiling now on your own dreaded experience with estimation. Don't be afraid it is same experience for all kind of estimation and estimators.Tell me, can you estimate/predict your child's growth and development phases, accurately? It may or may not take same order/sequence for every child, but surely timing of phases would be different for every child. Same thing applies to task-in-hand or project too...So, how one should deal with estimates?

"What time it will take to complete? Are you sure ?"

I (or anyone) feel chills deep down my spine, when someone asks it. Even though I will answer it it in *clever* yet an honest way.

"I will get back to you, once I consult with my team members, analyze the base data available and these queries resolved by your team."

Still management and client may ask a rapid estimate. Hmm, I will have to provide quick-n-dirty numbers based on early assumptions & risks, subject to changes with low confidence rating.

Even after analyzing details, I will give three estimates namely best case, most likely and worst case scenario.

In my opinion, these are the first few steps in traditional approach to estimation. In next few posts I would point out my view on estimating test automation & testing, which might be unconventional.Hope you will be with me and enjoy the journey towards exploring mystery called estimation!!!

Hate To Work, Be Lazy (Terms & Conditions Apply) !!!

noreply@blogger.com (Anand Tambey) — Sun, 27 Feb 2011 12:43:00 +0000

Shocked & frowned!!! Am I out of my head to give such an advice ?
Some people may feel happy inside too, for having someone appreciating their laziness. But sorry, it is not self fulfilling prophecy, read T & C Apply.

Are you happy for what you do in a day at work?
Someone had rightly said - "Doing same work in same way all the time, will produce the same results”.
B.O.R.I.N.G. Right!!!
What if your work is reduced? You can be more relaxed and more open to new & fresh thoughts. Who will do this? Whether someone already invented the way to reduce it OR you can work on an idea to reduce work.

In any case "You must work, to reduce your work".
EITHER searching for already available solutions and Tailor-Fit to your situation OR finally to devise yourself an automated way to work in faster and efficient way.
But this fire and motivation of finding new ways to work will never come to you, if you feel content and satisfied.

You should always review & think:-
What (the hell) I am doing?
I hate doing this part of work?
Could it be done in better way?
What if I don't have to work on it at all?

In a positive and constructive sense, you need to be little lazy and grow little impatience/hate to work in same old ways.Then only you would be able to automate your work and continuously working towards reducing the work.

Hence I always say:-
A Lazy person could be the best automation professional, if he is not lazy in implementing his ideas to reduce his work. ~Anand Tambey

3 W - Vital Tips For Automation Professionals

noreply@blogger.com (Anand Tambey) — Mon, 27 Dec 2010 22:29:00 +0000

An anxious test automation analyst asked a sincere question about his future -
"I'd worked in tool X for last 3.5 years and selected in a company based on this experience.However, in new company tool X(currently working on the old version) would be soon phased out. A very popular and widely used tool Y(new to me) will replace it.Should I specialize in tool X for the career(find a new job) or learn tool Y, where better job prospects are available?"

My answer was : (Elaborated for blog readers)

I keep six honest serving-men

(They taught me all I knew);
Their names are What and Why and When
And How and Where and Who.

~Rudyard Kipling

It is beneficial to learn, apply and master first 3 W's - What/Why/When as below. How/Where/Who comes little later and can be quickly learned if you had mastered 3 W very well.

Just enjoy and learn the concepts, the way to arrive at the right solution for particular situation and challenges by mastering 3 W's. You can master these using any tool/language but be sure to diverse yourself in different platforms/type of applications to hone your troubleshooting skills. You can apply your knowledge & experience in multiple tools/languages even for tool Y. So you are still in the business, just never stop to learn new things, tools, people & perspectives.

In a nutshell - It is essential to know - Todo

- What (Works best for the situation)

- Why (Know The reason for doing it)

- When (Right time and situation for doing it)

- How (Way to do, keep updated and always search for better - Tools, languages etc.)

- Where (Environment, company, value system, encouragement etc. - Work at appropriate one)

- Who (People doing it - You and Others - Always keep yourself surrounded with best people, be humble, provide & ask for help, respect, appreciate and keep learning from them.)

Definitions - Test Automation & Automation Professionals

noreply@blogger.com (Anand Tambey) — Mon, 27 Dec 2010 17:52:00 +0000

20% Automated Test Cases had potential to give 80% of ROI.(From Pareto)

A Lazy person could be the best automation professional, if he is not lazy in implementing his ideas to reduce his work.
~Anand Tambey

Automation is, work to reduce work.

Do not Document,Automate...

~Anand Tambey

Test Automation : Five Deadly Myths and Ground Realities

noreply@blogger.com (Anand Tambey) — Thu, 27 May 2010 16:41:00 +0000

Myth 1 – Automation can be completed just as manual testing with same number or less resources.
Reality –
- In fact, real projects takes 10+ cycles (1.5 to 2 years) of equivalent time and more resources/cost/risks than manual testing.
Myth 2 – Automation project is simple project.
Reality –
- Test Automation needs to be taken by senior management,very seriously as a long term strategic move like a mango tree.
- You would get actual fruits in late, but you need to wait and give support to grow it properly.
- If taken in short project or tactical mode, it may not result in desired benefits.
- There is one exception though, we could get maximum possible benefit even in short-term project by choosing right/vital scope and nurturing very well at initial period.
Myth 3 – Completion of automation suite follows linear path.
Reality –
- Unlike manual testing, % complete could not be expected for duration like 5 day testing, every day it would complete 20%.
- In fact initial period, % complete would be very less i.e. 5-10%, where it is time to get basics right i.e. strong framework foundation and focused development for re-usability and maintainability.
Myth 4 – Test Automation = Manual Test Cases
Reality –
- Not likely, as you need to carry out test steps and verification via programming using industry tools like QTP.
- Generally you need to devise extra programming and/or descriptive programming efforts to automate RIGHT manual test cases.
- Sometimes you need to take different flow/activities than test case.
- For example, test step to select a value from a drop-down list, test automation code will do existence verification, click on control, wait for all items to be populated and finally selecting specific value.
- Some of the test cases are too simple, one-time or too complex to automate vis-à-vis value returned, which should be ideally left out from scope.

Myth 5 – 100% Automation and 100% test coverage is possible via automation.
Reality – (Most common, now very well accepted myth but still included)
- This is not possible and feasible to achieve.
- As input & maintenance cost is too high and returns would be too low, if tried to.

Great Expectations

noreply@blogger.com (Anand Tambey) — Thu, 27 May 2010 16:13:00 +0000

Yes, be it a marital relationship or test automation, it is very common for stakeholders having unrealistic expectations about it...

Here is an attempt to set realistic expectations for stakeholders...testers,application developers,client and senior management...

Automation is not a magic wend, we need to nurture it carefully to achieve desired benefits.
Automation is a development project and does not follow traditional manual test management.
Unlike manual testing, % complete could not be expected for duration like 5 day testing, every day it would complete 20%.
In fact initial period, % complete would be very less i.e. 5-10%, where it is time to get basics right i.e. strong framework foundation and focused development for range, re-usability and maintainability.
You can compare it with hitting bulls-eye directly rather than scoring outside.
During initial framework development, added resources and stringent deadlines may not be useful/productive.
Once framework stabilizes, the % completion can achieve speed with adding more resources, provided initial period was nurtured successfully with taking care of variations/range of AUT(application under test) rather than depth and numbers/percentages of test cases covered.

Do you agree?

Do not document...Automate...

noreply@blogger.com (Anand Tambey) — Mon, 14 Sep 2009 09:54:00 +0000

If you ask where is the fun for testers? 

It is only in breaking the system and finding most critical bugs. Some would say Eureka..Whoa..or..even dance in their seats. Would you agree?

Here is a billion dollar question. Where is it a boring or routine task? 

Most of you may have these things in mind - detailing or documenting test steps before testing and reproducing steps in bug reports. Am I right? 

Though jotting down high-level scenarios, checklists, cheat sheets, negative (destructive) and evolving techniques based combinations (orthogonal) is fun.

Well, if it had hit your chord, please do read exploratory and heuristics based testing by Guru James Bach,  Kem Kaner et al. These techniques use minimal documentation, but provide most of the value of running undocumented tests.

However, I would like to bring one more food for thought, might be more said than done and might be a dream for many of us.(My dream since I'd started my IT career)

Do not document. Automate. As much as possible.

Reiterating. As a hymn, as a ritual.

Do not document.Automate.

Learning : A Key to successful test automation project

noreply@blogger.com (Anand Tambey) — Sun, 02 Aug 2009 05:47:00 +0000

Every day brings new challenges in this niche area of test automation. Be it a problems of recognizing objects, unexpected application behaviour OR an inconsistent failing of automation suite which was running successfully minutes before.

It is vital that test automation approach analyzes it,learn and respond to these challenges in effective manner. Here is one simple theme recommended called "Learning", if followed religiously, can do wonders.

Look Back and Learn Frequently

Retrospective: As per agile best practice it is one of the continuous learning approaches, where we recognize our strength, accept our weakness and sought opportunities/action plan for improvements. Do it frequently by asking these questions, during test automation development and track the action plan religiously.

Q.1 What did we do well, that if we don’t discuss we might forget?
Q.2 What did we learn from our mistakes/experiences? (We believe, can improve)
Q.3 What should we do differently next time?
Q.4 What still puzzles us?

Wish List/ToDo List : You can found wish lists in release notes for typically open source software/products. Yes, this practice could be applied to test automation also, where it provides value immediate as well as long term.Maintain list of frequent unresolved problems[from Q4] and automation suite features you wish to implement. Have a walkthrough with brainstorm on same, while looking back frequently or special meetings. Two benefits :- We can never let problems/features out of sight and as we resolve them we had a good knowledge base to reuse/rely upon.

Do you agree?

Is your automated suite fully automated ?

noreply@blogger.com (Anand Tambey) — Thu, 12 Mar 2009 02:46:00 +0000

Well, this question may seem very unusual, at first glance.

Usually an automation suite is delivered with heavy setup-n-configuration steps, detailed checklist and instructions to run suite. Now keeping this in mind, can you answer my question ? Does it make sense?

This is possible and there is always a way to avoid

Opening script in automation tool, load the libraries and clicking the RUN button
Manual configuration of environment and reference data
Manual checklists

The major benefit of Test automation is to run unattended without any user interventions. The simple point I am making here is nothing needs to be manual in terms of running a suite end-to-end.

Now answer this before your test automation suite go-live :-

Is your automated suite fully automated ?

What is your test automation goal ?

noreply@blogger.com (Anand Tambey) — Mon, 09 Feb 2009 19:05:00 +0000

As we'd built a thought process on having the RIGHT objectives of test automation, what could be those :-

Not to save time (only) , but to find errors. In-time, clear and informed to right set of people.
Not to save efforts (only) , but to attempt test case(s) worthy and effective. Please refer my last 2 posts http://anandtambey.blogspot.com/2009_01_01_archive.html.
Not to attempt 100%, but to attempt essential few. Consider potential returns, criticality and business benefits for selecting those few.
In short, the RIGHT testing is the foundation of RIGHT test automation.

When automation is not the right choice ?

noreply@blogger.com (Anand Tambey) — Fri, 23 Jan 2009 18:57:00 +0000

Automation is a good thing and makes you efficient. Whether everything can be automated ? No, here are few consideration where automation is not needed.

One off,ad-hoc and non-frequent activities.
One elephant and blind people, do they recognize elephant or pillar/rope? Perspective, subjective observations and feelings.
Exploring, searching and digging the unknown..no objective just sake of uncovering and knowing things better.
The efforts taken for making a full ROBOT are more than work/benefit out of it. Am I saying return on investment? yes.
Using ROBOT to kill a masquito? Simple and ineffective objectives...sometimes to maintain as-is and/or simple working around, is the right choice.

This point to a very fact, a well defined objective and cost Vs benefit analysis.

Hope you are with me now..and enjoying the journey...

What is automation ? What needs to be automated ?

noreply@blogger.com (Anand Tambey) — Fri, 09 Jan 2009 21:39:00 +0000

For me, automation is :-

Replace the activities with dumb ROBOTs, if you do something today, in the exact same manner, you did it yesterday and it eats your time(Dumb, Routine and repetitive tasks)
Replace the activities with dumb ROBOTs, if you do something today, in the different manner, you did it yesterday because of human gaps and errors.(Precision and Accuracy required)
Replace the activities with dumb, efficient and strong ROBOTs, if you can not do something as it is difficult/impossible to be done by human.(Like counting occupied tracks of CD)
Replace the activities with dumb ROBOTs, if you do something today, has taken time different than you did it yesterday because of human breaks and thinking time.(Saving human time)
Replace the activities with smart thinking ROBOTs, if you do something today, in a different manner, you did it yesterday, based on available means and conditions. (Complex decision making based on changing factors)

You can easily corelate above experiences to test automation. Watch this space for more in this challanging yet promising area of test automation.

Part-3 : Why Prompt Status Tracking and Reporting needed ?

noreply@blogger.com (Anand Tambey) — Wed, 31 Dec 2008 03:39:00 +0000

Overheard the following dialog between Test Leads to PM/Customer (rather hidden thoughts of them):-

Test Lead: Testing is nearly complete.
Customer/PM: You don’t want to tell.
Test Lead: Testing is half way through.
Customer/PM: What!!!Are you sure of getting it completed?

Test Lead: There are few or many bugs.
Customer/PM: I am asking Apples, you are giving Oranges.

Test Lead: Can not say.
Customer/PM: You are looking Blank.

Test Lead: I am afraid testing will complete in time.
Customer/PM: So Sad!!!

The above scenario depicts the typical condition where no or little status data is collected and no factual reports are being used.

The answer is Management by fact:

Testing is 80% complete.( Quantitative, not vague)
20% of high priority bugs, 60% of medium priority and 20% of low priority bugs. (Efficiency of System and testing)
For each category,20% will be resolved but 80% will be covered in next build.(Prioritization - Pareto)
Testing will complete in time, if we target 15% for today and will use last day as buffer.(Achievable Goal Setting-Daily)
With this X% of high priority bugs, which are not resolved yet? We can not release system for live deployment.(Challenge Faced : Having to Say NO)

Management by fact, is using quantitative measures and metrics to manage the Planning, execution, and reporting of software testing.What you can measure, you can manage it.

Part-2 : Why we should not ignore environment and data setup ?

noreply@blogger.com (Anand Tambey) — Wed, 31 Dec 2008 02:46:00 +0000

A typical review meeting and aftermath : -

Test Manager: I could see blanks in pre-conditions OR prerequisites? Can you explain?

Test Lead: Boss, it is not necessary, and moreover we are pressed with time. These test cases could be finished by our agile engineers.

Test Manager: What about test data & setup? Are you prepared as I have not seen test data in few test cases?

Test Lead: Boss, environment would be set up by the configuration manager. Well for test data, we can create on the fly.

In test execution cycle:

Test Manager: We are running behind schedule always and you people working on weekends?

Test Lead: Yeah the build given had configuration issues & was buggy. We wasted our efforts in attempting almost all test cases, where we knew, many test cases cannot be executed as earlier test cases failed.

Test Manager: Hmm, that's bad.

Test Lead: Also test data creation had taken time as everyone was preparing loads of data individually and efforts wasted for creating similar data twice or more by different testers.

Test Manager: Alas..we can learn the lesson but the hard way now...

In a nutshell :

Prerequisites or pre-condition plays a vital role in test cases, where it indicates the dependencies on other test cases OR particular environment. You can save efforts by proactively not executing unwanted or invalidated test cases based on same.
Test data and environment setup should be done & ensured before test execution in a planned manner and should be entry criteria. If time is a constraint, we should give estimates to allocate time to this vital activity.
Last but not the least test data should be prepared in such a way that it could be reused and fewer efforts spend on redundant data. The automation of test data creation could do wonders in this situation!!!

Part-1 : What if Test Plan Or Strategy Not There?

noreply@blogger.com (Anand Tambey) — Tue, 23 Dec 2008 20:36:00 +0000

You always know the importance of umbrella when it rains...right...and same time you feel you forgot it at home...similar the case with test planning, here are few interesting outcomes if you forgot to plan...:)

Customer : Well this is critical change and no time left for market launch.

Test Damager : Yes will do it...as you wish!!! We can always work on weekends.

Test Damager: Why the testing not started, just write test case and execute...

Test Engineer: No boss, build is late...

Test Damager: Well write few more then..you get paid for...

Test Engineer: (Cycle 1) Yes we executed all...but 80 out of 100 test cases failed...

Test Damager: Excellent job....

Test Engineer: (Cycle 2) Boss database is slow for 2-3 days...it is taking time to execute test cases...

Test Damager: Well we are checking with support....

Test Engineer: (Cycle 2) Yes we executed all over again ...but 50 out of 100 test cases failed...

Test Damager: Excellent..quality improving by time...

Test Engineer: (Cycle 3)Boss, we executed only 20 ...we are already at deadline...

Test Damager: Why is the weekend for..just cut down your movie plans....

Customer : Why it is showing wrong price...I'd paid charges and lost revenue..was it tested at all??

Test Damager : Well I'd executed 100% of test cases.Oh no, my test case`is referring to old requirement:(

In a nutshell:-

You would be testing what is not intended, needed or agreed..No Scope...
There is no way to know when to start and when to stop...No assumptions or gating criterias...
Surprises happens as always good things were expected....No risk analysis
The only objective of team is EXECUTE and it got EXECUTED at last...ultimately no focus..no objective....

Journey into world of testing

noreply@blogger.com (Anand Tambey) — Sat, 20 Dec 2008 18:00:00 +0000

We would be starting a journey towards understanding important phases in software testing named as :-

1. Test Planning and Strategy

2. Test Environment Setup

3. Test Execution with Status Reporting

Please stay in touch for next blogs and some original thoughts with best practices.