Software QA Post-It Notes

Singapore DBS bank services outage

noreply@blogger.com (Ding) — Fri, 06 Aug 2010 01:19:00 +0000

In the head lines of the Strait Times for recent DBS bank outage survey (Click for news), it points all the causes to two engineers and a faulty cable. The two field support engineers didn't change the cable via maintenance interface but using the support center's instructions. IBM claimed that if the correct procedures are used, the system will take precautions procedures automatically to preserve full data integrity.

As this is serious outage that not only impacting the image of DBS but also Singapore's local banking brand as a whole, MAS is taking this seriously by giving prompt sanctions. The bank in question is asked to set aside further SGD$230 million for additional regulatory operational risk. That is great. Nothing is more important to a bank than the money. So monetary punishment is an effective approach. It also send a positive signal to the rest of the banks to Singapore to look the the same issue. I am sure the it is not a localized issue to DBS.

However, I feel sorry for the engineers involved. They did what most good field support engineers will do. They took logical steps to diagnose the issue, divide and conquer the problems, escalate it to supervisors at the right time when they couldn't solve the problems, and give appropriate suggestion to change the cable at wee hour. For goodsake, they performed the cable changes solely based on the instructions from the support center. The "it's all your fault because you didn't using the right procedures to change cable" blame is cruel to them without understanding the root cause.

You can't blame the faulty cable as well. We all know that electronic products have a limited lifespan. It will happen sooner or later, won't they?

Then, why the main media as The Straits Time, point all their fingers to "two engineers and a cable"? The report jointly issued by DBS and IBM included a section for corrective actions. IBM go further mentioning that they discipline the direct personnel involved. I assumed they are the field support engineer and his counterpart in the IBM regional support center whom gave him instructions.

I feel the two engineers are the victims of a bigger issue within IBM: Lack of trainings to support teams for up-to-date issues recovery procedures and change management in-place is not sufficient to avoid risk. For example, why the regional support center fail to give the right procedures at the first place? Why IBM doesn't dispatch two field support engineers, instead of one, to data center to investigate (have they?)? Given the criticality of the data infrastructure, the fundamental question to IBM is, why this kind of cable-fault simple scenario is not covered in their quality control policies in the first place.

MAS blows up the issue to "lack of robust technology risk management framework", "not sufficient oversight of maintenance, functional and operation practices" and "now following MAS guidelines". However, doesn't this issue also show MAS itself, has problem in their monitoring and control exercise?

Engineers, always the first to blame and last to be appreciated.

How to gauge the quality of a product - quality metrices

noreply@blogger.com (Ding) — Mon, 02 Aug 2010 14:07:00 +0000

When should we say the product is ready to be shipped? When can we really stop testing? How do we gauge the quality of a product? How do we measure the risk of a project? All in all, we need to have data for decision making. Then what data we need to have to support our decisions? What matrices exactly do we need?

Below are some useful matrices that we can refer to.

Accumulated Defect Find Rate against Accumulated Defect Closure Rate

The objective of this chart is to decide the stability and shippable of the project

Two lines are deviated further indicates the project is unstable. You might need to invest more R&D resources on the defects. However, it's pretty common to have this pattern in the early phase of the project.
Two lines are merging indicates the project is stabilizing. This represents a healthy project. This is often happen during the mid to late phase of the project. During this period, we can move some of the R&D resources on new project initiatives. However, if this happens too early, it indicates not enough QA definitely and QA resources need to be invested promptly to ensure the quality of the release.
Two lines are merged indicates the project is shippable. Usually this would not happen automatically, but by intervention of project manager and leads to move the lower priority open bugs to next project.

While I worked in BigWorld, we used this method very often. I found this is very useful to keep all the leads on the same page. It will be easier to support our decisions which will involve more than a team, such as moving resources from one team to another. There won't have any emotional feelings on such decisions.

Defects Overview with Severity- a Pie chart

This is another popular pie chart for defects severity. Meaning, using a pie chart to show all the valid defects which has been filed in accordance to their specified severity might do the wonder. The severity of each defect can be classified as severity 1, severity 2 and severity 3 for example. Usual distribution of the severity might be 33%, 33%, 34% or about that. The high percentage in S1, for example as high as 70%, might indicate some underlying issues, such as testers are not testing enough simply.

Is this pie chart really useful? To me, it just gives you another piece of information. For example, what does that mean by 80%, 10%, 10% or 60%, 30%, 10%? Remember, testers might tweak the severity to get attentions on their defects. It may be early phase of the project that higher severity bugs are caught more and earlier. On the other hand, how about 10%, 30%, 60%? It may be from a very mature product that the product under-test has minor new features only in the release. Or, it may indicate you have a great development teams with quality in-mind and quality environment/systems (for example, prevent the developers from checking-in defective codes) in-placed?!

For whatever the reasons and outcome of this pie chart, it gives you a piece of information whereby you might have already know ahead. You know what kind of products you are testing, be it a prototype project, or minor features introduced in a mature product, be it you are in early phase or ending phase. Remember, you already schedule daily to scrub or triage the bugs so I am sure you have the high severity bugs (such as show stoppers or blockers) in mind to follow up.

Side note, I advocate "Just Enough Information" (or whatever the term) for project management to make decisions or trigger actions. For information or data, which will absolutely no chance to trigger actions or cause butterflying effects, to me is nice to have and for reading pleasures of everyone only.

QA Life Cycle (QALC)

noreply@blogger.com (Ding) — Mon, 02 Aug 2010 09:48:00 +0000

I am sure you heard of Software Development Life Cycle (SDLC). Quality Assurance Life Cycles (QALC) is another variate term which is not so common in use but it is important concept for SQA specialists. To put it simply, SDLC emphasizes on how the software developed, evolved and flowed from requirements to the end products. QALC focuses on the QA activities that integrated and flowed within the SDLC. Therefore, QALC is highly depending on the SDLC models in your environments. For example, the QALC of agile model is definitely not the same as in waterfall model.

Typical question we are interested is: Where does testing start? How should we end testing effort for each release?

There is no straight-forward answer.

Source: Engineering Notebook

In Waterfall model, each phase starts only the earlier phase has exited. The SDLC of a waterfall model consists of:

Analysis phase

Requirements phase
Design phase
Implementation phase
Testing phase
Maintenance phase

As you can see, the testing effort starts after the implementation phase exits. It can't start earlier than that. Therefore, the design bug which captured in the testing phase is always very costly. Having said that, this doesn't mean that waterfall model is definitely a bad model. It is still very applicable in mature products environment which requirements are predictable and requirements changes during development are minimal or close to none.

Next, I will drill into the QALC in waterfall model.

Checklists to set up quickly with Blogspot and Google with Adsense

noreply@blogger.com (Ding) — Sun, 01 Aug 2010 11:07:00 +0000

Firstly, you need to set up all the necessary accounts to get started.

Register a website or blog with Google sites or Blogspot. There are different features on each, if you only want to blog to share your articles, I would suggest Blogspot. Click here for an example of google sites with Google Adsense.
Register an account with Google Adsense - to add advertisement in your website and earn money.
Register an account with Google Webmaster Tools - to publicize your website to reach out more surfers.
Register with most popular affiliate programme: Amazon Associates.
Optional: Explore other affiliate programmes, such as Google Affiliate Network - To be explored.

Now, you are set to go.

The Blogspot way

You will get a unique Publisher-ID once you subscribe to Google Adsense. When you log-in to your blogspot, it will automatically link to your Google Adsense if you log-in with the same log-in email for both. To switch, click the [Monetize] tab. Here, you can configure Adsense, and Amazon Associate.

Configuring Adsense is straight-forward. Click on [Monetize] tab -> [Adsense Overview] ->[Customize/Add ads], then configure a layout to place the advertisement holders. Pick one that suits your needs.

It takes some time to familiar with Amazon Associate set up. Click on [Monetize] tab -> [Amazon Associates], you will notice your amazon associates ID is displayed. It is not a bad idea to add Amazon Product Reviews. You can refer to some examples at the bottom of this page.

Tips: To add Amazon product reviews codes, you can add it through a widget: "HTML/Javascripts" without a title. There, even when you change the blogspot design templace, you won't lose the scripts.

Highly recommending this new book in store: Make Money Online: Roadmap of a Dot Com Mogul. The author John Chow earns real money via blogging, advertisement and selective affiliate programmes. He also earn by teaching you how to earn money online. You will definitely learn something useful about making money from this guru. I think his ideas are refreshing and proven methods that you can apply directly.

Lean and Six Sigma - 2 - Value, Value Streams, Flow, Pull and Perfection

noreply@blogger.com (Ding) — Fri, 30 Jul 2010 14:19:00 +0000

The central idea is to analyze process and improve it. That's why we need Process Analysts. What is process? Imagine it is a black box which we throw in inputs and it produces outputs. It is a series of actions or activities to transform one to another.

For example, customer order product A. The materials, resources will go through certain processes and then produce product A to customer. Not all processes are valuable to customers, such as project coordinator which has lesser value to customer. We firstly study what processes or activities have values to customers. Those that has no values to customers are considered wastes, such as revamping the website, rework of some jobs. What customers want eventually has values to customers as customers are only focusing on the end products. Working backward, customers' requirements pull the materials or resources from upstreams to downstreams. Customer's pulling cascades all the way back to the lowest level supplier, enabling just-in-time production. Value streams are how the materials, or information, or people flow. We want to know if there is over-processing of information, over-production of materials or keep too much inventory, etc. When we talk about flow, it actually involves time as main parameter. We can measure time as waiting time, process time, idle time, lead time etc. In manufacturing environment, we focus on how the materials are being processed, for example through machinery. However, software environment is different story because programmer produces codes. Programmers are human, which is hard to measure their work rate predictably liked machines. But, we can make sure the software development environment is set up in such a way that it helps programmers to do work more efficiently. I am interested to explore and map them in term of software environment as well as the relationship with CMMI.

Summary:

Value - value-added activities, non-value added activities: necessary waste, pure waste

Value Streams - all activities that create value

7 types of wastes: over-production, transportation, inventory, unnecessary movement, waiting, defective outputs, over-processing

Process Map -> Value Stream Map

Flow - product or service flowing through process, main metric: time

Takt Time = (available time) / (customer demand rate for available time)

Pull

Kanban method

Visual Control - Andon -> Isn't this liked SCRUM master remove the blocker promptly?

Perfection - This is a journey, keep moving.

Reference: MIT Course: Introduction to Lean Six-Sigma Methods.

Lean and Six Sigma - 1 - Introduction

noreply@blogger.com (Ding) — Wed, 28 Jul 2010 07:39:00 +0000

Since returning to Singapore for jobs hunting, I have been asked a few times during the interviews about Lean and Six Sigma. I admitted that I do not have direct working experience with Lean and Six Sigma. But I can learn! Well, it is hard to convince the potential employers. Out of frustration, I decided to write about them.

Lean was emerged from post World-War II Japanese automobile industry to eliminate waste. Six-sigma was developed by Motorola to eliminate variation and improve quality systematically by reducing defects. The focus was shifted from reducing cost and improvement efficiency to eliminate waste and add values to customers continuously.

Now, I think I can apply '5S' even in my small flat to de-clutter the house. For example, let's imaging a work of managing housing documents, such as CPF statements, utilities bills, etc, which all of us are facing as it's part and partial of our lives. How do you organize those documents?

Sort - Categorize the documentations, such as group banking statements in one box, CFP statements in another.
Set in order / Straighten - Sort the documents in order, e.g. chronologically.
Shine / Scrub - Regularly clean the areas where you keep documents to avoid dirtiness. Eliminate spring cleaning. This is everyday work.
Standardize - Apply the defined methods in all areas in consistent ways, such as applying stickers to label each group of documents.
Sustain - To maintain it religiously, of course.

Lockheed Martin applies Lean Thinking. From 1997 to 2001, LM reduced the building time for space launch vehicle from 48 months to 18 months. Other company applying Lean include Rockwell Collins. They have close to 48% of reduction in inventory, and 25% of improvement in productivity after implementing Lean.

Let's look at Kanban. Kanban means card-signal. In simple form, it advocates supplying parts based on needs 'pull' from demands. When the parts and materials from downstream activities are finished, it signals upstreams to start replenishing the stocks. It sounds simple but yet it takes Toyota 30 years to apply it company wide. Therefore, there is huge gaps between knowing it theoretically and applying it practically. I think, I'm just a newbie still wondering around the main door of Lean Six-Sigma.

To be continued.

Reference: MIT Course: Introduction to Lean Six-Sigma Methods.