The Distress Test

How Safe is That Ecommerce Site?

noreply@blogger.com (Cathy) — Fri, 26 Oct 2007 21:28:00 +0000

You go to an online store, find the perfect pair of shoes to go with your perfect halloween costume, and gleefully whip out your credit card. You've heard a lot about how easy identity theft is on the internet, but with that certificate thing and the little lock in the corner of your browser means your credit card number is safe, right?

Yes and no. It means that the chance of someone 'sniffing' little bits of information coming from your computer to the shoe sales server is encrypted and not easily translated for their own shoe shopping excursion. Do you know what happens to your credit card information once the store has received your info, though?

In many cases, all the information you put in the forms to buy your shoes is stored in a database. Once that data is stored, it is probably readable by anyone with access to that database. A developer writing the software could view it. An angry, underpaid employee can view it. The Russian mafia hacking into sites with weak security could swipe the whole thing. A few web sites ask if you want your credit card information saved with the rest of your account, like Amazon, but really...most do not. Assume that anything you put in a form will be saved and stored somewhere.

Yeah, it's no joke that identity theft is easy. In system administration, we say your server is only as secure as your weakest password. When most of your general users have 'password' as their password, general theft is ridiculously easy. They also say most thefts are of opportunity - if you have $100 on the table at Starbucks, someone is going to take it. There are millions of jerks every day who wonder if they can 'haxx0r' your account and get away with it.

Most companies do not take even the simplest steps to protect their users' data. I recommended we encrypt all credit card numbers entered on our ecommerce sites at my company after a security breach at our hosting company. If our hosting company is at risk, our clients are at risk. This was mostly a move to protect ourselves from litigation, which was happening to a much larger company whose servers had been breached. Sending emails to our clients to tell them that their information might have been compromised was scary - we didn't know how they would react. After the next security breach with the encryption in place, we were able to send emails to all of our clients and assure them that we had measures in effect to protect their data. They appreciated our honesty in telling them that the breach had happened, and it was unlikely their data was compromised.

All cyphers are breakable given enough time, but some encryption prevents over the shoulder compromises, which honestly, are a far larger threat than a 'haxx0r' breaking into your server. The statistics on employee crime, so called 'shrinkwrap theft', is far more costly then the Winona Ryder shoplifters.

Is Your Process the Problem?

noreply@blogger.com (Cathy) — Fri, 26 Oct 2007 18:55:00 +0000

Following processes. What images does that conjure for you? For me, I think of corporatism, inflexibility, and inability to deviate from the rule book. Pretty shocking, since I'm someone who creates a lot of processes. These are complaints I often hear from people who follow *my* processes.

Where does this image come from? Tell me if this has happened to you. You call a bank, an insurance company or whatever with a mistake you found on your invoice, you call the support number and the customer service rep is completely unable to help you. Your problem isn't in the rule book, so they pass you off to the next department that is also unable to help you. The cycle repeats endlessly. You're exasperated, your problem still isn't solved, and all the customer service reps are tired of answering your calls. You send an email to a customer support address listed on the webpage expressing your exasperation, and you get an email response back saying they can't help you through email, call the customer support number.

This happened to me recently with a mistake in a booking reservation for the Thanksgiving holiday. As someone who has to create process design, analyze and tweak them, this is extremely distressing to me. Processes are important, they serve a purpose, and they need to be followed. However, the company's inability to get me in touch with someone who could solve my problem within a reasonable time frame shows serious problems with their processes. I think it bothered me more that no one was very bothered by how terrible an experience this was. If any one of the people I spoke with had been more empathetic, and appeared to genuinely understand the problem, I would have been considerably more pleasant in my conversations, and willing to accept it as a mistake (it happens, I understand that). It took me 3 days, several hours of phone calls and emails to finally get in touch with a supervisor who would listen, and resolve the problem. In the end, we were offered a $100 travel voucher for our next flight. The problem with my reservation, though, has not been resolved.

I would rather had my problem resolved the first time I called than a $100 certificate (what happened to the free tickets? They were probably giving away too many...). A couple of years ago this company was facing bankruptcy. If there were just 10 people a month who were having the same problem as me, and someone giving away $100 vouchers to placate the customer, that's $12,000 a year. For a multibillion dollar company, that may be a drop in the bucket, except obviously it added up enough for the company to face bankruptcy. The solution to this bankruptcy crisis was probably to lay off people and outsource the call center to another country, which is not solving the root problem. Cutting branches off a tree with diseased roots will not save the tree.

Processes should be designed to ensure consistency. Consistency means people spend less time thinking about what they should be doing at the next step, and can focus more time on real issues.

Believe me, in my job I have more than one battle about processes. Are they necessary, are they slowing things down, etc. Standard stuff. The processes to move things from clients to developers to QA to deployment are designed to document and track items being moved through. And to analyze the system for defects. When a defect is found, the question is asked, "Where is this defect coming from?". Then solutions and discussions are made to fix the source of the defect. The processes are designed to FIND problems so they can be resolved. The processes are not designed so that an item gets passed around in an endless chain. I certainly do become irritated when people deviate from processes, but that's when what they are doing prevents changes from being documented and analyzed, which opens the gateway to errors being introduced and more difficult to identify. Every step gives an opportunity that there is some path for someone to make a decision on how to move forward.

Processes are important, but make sure your process isn't the problem.

Making Mistakes

noreply@blogger.com (Cathy) — Thu, 18 Oct 2007 18:24:00 +0000

No one likes making mistakes. They like mistakes being pointed out to them even less. There are some people who can take bad news better than others, and some no matter how delicate, polite and diplomatic you are will take it personally. The only thing you can do is to focus on the objective: Get the problem fixed. Are you there to criticize and make someone's mistake even worse, or are you there to help facilitate discussion and come to a resolution?

Always keep perspective and focus on the solution, not the person, not the problem. Assume that everyone wants to do a good job, and give them every opportunity to support and encourage them to do so. Positive reinforcement is far more effective for increasing productivity and morale then punishment. Only bad behavior should be punished. Mistakes should be met with encouragement and motivation to do better.

Quality in Customer Service Care

noreply@blogger.com (Cathy) — Thu, 24 May 2007 18:25:00 +0000

I'm having trouble right now with my cell phone provider. I purchased a pre-paid phone back in March from their online store and returned it. The return policy says the money will be refunded within 30 days. By my calendar, May 24 is well outside the 30 days promised. Despite my repeated calls to find out where in the process my refund is, the only solution offered to me is, "I'm very sorry...if you don't have it by such-and-such date, call us back." I've done this...4 times.

Every person, every company makes mistakes. If you make a mistake, it's important to apologize. It's more important to rectify the situation. An apology with no action is empty and insincere.

"Quality" has a nebulous meaning depending on how you want to define it. It is undefinable quantitatively. Quality is an abstract, not concrete, thus I will definite it abstractly as "whatever will cause the customer to be dissatisfied and wish they had never bought the product".

You may debate ad nauseum about what will make a customer dissatisfied situationally, but there are some universal generalizations:

1) Failing to deliver what is promised.
2) Failing to respond to a customer in a way that makes them feel their concerns are valid.
3) Failing to rectify or respond to a customer's dissatisfaction when it is your fault.

Anyone, in any company, situation, or business/personal relationship will be dissatisfied under these conditions. High maintenance or low maintenance customer defines tolerance and willingness to let you fix the problem. Even the lowest maintenance customer will eventually reach saturation level if each of these points are not addressed.

Quality assurance does not mean the number of bug reports submitted. It means overall satisfaction for the customer. That means everybody - account managers, developers, customer service representatives - is a quality assurance specialist.

Don't Ask the Customer to Troubleshoot: Use Logs

noreply@blogger.com (Cathy) — Wed, 23 May 2007 22:54:00 +0000

Logs are essential tools for making sure things are "working under the hood". We just encountered a problem with one of our sites where we cannot verify whether emails being sent to a supplier contain correct product ids. While brainstorming about this, we came up with several ideas:

1) Place test orders and ask the supplier to forward their invoices to us so we can inspect the output. This is an inconvenience to the suppliers, appears unprofessional (we can't handle testing ourselves?), and is likely to return spotty or inaccurate results. We want to fix this today. If we receive 2 out of 10 supplier emails, do we wait for the other to reply to confirm? What if it takes several days for all of them to reply? Do we need all of them to reply to confirm correctness?

2) Place test orders, but change the supplier email to come to our test address instead of the real supplier. When confirmed, change the supplier emails back. The problem with this is we have to place about 30 orders in production to check most of the test cases. If we place and void them all (much to the annoyance of my employee), there are still 30 voided orders in the cart system. To the client, this would appear that we are testing, not confirming deployment, on the production server. The likelihood of error is high - placing a high volume of orders, changing emails, changing them back, voiding all orders. Too many steps means more chances for us to introduce error. If we have too many things to do and keep track of, we spend less time analyzing results.

The solution: piggyback emails. We add code that sends us a copy of all emails sent to a supplier. We get the benefit of 1) getting real data from real products and orders 2) suppliers and customer do not have to be involved. Each of these items gives us the most accurate information to alert us to problems and verify correctness with the least amount of effort required.

Even though the first two solutions are not optimal, we refactored to get to the right one.

QA as a Career

noreply@blogger.com (Cathy) — Mon, 25 Dec 2006 19:06:00 +0000

When I was looking to hire a new QA engineer to take over my normal tasks so I could spend more time in strategy, I received resumes from many brilliant and capable candidates. So many I was overwhelmed (and a little bit sadden about the number of smart, unemployed tech workers in Seattle). As much as I sympathized with them, I received very few resumes that were qualified for my position (listed C++ projects from college, but no web application projects). Most of them listed that their career goal was to start off in test and move into development. Did they not realize QA is a career path in itself?

I have many friends who are career QA people with no plans of going into development. Myself, I did what many people perceive to be the reverse. Actually, I've probably changed careers 3 times in 10 years. I began my early career as a system administrator. I was a perl tools developer. And now I'm a quality assurance lead.

Is being a system administrator or developer more "tech-sexy" than QA? Maybe. It certainly is much easier to find people who want to be developers and sysadmins.

In some companies, there may be a path where a QA person can move into development. The ones where my QA friends and I work do not have this progression. We're experts with the tools we use for regression. We can pretty much expect what kind of bugs we're going to see based on the developer who wrote the code. It would be a career change to go into development. New tools. New IDEs. Different set of goals and problems to solve.

There is certainly some value that I see for ex-QA people to become developers. However, there was a lot of value for me to go from system administrator to developer to QA. As system administrator, I had exposure on how to analyze server performance. I can brainstorm with the network administrator and communicate effectively on what needs to be accomplished to build better quality systems. If I don't have a tool to do the job I need, I can write one without pulling resources from developers whose sole job should be to billed for the time they work. I sympathize with the developers when managers have unreasonable expectations for how long it will take to build a new system. At the same time, I can vouch for managers when developers want too much time. I touch every layer of our applications - databases, code, and servers. If someone is having difficulty with one of these areas, they can consult me as an internal third-party.

QA doesn't have to be viewed as a passive role. Sometimes I click buttons. Sometimes it's repetive and boring. Sometimes I feel that I'm not taken seriously. But I solve problems. I have an active role in modifying processes to fix problems that tarnish our reputation. I push our system administrators and developers to think about what they are doing. I can relate to their frustrations. I argue. I negotiate. I debate. I compromise. I observe. I report.

Personally, I'm rather glad to not be the person responsible for handling a crisis when a server crashes anymore.

Teamwork

noreply@blogger.com (Cathy) — Mon, 25 Dec 2006 19:03:00 +0000

Teamwork is the name of the game. We only work at a fraction of what we're capable of without it. But how do you achieve it? How do you get everyone working together?

Leaders need to be a motivating force, driving people in the direction you need to go. But mostly, I believe that you need a history together. You need to be able to trust and help the people you are working with.

Lately there's been a buzz about whether a certain MMP game called World of Warcraft teaches good business skills. A friend of mine and I play together mostly as a duo and we're able to accomplish feats that take full groups of 5. We aren't really doing anything special, except we work really, really well together. We've been gaming together for over 10 years. We can almost anticipate each other's moves, and we make our move to best compliment each other. Do we have some sort of ESP? Not really. We trust that we're both going to do our part.

Can gaming teach good lessons? Sounds silly, but multiplayer games that require teamwork can teach good lessons. Now only if some people who play them remember the lessons in kindergarten to play nicely...

Function over Form

noreply@blogger.com (Cathy) — Mon, 25 Dec 2006 19:01:00 +0000

One of our developers passed us a link about how some of the most popular, most used websites are not attractive designs. Craigslist and Ebay, for example. However, there must be something compelling about them that people want to use.

The-Surprising-Truth-About-Ugly-Websites

Often people I talk to make assumptions about my company and what we do for our clients. We're an internet marketing firm. We have great designers. We do design, but that's not our primary focus. A snazzy website alone won't win you customers, visitors or shoppers. It's the content needed for both customers and crawlers. Is the information easy to get to? Web crawlers are a type of client, too.

Welcome to the Distress Test!

noreply@blogger.com (Cathy) — Mon, 25 Dec 2006 18:59:00 +0000

Welcome to the blog of a web QA engineer. Web QA is a different challenge than traditional shrinkwrap QA. While many practices still apply, there are a few that don't. How do you deal with issues arising from customers that insist on editing their own content, for example? This is not a problem typical QA encounters. Typically, customers tend not to have access to the source. In most cases on the web, the customers do have access to the HTML source.

Developers expect to work with QA during their career. Most web designers are not used to quality control and find it to be restrictive. Getting everyone to follow processes and understand that it leads to better quality and less work can be difficult. Distressing. Challenging. But the challenge is part of the fun. I'm treading new ground here. It's exhilarating too.

The focus of this site is to explore the challenges of QA in a web specific environment.