Real-Time Technology Solutions (RTTS) Blogs and News Updates

The technology just crept up on us....

2009-06-30T08:30:47.279-04:00

I was at my younger son's baseball game the other day and the game was running a bit longer than usual. My older son called me with his cell phone to see when I was getting home, because I would be barbequing and he wanted to eat. My wife called me to ask me the same question. Also, one of my friends text messaged me to see how the game was going, as it was a playoff game and his son is in the same league. I also took pictures of my son's 1st at -bat and then filmed his second at-bat with my Blackberry Smart Phone so I could show it to him after the game. In between innings I browsed through my emails for work and then checked the latest score on the Yankees/Mets game.

Then it struck me - when did the technology of work and home life blend together? Did I miss it? I had been reading about convergence for a long time, but it just occurred to me that it happened in such a sneaky, unsuspecting way, that it had crept into every part of our lives. My kids listen to iPods and play every game system imaginable, my Tivo that I had suped up runs on Linux. We all have flat screen TVs evolved from computer monitors. My Blackberry had become a camera, camcorder, browser and game system. Co-workers can always find me at home and family and friends can always find me at work. Now our work and home lives have been hopelessly intertwined with the same technology. Life will never be the same – for better and worse.

Posted by Bill Hayduk on Tuesday, June 30, 2009 8:30 AM EDT
Leave a comment | View Comments (0)

Do testers need Programming Skills?

2009-06-29T12:40:27.467-04:00

Does the sun rise in the morning? Ok that's not fair unless I am more specific. I've been reading articles/blogs/opinions/comments, etc on whether testers need programming skills. All of these articles make sound arguments and all are logical and on and on. I agree that testing skills and programming skills are two totally different things and that a good tester may not necessarily be a good programmer (or vice versa). The context of all of these is in regards to functional testing. When it comes to performance testing, the answer must be that the tester has to have programming skills. They do not need to be a rocket scientist in the language, and I tell everyone who asks and who I teach, that they need to be proficient in string manipulation coding. Every tool I know and use has a programming language behind it (C, C++, C#, Java, JSP, VB, VB.NET, and a few proprietary). If someone came up with a performance testing tool that had artificial intelligence well enough to not need custom coding to get an automated transaction to work, they would corner the market. Just one additional skill that is needed in conjunction with programming expertise is the ability to analyze and find patterns. Without these you would not be able to use your programming skills to manipulate a server response to get what you need.

As usual, if you have any comments or anything to add, please feel free to do so.

Posted by Jonathan Harris on Thursday, June 25, 2009 2:27 PM EDT
Leave a comment | View Comments (0)

Second Class Citizens - Really?!?

2009-05-27T15:12:20.617-04:00

Every now and then, on testing blogs, forums, lists and in the workplace, I hear the classic tester's lament "why are we second class citizens?"

Get over it!

There are really only two possible scenarios, if you feel like a second class citizen as a tester.

First, there is the remote possibility that you are a highly skilled tester, who adds significant value to the project, yet you are not viewed as valuable as the developers. I've got news for you - if that's the case, it's time for you to move on to a place that will appreciate you.

Second, and more likely, is that you did it to yourself. I think we testers as a whole sometimes play the victim role. We got bad or no requirements from the BAs. We got code that did not meet the requirements from dev. We got no time in the schedule for proper testing from project management.

This "second class citizen" lament has been bugging me for a while. And I'm not the only one. Matt Heusser recommends critical thinking and asking the tough questions:

As an industry, we seem determined to complain that we are treated as second class citizens.

What do I think are the keys to getting first class?

Well, one place to start is the questions above.

Alan Page, however, takes a more hard line stance, closer to my own viewpoint:

What really bugs me (believe me, I've had to edit the "bad" words out of this rant more than once already) is when people change the meanings as an attempt to raise the value of test, or show that test "has a seat at the table", or is a "first-class citizen".

I have to tell you, I'm sick and tired of testers whining about not getting respect.

If you want respect, you have to give respect and you have to earn respect.

Well said.

Now, I will readily admit that, earlier in my career, testing was not as fun. Back then, I did encounter some "second class" scenarios, where QA was not taken seriously. On those early projects I had to fight hard to earn respect. Some of those projects certainly viewed test as a necessary evil. I can still hear the refrain "just QA it" from one assignment. To this day, I have not a clue what that is supposed to mean. However, those projects are few and far between nowadays. I cannot remember the last time a developer did not take me seriously. I have been an integral team member on a number of projects, collaborating with developers on test plans and execution. If you are smart and get things done, you will earn the respect of developers and management alike. In my opinion, the adversarial relationship between dev and test is more cliche now than reality. Am I saying that you will not encounter a developer who does not respect you? Of course not. As long as humans are involved, anything is possible. But while it is possible to come across a few people who do not respect test, if you feel there is some organizational or cultural bias against test where you work, it may be time to look in the mirror.

Second class testers, step your game up!

Are you holding up your end of the bargain and doing the necessary leg work before logging a bug? Are you researching the spec, digging through the app's log file to glean additional information, writing clear bug reports with impeccable repro steps as including detailed descriptions of what you were doing, what you expected to happen (should be) and what actually happened (as is)?
Are you keeping up on the latest testing techniques and technologies? Reading blogs? Writing a blog?
Do you exhibit a critical and questioning mind when it comes to your application's business domain and technology platform? Are you going through the motions with your testing? Or are you really trying to understand how this thing is built? I like the analogy, I want to attribute it to a thread on the software-testing yahoo group, of test to building/engineering inspectors. How can you test something you know nothing/little about?

Posted by Tom Eble on Wednesday, May 27, 2009 11:14 AM EDT
Leave a comment | View Comments (0)

IBM Rational Software Conference

2009-05-26T16:38:40.039-04:00

Saturday I am heading back to Orlando for my 15th or 16th user conference. The first was in 1994 and was run by SQA, Inc. Robot and Test Manager were the only products back then. In 1997 Rational took over and since 2003 IBM hosts the conference which features RQM, RFT, RPT, AppScan and dozens of other industry leading quality assurance testing tools.

RTTS will have a booth in the exhibit hall again and I am looking forward to Jeff Bocarsly's, RTTS Division Manager, presentation on Internationalized Testing with Rational Functional Tester.

I am looking forward to seeing everyone we have known for years and learning what is new from IBM. Hopefully it will be fun, but not as fun as last year.

Posted by Ron Axelrod on Tuesday, May 26, 2009 11:57 AM EDT
Leave a comment | View Comments (0)

The Advent of ALM Software

2009-04-28T11:48:33.367-04:00

Over the last 12 – 18 months, the application life cycle and accompanying tools have gotten much play and adaptation in the marketplace. And – it’s white hot.

Application Lifecycle Management (or ALM) is defined as the process of software development as an iterative cycle and is coordinated through the use of software products. ALM can increase productivity, improve quality, promote collaboration among teams in different locations, speed up the development and test cycles and decrease costs.

A lead analyst at one of the top firms recently told me that 55-60% of all development and test management is managed and tracked in Microsoft Office. Wow! As someone who has been on projects using MS Office primarily as the asset tools, implementing these ALM tools should REALLY improve communication and information sharing along with providing reporting across assets. Welcome to the new millennium!

Typical ALM suites are typically role-based and provide software for requirements, transaction/use case/user story definition, project management, development management, change management, test management, defect management, metrics gathering and analysis, monitoring and reporting, and role workflow and methodology. An ALM tool may have some or all of these features.

One of the first ALM tools that we used was Test Director from Mercury Interactive, now part of the Quality Center suite at HP. Originally geared towards testers and now with a refocus towards business analysts and developers, Test Director provided a view of ALM through a quality prism and provided a central data store for requirements, test cases, manual test runs and defects with a distributed architecture (i.e. web) for the new distributed software dev teams. It was (and is) easy-to-use, web-based and provided a great central information area.

Other enterprise players that have joined the game include Borland, Microsoft (with its Visual Studio Team System), Seapine and MKS. Now IBM has joined in with its Jazz platform (Team Concert for developers, Requirements Composer for BAs and Quality Manager for testers). I would not be surprised to learn that Oracle is either developing (less likely) or looking to purchase (more likely) an ALM tool to compliment it’s entry into the development and test space with its tools for dev and test acquired from Sun and Empirix respectively.

And there’s more. The small and medium sized businesses have their tools to chose from. There is Artisan and Lighthouse and Rally and others. Not to mention TOMOS Software, which just won Gartner’s “Cool Tools in Application Development 2009” and is a SaaS based solution (and was incubated here).

So this movement by the vendors of dev and test products in this direction is good news for all of you practitioners out there who have struggled aggregating information and tried to make sense out of all the information that is needed to determine the progress and health of an application under build. Now all you need to do is pick the right tool for your needs and get going and you’re on your way to a more efficient software process.

Please comment if you are already using an ALM tool and let us know what you think of it!

Posted by Bill Hayduk on Tuesday, April 28, 2009 11:48 AM EDT
Leave a comment | View Comments (0)

Getting It Right The First Time

2009-03-13T12:23:49.069-04:00

When Microsoft released Visual Studio Team System (VSTS) 2005, I was excited to take a look at the new web testing component. Over the past 20+ years, I have used most of the performance testing tools on the market, so typically I first look at the basic functionality; how to record, how to playback and what numbers it gives me. I was pleasantly surprised at the ease of use and depth of functionality I found. The only awkward feature I mucked around with was pulling data from external files (datapools). The setup of this was not straightforward. The newest release, VSTS 2008, makes this task simple and very straightforward, so kudos to that enhancement. RTTS has performed numerous highly successful engagements utilizing VSTS because of its rich feature set and the price is attractive for clients. I am looking forward to the next release in 2010.

For anyone out there using VSTS, how do you like the tool?

Posted by Jonathan Harris on Wednesday, March 11, 2009 11:52 AM EDT
Leave a comment | View Comments (0)

Building a Reliable Scripting Framework

2009-02-12T14:49:22.832-05:00

Every performance engagement I do is both unique and repetitive. Unique in regards to a new application in a new test environment. By new I mean that it's different from any other test I've done before, even if some or many of the components are similar. They are brought together and used differently from site to site. By repetitive I am referring to the methodology I use to get the job done. Even though the process is the same, the details of getting the tasks completed change.

In every engagement I do, during the scripting phase when I create the automated test scripts, I automatically think of ways to make the scripts robust and reliable. Robust in the way it is scripted so that if changes are needed, it would have the least amount of impact on time and complexity to make the changes. Making the script reliable means handling known and possibly unknown conditions that would otherwise hang or stop the user. There are other factors in making a script reliable as well, such as after detecting issues having the ability to report on them. This way you can roll it back into the reliability of the script for next time it occurs.

This is something I have been doing for a long time, but recently put a name to it; a "Reliable Scripting Framework." Here is an example. Let's say you are testing a Citrix application and there are issues when running large numbers of virtual-users from a single test driver. This has something to do with the ICA middleware not being written to support many connections from a single machine. Sometimes typed characters get lost. This can cause the script to get an unexpected response. You have two options: (1) add code to detect that something went wrong and attempt to correct it, or (2) add code to test that what you typed was correct and avoid the error in the first place. Both approaches are acceptable, but it would take less thought and code to avoid the problem in the first place.

Here is a second example. In web testing there are times when a request fails intermittently due to a network or application glitch. If you try the same call again, it works perfectly. Though these are serious issues to attend to, when you performance test, having the ability to report on the error and retry would be nice. In most test tools you need to handle this yourself in code. If you do not explicitly handle it you may end up getting cascading errors which muddy up the waters when you try to analyze what happened. So putting a framework in place to handle something like this would make the script robust and more reliable.

If you have any form of scripting framework you use, I'd like to hear about it in the comments. Take a look in the near future for a whitepaper I'm writing about creating a Reliable Scripting Framework using these two examples.

Posted by Jonathan Harris on Wednesday, February 11, 2009 2:21 PM EST
Leave a comment | View Comments (0)

Rational Performance Tester Tips for use with ITCAM

2009-02-04T12:12:15.748-05:00

I've been using IBM Rational Performance Tester (RPT) since it was the Prevue tool long ago. I've used it in many ways, outside of the box, so to speak, for things other than performance testing. One of those things is for monitoring, specifically for production monitoring to verify the target system(s) are up and running correctly. Currently I am involved in an engagement utilizing IBM's ITCAM (IBM Tivoli Composite Application Management) product. As part of its use, we are creating RPT tests (tests is the term used in RPT for automated scripts). For those who may use ITCAM in the future, here are a few up-front working tips.

(1) After creating tests in RPT, you upload them to ITCAM. ITCAM will only see tests located in the root directory of the project, so you must place the final working versions there.

(2) ITCAM will only see custom code located in the "src\test" directory, so make sure you keep them in this default location.

(3) ITCAM does not currently see additional Java classes you create and import into your custom code (i.e. utility/reusable functions). You can get around this by creating a new custom code file and adding it to the beginning of the test. Don't add any executable code to the "exec" function. Place all of your utility functions after the "exec" function. ITCAM will now upload the custom code and since it is located in the default "src\test" location, you do not need to "import" it into the other custom code you create.

I hope this helps some. If anyone has similar tips regarding RPT use in ITCAM, please feel free to share in the comments.

Posted by Jonathan Harris on Monday, January 26, 2009 12:02 PM EST
Leave a comment | View Comments (0)

Vote!

2008-11-03T10:24:12.925-05:00

November 4 is just a few days away. What I cannot believe is that during the last presidential election only 59.6%, or 122 million people, voted (out of 204 million eligible voters).

As a result of low voter turnout in 2004 and previous presidential elections, the United States is ranked near the bottom of 172 countries in terms of voter participation rates (our ranking is 139).

We talk about freedom and liberty but we do not take advantage of one of the greatest privileges we have.

So while I am a huge supporter of one of the candidates, I do not care about who you vote for, I just am passionate that you VOTE. Get up early or leave work at 5:00 for once or file an early voting form, but most importantly find an hour during the day where you can cast your vote. It will make you part of the process and it does make a difference, so get out there and VOTE!!!!
Posted by Ron Axelrod on Friday, October 31, 2008 5:07 PM EDT
Leave a comment | View Comments (0)

Plan for Success the NFL Way

2008-10-20T11:35:24.265-04:00

For the last couple of years I have been coaching NFL-sponsored 5-on-5 flag football for my younger son (who is 8). In the first 2 years, coaches were allowed on the field, which made it easy to diagram plays in the huddle that made sense to 6 and 7 year-olds. This year, since we moved up a division to 8-10 year olds, the coaches must call plays from the sidelines. That made the visualization of diagramming plays on a hand-held whiteboard impossible, what with only 30 seconds in between plays.

So utilizing my background in software, I created a Plan, a strategy document that provides visualization of what we were trying to execute. I started by first providing a visual of each pass route (i.e. ‘square in’, ‘post’ or ‘button hook’) and the showed different formations (‘trips right’, ‘twins left and running back back’ or ‘stack back’) and then combine the formations with the pass patterns to form a play. Now everyone understands what we are trying to do and is on the same page (even 8-year-olds!). And during the game we strategize to call plays that will produce the results we are looking for depending our desired outcome and calculating the risk involved for each play (i.e. interceptions).

If this works for 8-year-olds, why not for software testing? This is no different than creating a Test Plan, or strategy document, that outlines the approach to testing, the strategy utilized and the expected outcomes, depending on the risks (security, usability, performance, etc.). Yet I constantly observe that software projects do not have formal test plans that provide all participants (stakeholders) with the strategy (choice of plays), structure (plays and patterns) or desired results (first down or touchdown) of the tests. Believe me, if you want to be successful in developing and testing software (or in flag football), create a plan that keeps everyone informed.

By the way, we currently stand at 2 wins, 2 losses and 1 tie (we tied the only undefeated team) with by far the youngest team in the league. And everyone plays the same amount of time and everyone has fun. For those not familiar with organized kids’ sports, everyone sharing time and having fun is usually not a path to success (see ‘Kicking and Screaming’ with Will Ferrell about kids’ soccer). But with a plan, we’re getting it done. Good job, kids!!!

Posted by Bill Hayduk on Monday, October 20, 2008 11:34 AM EDT
Leave a comment | View Comments (0)

The Financial Services Mess

2008-09-23T17:09:10.032-04:00

What a mess. Mortgages were sold to people who couldn''t afford them by companies who sold them to other comapnies who resold them to firms who repackaged them to others who created new instruments and resold them and now no one can pay anyone and the entire US economy is about to come crashing down. Does that about cover it?

Some of Lehman''s assets, including 10,000 employees are being acquired by Barclay''s, which means 14,000 people will be on the streets. Bear Stearns is already a memory, with the employees scattered around other brokerages, hedge funds and smaller players. Goldman Sachs and Morgan Stanley are becoming banks to survive.

About once every 8 years Wall Street comes up with a new gimmick to make money and ends up shooting itself (and everyone else) in the foot. The junk bond scandal, the savings and loans, the dot com "new new economy" and now the sub-prime stuff.

Which means the brokerage industry that I grew up working in and with is no longer. A self-inflicted gun shot wound to the head - this time a kill shot. RIP Wall Street. You''ll never be the same. And you did it to yourself. Again....

Posted by Bill Hayduk on Tuesday, September 23, 2008 5:08 PM EDT
Leave a comment | View Comments (0)

Why is Software Quality Still the Poor Stepchild of Software Development?

2008-09-23T16:34:19.328-04:00

In the recent past, our organization reviewed the software quality planning for a very large, complex software implementation for a large organization. The implementation was being performed by a well-known consultancy, one which had successfully completed multiple such implementations in the past. The initial overarching project planning document was approximately 300 pages long; not more than 5 pages were devoted to testing and software quality.

Now, in all fairness, the document did refer to future planning tasks for testing cycles that were scheduled to be written. And some of them were, eventually.

The main point here, though, is one of emphasis. If, at the current stage in the maturity of the industry, software quality attracts so little attention at the beginning of a large project, and one implemented by a respected and successful organization, then it is clear that the case for investing in software quality has not been made to the industry.

Now, you might look at the recent history of the field, and conclude that all the companies that have been building their own software and have deployed with what, on paper, looks like insufficient testing, have “gotten away with it.” The economy has not faltered (at least, not because of software defects), and there haven’t been an overwhelming number of reports in the press about companies stumbling due to major glitches in their software. NIST, the National Institute for Standards and Technology, recently pegged the cost of inadequate software testing at $59 billion. Not chump change, but, you might argue, an acceptable cost in a trillion-dollar economy.

This is looking at it in the wrong way. Software quality investments should be looked at in the same way we look at insurance policies, because that is essentially what they are – mechanisms for reducing risk to the enterprise. Just as no individual can afford to drive without insurance because the personal risk is too high (even though society may be able to absorb the costs of the uninsured), no organization can afford to short on software testing – because the risk is too high. Anyone who views this from the perspective of organizational risk will “get” it – and the argument that the status quo has worked in the past falls flat from a risk perspective, because statistically, the longer one goes without a major failure, the likelier a failure becomes, not the opposite.

Posted by Jeff Bocarsly on Thursday, May 22, 2008 2:41 PM EDT
Leave a comment | View Comments (0)

Tools that make consulting a pleasure

2008-08-29T12:49:57.668-04:00

Over the years I have had many different laptop computers that I used on client sites. One of the nicer ones was a Dell Inspiron with a 15-inch screen and a 1600x1200 screen resolution. The drawback was it weighed over 10 pounds. In recent years I downsized my machines to a 12-inch screen with a 1280x800 resolution. I did this for a few reasons. First was that they only weighed about five pounds and second is that on my numerous plane trips I could actually open it up without having it pressed against my stomach; especially when the person in front of me reclined their seat.

Well, over the past year and a half I flew enough to attain platinum level status in the frequent flyer program for US Airways. I am getting upgraded to first class ahead of the hordes of gold level people. So I upgraded my laptop from a Dell XPS1210 to an Inspiron 1720. This baby rocks. It has a 17-inch screen with a 1900x1200 resolution. Here is the kicker; it has a full keyboard with a numeric keypad. It is a pleasure to use on client sites. I outfitted it with VMware so that I can create a virtual machine and load whatever tools I need in a matter of minutes, and when I'm done I just blow it away.

For added toys of convenience I found a USB powered network hub with four ports. I can't tell you how many times I was in a room with other consultants and there was only a single network cable. I also purchased a USB mobile battery to power USB devices (and my cell phone) and a USB fan and light. To round it all off I got a 320 GB mini drive. With all of this cool stuff I bought a rolling laptop case that also fits nicely on top of my suitcase for the airport (until I check my luggage). Now I have my sights on a mobile printer. I'm looking at a Cannon BJC85.

Does anyone have a BJC85 that can share their opinions of it? What about any cool gadgets or software?

Posted by Jonathan Harris on Thursday, August 28, 2008 11:46 PM EDT
Leave a comment | View Comments (0)

Recruiting Blunders

2008-07-07T09:44:48.011-04:00

Someone recently sent me an interesting article on recruiting blunders. It talked about candidates sniffing their armpits when walking into the interview room; receiving phone calls during the interview and asking the interviewer to leave the room so they could have some privacy; and other ridiculous errors that some candidates commit.

This reminded me of some of our most unusual candidates. While most of the candidates I have interviewed are excellent, some of them boggle the mind. Some are not only unprepared but they seem to speak without thinking. Many of the things they did or answers they gave were so off the wall that they just cannot be made up.

One candidate pulled out a 2-quart bottle of water and slammed it on the table at the beginning of the interview. Then every time he was nervous he would take a huge gulp and finish with a big “Ahh.” Another candidate asked where she would sit if hired. We showed her the cubicle and she said that would never do. Then she saw an office to her liking and asked if she could have that. I told her no, it is the President’s office. We also had a candidate that had greased up his hair with so much oil or gel that it looked like he had just come out of the shower. After a 45 minute interview his hair was still soaking wet.

One of my all-time favorite candidates had an interesting answer to a “tell us a little bit about yourself” question. Near the end of a long-winded monologue the candidate proudly boasted about being a very friendly and cheerful person whom friends have nicknamed, “The Cheerful Idiot.” Definitely not the best way to describe yourself to a potential employer.

We also ask the candidates what their strengths are followed by their weaknesses. One candidate gave us the standard answer for the former, but when we asked for weaknesses we were flabbergasted by the answer. First the candidate said they tend to be late, then said they lose focus when under pressure, then added they had trouble meeting deadlines and finally said they like to goof off from time to time. That candidate still holds the all-time record for weaknesses at four.

There have been other colorful answers from a plethora of other candidates. Some of the best are listed below:

Why should we hire you?
“I don’t know?”

Tell us about an event that really challenged you?
“That’s a silly question.”

Rate your self on a scale from 1 to 10.
“A 10.”

When would you be able to start?
“In a couple of months, I need to rest.”

Would you accept the offer?
“Only if I do not get a better one.”

What do you know about RTTS?
“Not much. I had to travel to get here and did not have time to look at you web site. I was hoping you would tell me what you do.”

Why didn’t you wear a suit to the interview?
“It was hot outside.”

Why did you major in Computer Science?
“I didn’t know what I wanted to do and my brother told me to try it.”

What is your strongest programming language?
“English.”

While many of the answers are very truthful and honest, they did not put the candidates in the best of light. But at least I have never had anyone smell their armpits.
Posted by Ron Axelrod on Thursday, July 03, 2008 2:58 PM EDT
Leave a comment | View Comments (0)

A View from the Road

2008-05-24T11:08:43.917-04:00

I recently went to 2 conferences: Software Test and Performance (in San Mateo, CA) and STAR East (in Orlando). I noticed an interesting trend at these shows.

There was the proliferation of new companies offering test-related tools that either competed with the big boys (HP, IBM, Microsoft, Borland, Compuware) or provided something in a different slant. Companies covering code analysis, security testing, application monitoring, test management, and traditional automated functional and load testing were everywhere. What this leads me to believe is that the software testing industry is still growing and thriving with new products that may be hitting the market under the noses of the primary players. It has always been the path of the big boys to see what the market needs, keep an eye on the new up-and-coming software firms and then swoop in and buy one to enter the marketplace (See HP's and IBM's acquisitions of SPI Dynamics and Watchfire last year in the application security testing space). The IPO is dead thanks to Sarbenes-Oxley, but the targeting of a takeover is the new way for these ISV's to get to the end game.

We are entering the age of relevance of the Tester. Since the adoption of the agile processes (XP, Scrum, etc.) and the slow death of the waterfall process, Testers are finally getting a seat at the table of IT. So, as application lifecycle management (ALM) adds the quality component, that means more focus on the Tester role and more tools to make that role successful.

All in all, the Tester and the tools for the Tester are at an all-time boom. Enjoy new-found respect, job stability and growth, you previously unappreciated toilers of software!

Posted by Bill Hayduk on Monday, May 19, 2008 1:21 PM EDT
Leave a comment | View Comments (0)

A View from the Road

2008-05-24T10:59:44.120-04:00

I recently went to 2 conferences: Software Test and Performance (in San Mateo, CA) and STAR East (in Orlando). I noticed an interesting trend at these shows.

All in all, the Tester and the tools for the Tester are at an all-time boom. Enjoy new-found respect, job stability and growth, you previously unappreciated toilers of software!

Posted by Bill Hayduk on Monday, May 19, 2008 1:21 PM EDT
Leave a comment | View Comments (0)

Not seeing the forest for the trees

2008-05-15T10:37:27.630-04:00

Seeing the bigger picture is an important skill in all aspects of business (and life). Many times I see someone performing a task a certain way over and over, while I’m thinking “Why not do it this way?” or, “If you do this part of the task this way once, the rest would be easier and repeating it would be more efficient.” What makes me smack my head and proclaim "Doh!" is when I'm the recipient of such advice. Sometimes when you are down in the thick of detail, you don't see the forest for the trees. In my last blog entry I talked about statements-of-work (SOWs). Because creating them is a significant component of my work, here is one head-smacker to add; and by the way it also falls under the heading of Evaluation and Improvement, both part of the RTTS methodology (Doh! times two).

Usually, after creating and submitting the SOW we have a follow-up call with the prospective client to discuss the document. We seem to always get into a conversation on process, needs, requirements and tools. Even though I enjoy talking about performance testing and tools, it would probably be a good idea to include an explanation of the previously mentioned topics within the SOW itself. For example, regarding process, describe the steps and artifacts; what is involved, setup and requirements, etc. Since there are a number of performance and scalability test tools available, the SOW could include a generalization of what the available tools are; how they work; and what they can and cannot tell you.

Fitting title for the subject. Have you had any head smacking Doh's lately?

Posted by Jonathan Harris on Monday, April 28, 2008 5:19 PM EDT
Leave a comment | View Comments (0)

Following Directions and Asking the Right Questions

2008-04-23T12:20:42.595-04:00

I write quite a few Statements of Work (SOW); some call them proposals and I'm sure there are a number of other terms as well. The RTTS format I follow fits perfectly with its intended response; acceptance by the client or winning the project in a competitive situation. I take the client’s information and requirements, sift through them for relevance to the project at hand, and match the details against what can and cannot be done with automated performance and scalability testing. When communicating with potential clients and putting together the SOW, there are two aspects that must be covered even before getting to the “how long” and “how much” part. The first is that we understand their business motivations and have the methodologies in place to address them successfully. The second is that we understand their technical requirements and have the tools, skill sets, methodologies and a plan to successfully address them. However, I have found that not all companies maintain relevancy to the project at hand.

Recently, a prospective client issued a Request For Proposal (RFP) and supplied quite a bit of information regarding the testing environment, the application, business transaction volume requirements, expectations and instructions on what to include and exclude from the proposal and strategy presentation. Following the initial RFP was the opportunity for all responding vendors to ask questions. The unusual part, or at least out of the ordinary as I have not had this occur before, is that all of the questions and their answers were combined into a single response document and returned to all of the vendors. I believe this was in response to one of the questions asked by another vendor asking for access to all of the questions and answers (maybe to size up the competition, huh...).

After going through all of the questions and answers, I can surmise that there were maybe four or possibly five vendors involved. Some of the questions were duplicates that I had asked as well. Some of the questions, though valid, were coming out of left field from my perspective. One of the questions referred to additional services and tools when the RFP explicitly stated to include only tools and services needed to test the application. Another question asked for the procedure for reporting issues. As I can see it is an important point once the project starts, I cannot see its relevance in forming a testing strategy. One of the questions asked if the demonstration was against the client’s application. One of the requirements of the presentation was to do a demonstration of the tools selected for the testing with an overall length of the strategy presentation and demo not to exceed two hours. If you've ever performed a demo against a live application you have never seen before, it can take hours if not longer just set it up, not to mention the possibility of discovering the nuances of the script modifications needed to get it to play back correctly. There were a number of questions asked where the answers referred vendors back to information contained in the RFP. I can only guess whether the people asking the questions were testers or salesmen.

Anyway, from my perspective, it's all about decoding information, inferring, making educated hypotheses and putting together the best SOW possible. And, there are always good questions to ask, just make sure they are the right questions.

Posted by Jonathan Harris on Wednesday, April 23, 2008 12:04 PM EDT
Leave a comment | View Comments (0)

Code Coverage? Why do it by Hand?

2008-03-12T12:54:36.182-04:00

Code coverage is one of those items that is better done by computer, yet people still insist on doing by hand.

When it comes to code coverage – estimating how much of your code is exercised by your testing regime – there are a number of top-notch tools. There are commercial tools for most technologies and open-source or bundled tools as well (just Google “open source code coverage”). You might think that, with all the effort and expense that is put toward software testing, development organizations would use these tools to accurately measure what their testing efforts actually buy them.

Not so.

Relatively few organizations do so, even when the tool is open-source and the only cost of ownership is the cost of installation, training and administration. The reasons are many: “Oh, our BA’s know the application so well, they can fully cover it,” or “The developers are handling that with their unit tests…,” or “We’re not there yet; we’re waiting until we shore up our core process.”

Well, maybe. Some reasons are better than others. Few organizations actually invest in even checking out whether there is an appropriate coverage tool for the technologies they are implementing with, or, if there is, what the cost might be. Some do invest in code coverage tools, and then only use them to measure the coverage of unit testing, but not the coverage of Integration or System testing. (Or UAT for that matter.) It is another area where the engineering and technology are mature and available, yet most organizations don’t take advantage of it. It is another example of how technology is ahead of process by leaps and bounds.

Posted by Jeff Bocarsly on Thursday, February 14, 2008 4:43 PM EST
Leave a comment | View Comments (0)

Architecture vs. Design - An Artificial Distinction?

2008-02-07T12:20:07.439-05:00

I recently read a blog entry by Jason Gorman on "Architecture vs. Design - An Artificial Distinction?" where he talked about the scope of architecture and design being arbitrary and the meaning ultimately resides on your point of view. I agree 100% that almost anything is arbitrary based on your point of view, but being in the software testing arena, architecture and design are two totally different things. The distinction comes into play based on a timeline. In the beginning, software and systems are designed (conceptualized) as to what they are intended to do and what they are supposed to support. The architecture of these applications and systems denote their implementation. Many definitions of design talk about planning. Definitions of architecture talk about structure. It makes me think about the comic strip that came out many years ago where each cell in the strip shows the interpretation of a tree and swing by each of the producers.

As proposed by the project sponsor

As specified in the project request

As designed by the senior analyst

As produced by the programmers

As installed at the user's site

What the user wanted

From conception to delivery you never really know what you'll get. In the end in my world, the proof is in the pudding. Design is one thing, but what you end up with can be something entirely different. When I teach methodology, my definition of architecture is that it defines how components work, how they interact with other components and how we interact with them. Ultimately, architecture determines how well or poor a complete solution will work. There are so many ways to put today's technologies together and depending upon the circumstances, needs, resources, requirements, etc., a system with an elegant design may or may not be put into architectural practice to yield the best possible results. Again, my position on architecture and design is that there is a definite a distinction.

Now you know my position on architecture and design, what's yours?

Posted by Jonathan Harris on Wednesday, February 06, 2008 1:53 PM EST
Leave a comment | View Comments (0)

The Utility-Complexity Curve

2008-02-07T12:12:37.189-05:00

As applications are designed and developed, they typically start out with a minimal feature set that is sufficient to garner enough market share to make the product viable. Then, the feature set is built out, either in response to the original vision, competition, or both.

Initially, as features are added, the application becomes more usable, in exchange for a relatively modest increase in complexity to the user. At this point, the Utility-Complexity curve is on a steep incline – with each release, the app becomes significantly more usable, for a modest cost in complexity for the user. However, this game doesn’t last forever – as more features are added, the app continues to become more complex, but the utility of the app doesn’t change by as much – the features being added speak to too tiny a segment of the user population, or are too trivial. At this point, the Utility-Complexity curve peaks and the intuitive character of the app has peaked.

Eventually, as more releases are designed and delivered, the complexity continues to increase but the utility of the application actually declines, as “feature-clutter” sets in. Subsequent releases produce a tool that is incrementally less useful to the user. The Utility-Complexity curve turns over and starts to descend.

Why does this happen to good applications with well-considered designs (at least, initially)? It happens for perfectly good capitalistic reasons - the need drive revenue by continually releasing new versions with new features. How can you convince buyers to pay annual maintenance or upgrade to a new version if you’re just releasing bug fixes and no new features?

So, every new product is on a path from an initial good, intuitive design to a maturity in which the intuitive nature of the original design is violated, where the app becomes less usable with additional features. Vendors continue to add features well beyond the optimized design for their products, and the products become incrementally less useful to the user. I wonder if anyone has tried to calculate the loss in worker productivity due to this sort of feature-creep.

Think about some of the most common applications that you have seen released with significant re-design or enhancement, especially where the changes require a great deal of re-learning by users. Are users 100% more productive with the new application? 50%? 10%? Most likely not. They are probably no more productive than they were with the earlier version, minus the re-learning, which puts productivity in the negative. Why did the vendor do it? Just to have a reason for people and organizations to buy again, and buy more.

This applies across the board - some tools on the market today are hitting or crossing the Utility-Complexity peak. They started out with nice, minimal designs, making them very intuitive and useful products. The initial phase of releases filled out the functionality in areas where things were a bit thin at the outset, and with the addition of features, utility and ease-of-use continued to rise. Somewhere in the more recent releases though, these tools have peaked. The new stuff is just getting in the way. Which means competitors will just come on the market to start the cycle again…

What do you think? Do you agree with my observations and prediction? Or do you have a different opinion?

Posted by Jeff Bocarsly on Friday, February 01, 2008 2:18 PM EST
Leave a comment | View Comments (0)

You Get What You Pay For

2008-02-04T12:20:33.709-05:00

Over the holidays I ran into my neighbor at a party. I asked her why she switched her landscaper last summer. She said her previous landscaper was the least expensive one she found, but did not do the job well, in fact he was awful. Then she added sometimes you get what you pay for. I knew what she meant. Whenever I went for the lowest bidder on a home project, I regretted it.

The conversation reminded me of many of our clients that went with offshore solutions. In the majority of cases it was upper management’s decision to select an offshore consulting firm, mainly because of cost. While $20 per hour for resources was a common price (it has since increased), I had heard as low as $8 per hour. But most of the time these engagements ended in disaster.

One of the most memorable stories involves a client that asked their internal U.S. developers how much it would cost to develop a new system and the developers said five million dollars. The management team then went to an overseas outsourcing vendor and the bid came back at two million. They fired their developers and signed up with the outsourcer. Twelve million dollars later they had a partially built system that was a year and a half late.

I have been witness to quite a few other stories like this. For example, one client contracted two of our test automation experts. To save costs they also employed a dozen overseas consultants. The overseas consultants were instructed to create a specific number of test automation scripts in a two-month period. We were tasked with a similar workload. One week before the scripts were supposed to be run, the overseas group said that nothing was completed. When asked why, they blamed our Test Engineers and said we had not provided them with the proper information. Fortunately we were prepared for this false accusation problem because we had experienced it a number of times before. We had documented everything in emails, including the overseas management’s acknowledgement that we had sent them all the information they needed. They backed down immediately and agreed to work around the clock to fix the problem. Unfortunately this meant that one of our employees had to work 12 hours on Thanksgiving Day and another 13 hours the following Saturday, but with our help, the testing was successfully completed. The bottom line is the overseas consultants were not working towards completing their tasks until they were called upon to show their status. There was no return on their low cost.

We are not the lowest cost provider of software testing and we are also not the highest, but our clients tell us we are the best value for the buck. It is the main reason why a significant amount of our work is from repeat business - our clients say we exceed their expectations and they get more value than what they paid for.

Posted by Ron Axelrod on Thursday, January 31, 2008 3:53 PM EST
Leave a comment | View Comments (0)

Happy New Year! (Error Code 2048)

2008-01-31T18:25:00.115-05:00

I’m sure I am not the only one that brings my career experience into my daily non-work life. So, when I attempted to send several text messages to friends and family members shortly after midnight this past New Year’s Eve (12/31/2007), I was not surprised to encounter messages on my cell phone stating System Busy or Failed to Send Message. It seems that many of us in the larger metropolitan areas had the same idea at the same time and there was just not enough capacity to handle the volume of calls and text messages. So, it was at this point in the evening, over a glass of tasty sparkling white wine, and after wishing good tidings with my newlywed wife, that I had some thoughts pertaining to the technologies and challenges that I will probably encounter within the upcoming year, similar to a press release that RTTS distributed discussing similar concepts.

Here were some of my thoughts before resuming my social engagements for the evening:

RIA Rises - Rich Internet Applications using Web 2.0 technologies will probably make performance and scalability testing more challenging due to the dispersion of physical and logical data sources, the complexities that are now being pushed onto the web clients and the quick paced evolution of communication protocols, including SOAP, REST, JSON, and XML-RPC, amongst others. I don’t think that the Click-N-Test Methodology is going to be very effective in this arena.
Enter the Enterprise - Lately I’ve been seeing corporations creating Intranets that go way beyond the role of providing static documents. Intranets are starting to be integrated into the role of the business, providing process workflow, enabling access to corporate applications, establishing social networks, and facilitating collaboration via Wikis. If providing this functionality to a business with hundreds or thousands of employees, performance becomes a critical factor in determining the success of the application, the productivity of employees, and in some cases corporate morale. Again, applying the principles of Performance-Testing-In-30-Minutes-Or-Less will not work to one's advantage.
Meet the Mainframe - No, I don’t think that Mainframe systems are on the rise again. However, with the momentum of server virtualization picking up, the Mainframe paradigm is taking hold again. Server consolidation will mean that great care needs to be taken when establishing application capacity and assigning system resources to components that are going to share a physical platform.

So, if I had to sum up my thoughts, I would say that performance testing will become even more critical as business and technology continue to merge, and automated performance testing will become even more challenging as result of the increasing complexity of this generation of applications.

What are your thoughts pertaining to these technologies and challenges? Which other ones do you expect to encounter within the upcoming year?

Posted by Matthew Adcock on Wednesday, January 30, 2008 3:24 PM EST
Leave a comment | View Comments (0)

Educating the Business: It Ain?t Widgets

2007-12-24T12:41:27.359-05:00

The biggest communications failure in modern business is not between marketers and customers, nor between management and workers, nor between businesses and their partners or investors, nor between regulators and businesses.

The greatest communications gap is between business resources inside an organization and the internal software teams that build custom software to support them. I say that this is the largest communication gap in the modern history of business because:

· it is pervasive, crossing multiple verticals

· it encompasses nearly everything that modern business does

· it affects business decisions virtually every minute of every day

This disconnect between software development teams and the businesses they support is the 800-pound gorilla sitting in the corner of every meeting and every discussion that occurs between technical software resources and the business resources that depend on them. Business types are from Mars and technical types are from Venus (or the other way around, if you like). They are, too often, ships passing in the dark.

The essence of the miscommunication is that business resources are inclined to think of a software development group as a widget factory, and the technical side in almost every organization has utterly failed to educate their business sponsors about the true nature of the beast.

The Widget Factory

Business folks are, by definition, trained to look at the world as a set of chunkable, repetitious tasks – and they are trained to write project plans and budgets around numbers of tasks and subtasks, each with an estimated time associated with it. Once they have all the tasks listed, and the time estimated for each task, they magically have a cost estimate on which to budget a development project. This is not a bad way to think – it works for much of economic life – the whole manufacturing sector uses it, and it applies to any area of the services economy where tasks are completely well-defined and repetitive. Every widget factory, I imagine, has this sort of predictability associated with it, so the projects for Widgets, Inc. always come out on time and at budget. This could work for software development, in principle, except for one small problem: it ain’t widgets.

The True Nature of the Beast

Alas for everyone involved in software development, the development groups, the support staff, the managers, and the business users, software development is largely a creative activity that just doesn’t work with the widgets model. And this is the key point that software teams fail to convey to the business side, and utterly fail to educate their business colleagues about.

However, businesses must run on budgets, and so everyone on the business side expects that the software side can work in a regime of predictability just like they do. And because business is the sponsor, the software side doesn’t like to say “no”, or “let’s strategize on this”. Instead, everyone plays along, produces a project plan, a set of dates, a set of effort estimates, and off they go again. As dates begin to slip, and hours worked rise, and deadlines loom, the same tempers that frayed last time fray this time.

Is There A Better Way?

In the most concrete, practical terms – probably not. Businesses need to run on budgets and project plans. Software development will only require more creativity as technology progresses, not less. Neither of these will change.

The thing that can change – the education part - can make a difference. All parties, software and business, will benefit if both sides are more aware of the nature of each others’ domains. It won’t change the stress associated with delivery deadlines, or the vagaries of trying to build a project estimate. But it can create an environment of greater understanding between the two sides that will smooth the path when the going gets rough. And the burden of educating the business side about how software is built – that is on the software side.

Posted by Jeff Bocarsly on Monday, December 24, 2007 10:51 AM EST
Leave a comment | View Comments (0)

Dear Santa

2007-12-21T16:15:50.843-05:00

Since this is the holiday season and I am in the spirit, I figured I would apply a tried-and-true way from my childhood of getting my wishes fulfilled. With that in mind…

Dear Santa:

I was good this year (I think) and I was hoping you could help me out with my wishes. Below is a partial list. Feel free to use your infinite discretion.

1) A PC that does not freeze on me every other day. I know I have asked for this in the past, but no matter how many upgrades and state-of-the-art purchases I make, I still freeze up.

2) A clean desk. I know you did not mess it up, but whatever you could do to fix this issue would be greatly appreciated by me (and my co-workers).

3) Test tools that are way too complicated for people to use. Enough with the wizards and stuff in the tools. Give me tools that only consultants can figure out.

4) Vendor conferences at different times of the year in exotic locales. Why do all the vendors have them in May and June? And why in the desert in summer? How about the Bahamas in winter? That works for me!

5) While we’re on the vendors, how about deals with no paperwork. No more 50 forms and 22 acronyms inputted into 30 different systems. One email and that’s it.

6) Can you make cell phones with ear zappers? If you talk on the train/bus while others are around, you get zapped.

7) I need a “Guide to the Y Generation” book. What’s with YouTube, Facebook, MySpace, etc? Let’s force people to have to speak face to face, not Facebook to Facebook. (I should put that comment on my LinkedIn page).

8) Anabolic steroids and HTH for my Knicks. Hey - if it works for Bonds and Clemens, why not? They can’t be any worse.

9) While we’re on sports, how about blinders for my Dallas Cowboys’ Tony Romo? Every time he looks into the stands at Carrie Underwood and Jessica Simpson, he stops looking at T.O. and Witten!

10) A customer who would like a 10-year deal for as many people as we have.

11) A golf club that hits a ball straight. The ones I have bought all seem to be damaged.

12) A fantasy football season where one of my top guys does not break his leg (see Willie Parker and all of my other players over the past).

13) A fantasy tester game where RTTS people are picked in the first round.

14) Oh yeah. One more. Peace on earth and good will towards men (and women and children).

Thanks Santa!!!

Respectfully,

Bill Hayduk

Posted by Bill Hayduk on Friday, December 21, 2007 4:15 PM EST
Leave a comment | View Comments (0)

The Holidays Have Gone Plaid: Prepare for the Back-To-School Season

2007-12-21T15:35:22.484-05:00

Black Friday. Cyber Monday. Green Monday. What's next, Plaid Friday? I don't think I could ever slip in a reference to Spaceballs, but hey I'll try anything for a laugh, or two.

Anyway, it's been quite some time since my last entry, but it's not due to lack of eagerness. I have simply been engrossed in a couple of time-sensitive customer engagements, during which we assisted with holiday e-commerce preparations and an enterprise rollout of a Microsoft SharePoint solution. So, my free time has been costly and as a result this holiday season is rushing past me at ludicrous speed. And I have for the first time purchased all of my holiday gifts through online vendors. There are lots of reports out there publishing positive news for this 2007 Holiday Shopping Season and sales seem to be projected to grow by 20% when compared to last year's online purchases. I have heard of several e-commerce vendors that have experienced significant customer dissatisfaction due to sluggish responses and/or the inability to complete the checkout process due to some form of application error. However, it seems that the vendors that had issues during the 2006 Holiday Shopping Season have prepared themselves very well. One of my personal favorites, Overstock.com, was featured in several articles discussing its investment of almost ten times the amount of resources in previous years towards the goal of better application performance management. And yes, I did shop there this season. For the most part I have had few issues with the large online vendors, most of which occurred on Black Friday. From a personal experience, what I do see is that some of the brick 'n mortar vendors may be experiencing an unanticipated deluge in sales. Many of these smaller web sites have been slow, but even more significant, the order fulfillment process has been painful; I am still waiting for several gifts from these smaller vendors.

So, why did I make the Back-To-School reference? Well, there was a published report this past September 2007 that Online Back-To-School Sales were ranked second in volume when compared to the holiday shopping season. If there were any vendors that experienced issues this past Back-To-School shopping season, I urge them to see what Overstock.com has accomplished. Their plans took some time to implement, but I feel it was well worth it, since I continue to be a return customer.

Happy Holidays!

Posted by Matthew Adcock on Friday, December 21, 2007 11:54 AM EST
Leave a comment | View Comments (0)

Road Warrior-ism

2007-12-21T15:33:31.796-05:00

This may be a little off the blog topic as it does not refer to technology, but since I've have been away from home a lot lately on assignment, thought I’d say something about it. All hail the Road Warrior; those that get to travel all over the world to new and exciting places, and do all sorts of fantastic things. I've been a Road Warrior since I started consulting in the field of testing back in 1988 and whenever I talk with someone about where I've been and for who and what I did there, I get questions like "did you go see this?" or "did you go do that?" Of course they are referring the attractions somewhere within a hundred miles of where I was. In most cases the answer is, "no, I didn't really have time to, but I'll make time for it the next time I go." Now don't get me wrong, I do get to do some fun things. I've visited the airplane graveyard and the Biosphere in Arizona, taken the underground city tour in Seattle, dug for gold and gems in the north-east, gone to countless amusement parks, museums, visited famous beaches and have eaten at many high class restaurants; all of which have been made possible in conjunction with business trips. First and foremost I service the client who has contracted to have me there and in many cases I continue to work back at the hotel. For me this is great since my personal view of my career is that I get paid to play; the work is very challenging and I'm always getting into new situations and devising new solutions.

One of the greatest things about traveling is the frequent traveler programs. When I lived in New Jersey I flew out of Newark and used Continental Airlines since Newark was their hub. I made Gold level status year after year (almost), a couple of them I almost made platinum. Continental had a promotion that if are Gold for 5 years in a row, you stay gold for life. Wouldn't you know it, the fifth year I was on a long term assignment locally and didn't make it. Now I live new Phoenix, so I use US Air (formerly flew America West but bought out by US Air). I've made gold all five years since I've been here and this year will make platinum (RTTS has been busy). One of the benefits of the frequent flyer program is that you get to board the plane before non-frequent flyers, thus ensuring overhead space for your roll-aboard luggage. Others include, depending upon the status level you are, space available for free upgrades to first class and a tag added to any checked luggage marking it as priority so that it comes up on the baggage carrousel first. Also, depending upon your status level you accumulate a percentage of flight miles above and beyond the actual miles flown, which are good towards free flights.

Then there is the hotel frequent stayer program. I like staying at Marriott brands. There are status levels depending upon the number of nights you stay. I've been Gold on the Marriott Rewards program forever. You get points depending upon the amount of money you spend at the hotel that can be used for free nights and they even have a catalog of merchandise you can use your points on. Two of the perks of having elite status at Marriot are the free upgrades to bigger and better rooms and at the full service hotels, admittance to the concierge level and free food (continental breakfast and cocktail hour at night).

Then there are the frequent renter programs for vehicles. I typically use Avis. There aren't any formal status levels, but you get additional flight frequent flyer miles when you rent from them and free upgrades to better class cars. The biggest perk is that you don't have to wait on line. When you arrive, your car is ready with your contract inside.

The points I accumulated over the first 3 years as a road warrior, I was able to take my immediate family (my wife, 5 kids and me) to Cancun, Mexico. All of the flights and the 2 hotel rooms were free. Now that the family is spread out between Arizona, Florida and New Jersey, we use the points to visit the kids and for the kids to visit us. Priorities change over time.

Enter the demons. I was waiting at the gate for a flight recently, watching all of the people and started thinking about traveling. What I find is that when its getting close to boarding time I automatically migrate as close to the jet way door as I can get without crowding anybody, knowing I am elite status and can get on the plane sooner than most. This particular time I was upgraded to first class. The normal process is that those needing assistance are called to board, followed by first class customers, then it’s by boarding zone number. I start to get irritated by the zone 4 person standing between me and the jet way door. Why are they in the way of me and the others who can board before them? Don't they know the boarding sequence? I plan my route on how to get around them when they call first class to board. I'll say "excuse me" and proceed around to their right. Then a woman behind me asks if I am in first class. I say "yes" thinking to myself "first class and elite, thank you very much!" She was in first class as well, but it made me think that you really cannot judge by looks since I see all kinds of people in first class dressed in the full range of attire.

When arriving at my destination airport, my checked bag was one of the last to come up onto the carrousel; so much for tagging it priority. Irritated, but outwardly showing it, I headed to the rental car shuttle. What I've learned is that some airports appear to honor the orange "Priority" tags and some don't. Case in point, US Air at Newark airport never does.

OK! I have an "Avis First" priority reservation. I've had it for a week now. I was here last week and the week before without an issue. I'm dropped off at the "Preferred" counter and have to show my driver’s license and credit card, etc. Hello! Where's the consistency?

I am assuming for the most part that you probably don't know me personally and have not met me personally. I am not one that tends to get stressed or irritated as I mention here. I believe things happen and we just have to navigate our way around them and through them. I have always enjoyed traveling as part of my job and have no plans to stop. The benefits, both for career and personal endeavors, make those momentary demons seem insignificant.

Posted by Jonathan Harris on Sunday, December 16, 2007 9:56 PM EST
Leave a comment | View Comments (0)

RTTS Knows How To Attract and Retain Talent

2007-12-13T15:14:46.570-05:00

A recent article titled “How CIOs Attract and Retain Talent” featured on CIO INSIGHT gave a wide angle look at the trends within IT recruiting. It depicted how some of the leading organizations within IT are attracting and retaining the best talent during a time when demand for resources is high and IT majors are scarce.

As I read through the article I identified many of the recruiting best practices in staffing and retention that are also utilized by us. These include intensive on-the-job training, professional development, and a clear path to career growth. RTTS provides the highest level of training to all of our new hires: 325 hours to be exact. We encourage our employees to pursue professional development opportunities on and off site through mentoring, eLearning courses, and lunch-and-learn webinars and industry events. Our pyramid of professional growth clearly outlines the path to career growth in our organization so that a potential candidate can envision their future career goals being met in two years, five years and beyond.

To attract and retain the best talent, RTTS emphasizes work/life balance, offers competitive salaries, the opportunity to work with our impressive list of Fortune 500 clients and gain exposure to many areas of the business community, a cooperative work environment, and a list of benefits too long for this blog entry!

Our target audience of hires includes recent undergrads, MS/MBAs and career changers. RTTS employs many people taking the first steps in their IT careers and many who have long since grown into senior positions since graduation day. IT majors may currently be scarce but RTTS continues to grow because our company culture and policies actively engage top talent and we are committed to the success of our employees.

I encourage any and all Computer Science and IT faculty as well as Career Services staff to share this entry with students. If you are an HR professional let me know what are some of the things that your company is doing to attract talent. If you are a candidate, what practices are most important to you? I look forward to your responses and a continued exchange of ideas!

Posted by Margaret Lively on Friday, December 07, 2007 11:15 AM EST
Leave a comment | View Comments (0)

Top 10 Interview Do's and Dont's

2007-11-21T16:22:44.353-05:00

Before joining RTTS, I was a member of the Career Services staff at Fordham University. I worked diligently to create opportunities for students to meet with employers on campus and prepare them for the interview process. I reviewed and edited resumes to perfection, administered workshops on how to “Network Beyond the Handshake,” and role-played the interview process so students would have the necessary skills to dazzle recruiters. When undergraduate and graduate students transitioned into the professional arena I walked supportively beside them through each step of the process, calmly coaching from the sidelines, whispering career advice in their ears.

As Director of Recruiting at RTTS I am on the flip side. Now I am the recruiter sitting across the table from candidates, reviewing their resume to see if they meet our standards, and asking the tough questions while listening intently for answers that let me know whether or not they will be a fit for our organization. During the first few weeks in my new role I longed to take a time out during an interview so I could stop and let the candidate take a much needed deep breath, or provide counsel to them on better ways they could have answered the questions they missed the mark on. Although I was not the one in the “hot seat” I found myself squirming around in my chair when I heard candidates provide answers that made no sense, went on forever, or were so inappropriately personal that I could feel my face getting hot out of embarrassment for them!

Over the past six months in my new role I have grown accustomed to the inevitable interview bloopers and have compiled a list of Top 10 Interview Do’s and Don’ts from my experiences as both a Career Services professional and Corporate Recruiter:

DO research the company inside and out. Have a clear understanding of the company’s identity, the specific services or products they provide, and how the role you are interviewing for is positioned within the organization.

DON’T ask the recruiter to tell you about the organization. It is your job to be prepared and make a terrific first impression. If you don’t have a clue as to what the organization is all about, odds are you will not make it past the first interview.

DO dress professionally for your interview. Regardless of what the dress code is within an organization (i.e. business casual); an interview candidate should always look polished and professional. Proper business attire is expected, not optional.

DON’T arrive late for an interview. It isalways a good rule of thumb to arrive 15 minutes ahead of schedule.

DO answer the question, “Tell me a little about yourself,” in 60-90 seconds. The recruiter is looking for you to summarize your resume and provide some insight to your future goals. Be succinct in your answer and careful not to ramble on.

DON’T ask questions until you are invited to do so. An interview is your time to respond to questions and shine. Interrupting the recruiter or jumping in with questions prematurely will make you look pushy and aggressive – a sure turn off!

DO ask, “What are the characteristics of the ideal candidate for this position?” The employer will have the opportunity to go into detail about the role and describe who they see as the ideal fit for the position. You will find that many of your questions will be answered all at once.

DON’T inquire about work hours, vacation, or fringe benefits. At least not during the first (and in some cases second) round of interviews. These discussions are more appropriate once you are aware that an offer will be extended or you have it in your hand.

DO follow up with a thank you letter ASAP. Most candidates will send theirs via email which is the most widely practiced and accepted form of communication. However, don’t underestimate the power of the written word. A hand written thank you will be a rare and welcome treat!

DON’T post inappropriate materials to your social networking site. Recruiters access these sites to learn more about candidates of particular interest. It is important that you remember to keep your profile looking impressively professional – leave the photos from last Saturday night’s adventures set to private!

What do you think? Are there any Do’s and Don’ts that you would like to see included on this list? Your comments are a welcome addition so please leave them below.

Have a Happy and Healthy Thanksgiving!!!

Posted by Margaret Lively on Wednesday, November 21, 2007 2:20 PM EST
Leave a comment | View Comments (0)

Games People Play

2007-10-01T14:11:01.463-04:00

I just read that the Nintendo Wii outsold the Sony Playstation 3 by a ratio of 4 to 1. I am very familiar with the game console systems since my sons (10 and 7) are real gamers. They have the Wii, Playstation 2, GameCube (the Wii's predecessor), PSP (handheld), Nintendo DS (handheld) and the old Nintendo GameBoy Advanced, along with a computer (Dell's XPS) that was specifically designed to play games. Did I just list 7 different gaming systems for my kids?

What happened to playing outside with your friends? Now it's play dates or gaming and there is also the hybrid - a play date that includes gaming. I know Sony, Nintendo and Microsoft are making buku bucks from selling these systems and there are some really interesting games, but how did this happen? And why are we enabling this? I even know 2 adults who work all day and then play some online game all night long. I know other adults who are gaming junkies.

I miss the days when, as kids, we would pick up a ball and bat and play baseball all day long with all the rest of the kids in the neighborhood. Or football or basketball or soccer or tag or some other active, competitive game. Now it's the Wii or PS3 or computer games. Not that there aren't competitive video games and not that the kids don't break a sweat (I actually saw my sons sweating from boxing on the Wii). It just seems like this is fundamentally wrong. But just try taking it away from the kids and they cry about how all of their friends have systems and why they will be ostracized for not having the latest and greatest games.

Maybe I can take a negative and create a positive. Maybe I can contact Sony and Nintendo and Microsoft and see if my kids can become beta testers for the games. And maybe I can bill them out as gaming experts. Maybe we can make extra money and put it away for college. Or maybe I can use the money to buy that flat screen TV with the great graphics that I have my eye on. That would work great with my Xbox…

Posted by Bill Hayduk on Monday, October 01, 2007 2:10 PM EDT
Leave a comment | View Comments (0)

The Holiday Season is Coming! So is Microsoft?s Patch Tuesday!

2007-09-19T12:28:58.370-04:00

Are your applications prepared for the Holiday Season Shopping Blitzkrieg? How about Microsoft’s Patch Tuesday? Recent events regarding Skype’s two day system outage again revealed that we must take into account how an application and its infrastructure perform under expected and manageable conditions, in addition to proactively responding to excessive demand during unpredictable scenarios; the latter always seeming to be the most challenging and subsequently news worthy.

The systemic computer restart that occurred on Patch Tuesday revealed another set of conditions that organizations should be prepared to handle; a very large number of clients all requesting application services within a very short time frame. In the case of Skype, a fairly stable system was suddenly barraged with a large demand for access to its systems. Although the type of chaos that occurred due to the proprietary retry algorithms was unique to Skype, the behavior is similar to an e-commerce application that offers special on-line sales for a period of fifteen (15) minutes or an on-line news site that provides high-profile breaking news stories. The result is a seemingly infinite number of user visits, when compared to system capacity, which quickly overloads your application and/or its infrastructure. And in extreme cases, the demand can knock your systems off-line. So, what can be done to prevent this? First, performance tests can be orchestrated not only to validate the expected behavior of your application, but out-of-the-box performance scenarios can be devised to determine if there are potential scenarios that need to be accounted for. Second, determine the performance characteristics and limitations of your application. Third, enable mechanisms that intelligently determine when your application is reaching its limitations and subsequently govern/throttle access to the application’s services. In it's simplest form a message that says your application is busy is much better than allowing an unfettered number of customers vying for access to scarce resources and potentially undermining the stability of the application.

Although it’s not optimal to turn customers away from your application and possibly diminish immediate financial returns, sometimes it may be beneficial to be the Grinch, minimize your losses, and reassess your application’s needs for moving forward. It's better than making the news for all the wrong reasons.

Posted by Matthew Adcock on Tuesday, September 18, 2007 10:32 PM EDT
Leave a comment | View Comments (0)

RTTS Announces Online Distance Learning Program; First Courses Centered Around IBM Rational Tools -- Software Testing Firm Helps IT Professionals Grow Careers On Their Own Time

2007-08-27T10:26:26.276-04:00

New York, July 31, 2007 -- RTTS, a professional services organization specializing in software quality assurance, announced today the launch of its worldwide online distance-learning program. The online distance learning program is an exciting expansion of RTTS' well-regarded in-depth testing tool training courses. The first online courses will be centered around IBM Rational (NYSE: IBM) tools, with additional courses from HP (NYSE: HPQ), Microsoft (Nasdaq: MSFT) and Compuware (Nasdaq: CPWR) to be added shortly.

"We are offering the IT testing community the opportunity to learn new skills on their own time, at a cost-savings to themselves and their employers," said Bill Hayduk, CEO of RTTS. "We have taken our successful classroom courses and divided them into convenient, hour long segments over a two-week period. IT professionals no longer have to take a full week off from work and fly to another city to improve their skill set – they can do the sessions on their lunch hour, and work on the exercises on their own time. "

The online distance learning courses will be taught in real-time by RTTS' group of expert trainers. The typical curriculum will include introductory courses from the vendors that will be supplemented by RTTS' advanced courses. RTTS' internal team of experts developed the RTTS training programs. "All of our instructors have worked here at RTTS as software testers with a wide variety of Fortune 500 clients," said Eric Smyth, Director of Educational Services, RTTS. "So we are end users of the software as well as instructors. RTTS' online distance learning curriculum focuses on topics that have been experientially useful to us as testers as well as the real-world problems that we know will come up, and the real- world tools that will solve those problems."

RTTS is also offering a unique support system for its online students. While questions can be asked in real-time during the sessions, RTTS will also offer online "office hours" twice per week where students can call, email or IM their instructor to ask questions and get help.

For more information on the online distance-learning program, including schedule and requirements, visit http://www.rttsweb.com/training/distanceLearning/.

About RTTS: RTTS is the premier professional services organization that specializes in providing software quality for critical business applications. With offices in New York, Atlanta, Orlando, Philadelphia and Phoenix, RTTS has been serving Fortune 500 and mid-sized companies since 1996. RTTS draws on its expertise utilizing best-of breed products, expert engineers and proven methodology to provide the foremost end-to-end solution that ensures application functionality, reliability, scalability and availability. For more information visit www.rtts.com. Real-Time Technology Solutions, Inc., RTTS and the Real-Time Technology Solutions, Inc. logo are trademarks and/or service marks of Real-Time Technology Solutions, Inc. in the United States and/or other countries. All other company names, product or service names mentioned in this document may be trademarks of the companies with which they are associated.

O Ye Economic Buyers of Tools ... Caveat Emptor

2007-08-16T13:50:36.406-04:00

It is remarkable to me how many people (read: managers) involved in software quality efforts don't really consider the proposition of investing in an automated test tool for their organization with any great care. The most discerning (and usually the most technically aware) actually do pursue the purchase with due diligence, of course, but those are not the folks I'm thinking about. Often, we encounter the manager who just "bought the industry leader" without inquiring whether it really is the best tool for his/her problem, and without even seeing a demo of the tool. Just a pure marketing-buzz buy. The next step up is the buyer who 'gets it' enough to request a demo of the tool, but then decides to buy based on the demo (performed on the vendor's home-built application target, designed by the vendor's marketers to show just what the marketing squad wants the potential customer to see, and nothing else). This buyer is wowed by the demo, and never askes for the obvious: a proof-of-concept in their own environment, on their own target application. Why sales like these happen so often is a bit of a mystery to me. Why don't buyers ask for the gold standard before buying (does it work on MY application in MY environment?)?
Well, risky as it is to try to explain behavior I truely don't fathom, I'll throw caution to the winds. I think it is a combination of two things, both thoroughly American. The first is the idea that you can get something for nothing. A more American idea there never was. Everyone is brought up to look for a 'deal', a 'sale' and a 'great buy'. Its what makes people give mortagages without verifying income and what makes other people take mortgages without reading the fine print. Its what makes our economy go 'round. The second is the visceral American reaction to technology, better known as the gee-whiz factor. The cooler technology can be made to look (regardless of what is under the hood), the greater its market cache. That is why iPods fly off the shelves, even though my music afficionado friends tell me it is inferior to some competing products. Put both of these together, and you have a marketing knockout-punch, at least for lots of folks involved in software quality.

For my two cents, I'd rather buyers think this one through a bit more. That may sound counterintuitive from someone who makes his living by implementing with these tools - the poorer the tool fit to the target application and project, the more work (for us) to make it work, you might think. And you'd probably be right about that. But the better the fit, the better the better the overall quality of the job. We can produce better, more robust, more effective automation projects with better tools and better fits between tool and project. So...automation tool buyers...caveat emptor!

Posted by Jeff Bocarsly on Thursday, August 16, 2007 1:46 PM EDT
Leave a comment | View Comments (0)

Home Is Not Where The Work Is

2007-07-27T00:18:10.245-04:00

I just got back into the office after spending the last 4 business days working from home. Apparently a "steam-pipe blast blew a 15-by-25-foot crater in a midtown street (7/17), spewing a geyser of near-boiling water, gluey muck and rocks 120 feet into the air and onto terrified spectators as they ran for their lives". We're at 360 Lexington Avenue, between 40th and 41st Streets in Manhattan.

Anyway, because of the advent of VPN and remote desktop, IM and cell phones, working from home is supposed to be "the next best thing to being there", or so it is stated. What I have been reminded since I last worked more than 1 day in-a-row from home (back during the 9/11 aftermath), is that it is not.

The lack of face-to-face social interaction with co-workers replaced with email and instant messenger just doesn't do it for me. Yes, the wife, kids and dog come in and out of the office and remind me that I am not alone in the house. But it's not the same.

And it got me to thinking about firms and projects that use remote workers, along with this mode of communication and interaction, as a defacto standard. Where is the time when you get to know each other, understand each other's likes and dislikes, strong points and weaknesses and compliment each other? Where is the give-and-take banter that promotes teamwork and loyalty? It isn't there and no one can convince me otherwise. Using IM shorthand such as brb, l8r, rofl and symbols such as :)) and ^_^ leaves me feeling :/ ( frustrated) and just doesn't cut it. And the feeling of disconnection that borders on isolation just does not seem to me to be a great way to work.

Anyhow, the office re-opened today and even with the commute, the hole in the ground in front of the building and all the other issues with working in a bustling metropolis, I am happy to be back in the office. IM, email and text messaging all have their places, but I'll take a good water cooler conversation with a real person face-to-face any old day.

Posted by Bill Hayduk on Wednesday, July 25, 2007 5:31 PM EDT
Leave a comment | View Comments (0)

Point, Click, Done!

2007-07-17T09:16:56.714-04:00

How far from the truth is that? When it comes to sophisticated performance testing tools, and even the not so sophisticated ones, how many times has anyone been able to just use the out-of-the-box functionality of a testing tool to accurately and thoroughly test an application? Let me guess at this... (thinking) (thinking) (thinking)... None! Ok, in all fairness, there are rare instances when someone tests a simple application, really simple (like a static web site). Anyway, here is my thought as I was flying home from a recent mentoring engagement.

Actually, this applies to all of the tools I use or have used. They all pretty much have the same base functionality. They record, they play back, and they have some form of programming language (compiled or interpreted). The implementation of these functionalities differs, of course. Regarding programming related functionality, in most cases if a specific functionality is not there, you can usually program around it (yeah, right!). For the newbie’s out there, programming around it only applies if you know it is something you need. This is all well and good, but what I was thinking about was the fact that I need to keep programming (or using) the same custom functionality.

Specifically, since the mentoring I was just doing was against a web application, I needed to use custom code functions that are typical and customary to doing web testing. For instance, pulling a list of values from a listbox, randomly picking one and using it in the next request. What about a function to randomly pick a link on a web page? Wouldn't it be great if there was a function that performed this act that comes with the test tool? Of course there are always unique situations for each application that will require custom coding, but the typical things like I just mentioned would serve 2 purposes. First it is code you do not need to maintain or carry with you (importing) and second if it is coded into the test tool itself, it would probably be just as efficient or maybe even more efficient than your custom code (assuming the tool vendors have the right programmers and/or you’re not dealing with interpreted script code).

This blog entry was geared towards newbies since I was recently training people new to these tools. They were all programmers in some form or other (very technical, which is always a pleasure when teaching a technical subject). So for typical code neccessities, let’s have the tool functionality built in. But for custom code – hey, all die-hard programmers out there, we all love our own code creations, right? (wink, wink)

Posted by Jonathan Harris on Monday, July 16, 2007 5:58 PM EDT
Leave a comment | View Comments (0)

The Big Three Show Their Wares

2007-07-11T15:45:02.027-04:00

OK. I finally have (somewhat recuperated) from my whirlwind tour of Compuware in Detroit, IBM in Orlando and HP in Las Vegas for the last 3 weeks (not to mention STAR East 2 weeks before Compuware and 2 weddings on the weekends between all of the conferences and a brief vacation to the Chesapeake Bay area).

My first stop was Detroit and Compuware for the Partner Summit. Compuware again has shown that they get technology. Some of their offerings convince me that they are going in the right direction for customers. Optimal Delivery Manager is the type of dashboard that will allow users to manage test activities from project management to requirements to functional and performance test activities – providing much needed traceability. It borrows some of the key features of Changepoint (their PPM offering) and provides an ability to manage all facets of testing. Peter Karmanos, the founder and CEO of Compuware, was the primary keynote. ‘Pete’ (as he is admiringly known throughout Compuware) reminds me a bit of Jack Welsh, the gruff and brilliant ex-CEO of GE, in that he is says what he thinks. It was interesting for me, seeing someone who started a firm as I did and actually take it to a place where it is global in scope and hitting revenue numbers that well exceed $1 billion. Stephanie Moore of Forrester Research ended the conference with an interesting take on why outsourcing to India is failing and how niche firms who specialize are successful.

The next stop was IBM and the IBM Rational Software Conference in Orlando. From our perspective, there was approximately a 50% increase in the number of partners there. So IBM, who wins many awards for their approach to partnerships, has really (and finally!) infiltrated their concepts into the Rational group. From a quality perspective, the new version of Rational Performance Tester (RPT) has really become a ready-for-primetime player in the load/performance/scalability tools arena. The latest release adds IP aliasing, root cause analysis, additional protocol support (for Citrix, SIP, Seibel 9.8, mySAP and SAP GUI), and has added SOA quality testing. Rational Functional Tester (RFT) has added support for Infragistics, AJAX, .Net, Firefox and Adobe Flash/Flex. IBM is also going after the process market with a quality process management tool that provides traceability with requirements and defects. IBM sees a differentiation between the quality initiatives of subject matter experts, implementers and coders/testers and is working towards providing each with tools for assisting in the testing field. And IBM feels there is a huge need for process. IBM also announced 2 acquisitions – Watchfire and Telelogic (more on this in a later blog).

The last stop was the HP Software Universe at the Venetian in Las Vegas. We were interested to see how the previously named Mercury Interactive, the biggest player in the testing space, had been assimilated. The answer - Mercury is gone. It's all HP now. At the keynote speech by Mark Hurd, the CEO of HP, my thought was “is there a more impressive CEO in the technology field than Mark Hurd?” If there is one, I haven’t seen him/her. He is a straight-shooting, no-bull type of leader. Mr. Hurd explained that HP needed to automate much of its business before it could preach this to its customers. HP’s goal is to reduce the cost of IT for itself and its customers and highlighted 5 areas of improvement: (1) increase efficiencies and automation, (2) the needs of business outweigh the ability to deliver, (3) improve quality, (4) speed and agility matter, and (5) reduce risk. HP then highlighted how they would accomplish these issues. There seems to be a big play to integrate Quality Center tightly with requirements and traceability. HP is also very interested in the data warehouse sector. Their recent release of NeoView, a data warehousing appliance, will have a significant impact on the lower end of the marketplace, especially in pharma. And HP’s SOA strategy, which ties its previous acquisition of Systinet (for SOA governance) with QACenter (for quality) and Business Availability Center (for management) to me, is the most complete end-to-end solution in the marketplace. Very impressive. And HP, on the heals of the previous week’s announcement by IBM of its purchase of a security firm (Watchfire), announced the acquisition of SPI Dynamics, an RTTS alliance partner.

In summary, it is interesting to note that the major vendors have moved away from a ‘software testing’ message to now more of an ‘application lifecycle quality management’ (ALQM) message (as has RTTS). Both IBM and HP preach increased speed and agility of the business needs of customers. All three vendors (including Compuware) are in a rush to get SOA solutions firmly entrenched in the mindshare of customers. All three are touting traceability and management oversight by tying their project and portfolio tools to the ALQM space. All now have solutions for security testing of applications (Compuware has application security testing wrapped in its DevPartner tools). Borland and other smaller players seem to be getting left in the dust. The 4^th big player (Microsoft) is making a very aggressive play in the marketplace via Visual Studio Team System and has the distribution channel (the Visual Studio platform) to pull it off. In seeing the ‘big three’ present their views of the software quality world, I see much more consolidation coming this way, with companies and products being added from the automated build, change management and operations sectors being acquired to complete the offerings. Let’s see where the marketplace goes and who is left standing in 12 months. My prediction is there will only be 3 big players left in the ALQM space.

Posted by Bill Hayduk on Wednesday, July 11, 2007 3:03 PM EDT
Leave a comment | View Comments (0)

The Sopranos - Everyone Dies

2007-06-25T16:13:37.093-04:00

I just got back from 3 weeks of conferences (Compuware Partner Conference, IBM Rational Software Conference, HP Software Universe) and was in on the Disney compound for Rational when the last episode of the Sopranos ran. Disney does not have a deal with HBO, so believe it or not, nowhere on their property could we see the last episode. Luckily Joe Mundy, RTTS' Southeast Regional Manager, lives in Orlando and we were able to persuade (beg) him to have us over. My take on the "black out" ending by David Chase is this: he copped out in case he wants to do a feature movie. This way he can say no one was killed and he can bring Silvio out of his coma.

Since I have been watching it since the beginning and since it was not completed, I would like to finish it for Mr. Chase. Here goes...

Two hit men were sent in to do the job. AJ goes first. I am tired of his act. Tony and Carmella go together, in each other's arms. As the hit men are leaving, they notice Meadow coming in. They bump her off also. Now - just to clean up any lose ends via the Soprano family, one shooter pays a visit to Janice and gives her the Adriana treatment (is everyone as sick of Janice as I am?). The other goes to the hospital and gives Sil the Christopher Maltisanti breathing course. As he is leaving after the hit, he places a call to his boss to inform him that the task has been completed. Cut to Paulie Walnuts putting down the phone, smoking a cigar and toasting himself as the new boss. Now - fade to black.

There you go - I ended for you. Any questions?

Posted by Bill Hayduk on Monday, June 25, 2007 4:13 PM EDT
Leave a comment | View Comments (0)

The LIRR and Microsoft Task Manager? Perfect Together

2007-06-13T15:59:51.296-04:00

No, I am not the New Jersey Governor. And what could I actually be talking about regarding application performance and scalability? Well, without going into a Dennis Miller-like rant (or maybe I will), I’m simply referring to the rather disappointing manner in which the Long Island Rail Road (LIRR) or any mode of public transportation (e.g. airlines) manages to inform its customers of delays, changes, and cancellations. Have you been to LaGuardia Airport recently? Anyway, we’ve all been there, frustrated, standing in seemingly pointless lines, parked on the runway, or sitting on a stalled train that inertia has for all intentions prohibited the train from ever moving again. "Why?" I ask the train conductor. Cue the shoulder shrug…

Well these days it seems that you are even lucky to get that apathetic acknowledgement of what is happening. It’s very frustrating to know that “something” is happening, but not to get any feedback or information pertaining to your delay. If someone, please someone just stand up and explain to us what is happening. Inform us. Set expectations. Manage my satisfaction. Keep me as a customer. That’s fairly simple, uh?

Segue to application performance and customer satisfaction…

So, this morning when I boot up my computer and try to load any program, such as Firefox, nothing happens. I try again. I still see nothing. Gees, I just want to see if that flat screen TV that I ordered had been shipped. Nada. Niente. Nothing. I see nothing except my Desktop and its lineup of icons, but I can hear the hard disk churning away. What is happening? Cue Microsoft Task Manager. Exit stage left LIRR (for now). Microsoft Task Manager is a nifty program that allows me to quickly glance at how my computer’s resources, such as the memory utilization and CPU utilization (amongst other things), are being used. When Task Manager finally loads, I can start to poke around and see what is causing my hard disk to spin like the dickens. I look at the process list. I sort the list by CPU to see what process is currently requiring the most horsepower on my computer. Low and behold, it’s one of the well known antivirus programs. As it turns out, it was performing a system wide scan on my computer. OK, that’s why I have this software; to look for virus and spyware that might be lurking in the depths of my computer. But again, just like the gate agent on my most recent flight who couldn’t seem to manage an answer why we were still waiting to board the plane after 2 hours, why couldn’t this program actually inform me what it was doing? Why couldn’t a simple notification that it was scanning my computer be produced? Maybe a simple message box could have sufficed? Let me know what’s happening. Managing customer expectations goes well beyond running load and volume tests in order to determine application bottlenecks. In the more holistic process of software performance engineering, providing feedback is a key attribute in keeping end-users satiated.

So, I’m not here to pick on this antivirus program, but the lack of feedback from software is more of a norm, rather than the exception. When I submitted my credit card for that flat screen TV the e-commerce web site that I was using gave me no indication of what it was doing while I waited for 20 to 30 seconds to get a confirmation web page back. I think that flag or globe in the Internet Explorer toolbar was waving or doing something to indicate that I actually did submit my payment information. Or did it? I tell you it was not comforting knowing that I just made a large financial investment over the internet as I waited for some clue of what was going on. All I did know is that I just clicked a button to send my credit card information to some remote location and I all had to show for it so far was a blank screen in my web browser. I was thinking, “should I hit the back button and tried it again?”. I’m sure I’m not alone in thinking this way. And I’m sure that non-technical people are even more perplexed on what to do. What could have been done? Perhaps the web application that I was using could have launched a little progress window stating that it was doing something, rather than relying on the web browser’s flashy toolbar as a progress indicator. Perhaps rather than relying on the mouse cursor to change to an hourglass, the application could have provided feedback that was pertinent to the operation it was currently processing. Perhaps the next time I won’t have to rely on Microsoft’s Task Manager to tell me that my antivirus program has taken over my computer to run a system scan. I am betting that this not-so-groundbreaking realization of software performance engineering is much easier to implement when compared to getting some clue why my train is not moving. And I’m sure that the customer perceived performance of your application will be well revered, unlike applications that require people like me to use Task Manager to figure out what is going on.

END RANT
Posted by Matthew Adcock on Wednesday, June 13, 2007 3:59 PM EDT
Leave a comment | View Comments (0)

Why isn't it a standard that all components have interfaces for automated testing?

2007-06-07T14:39:19.453-04:00

Why isn't it a standard that all computing environments and components have interfaces for automated testing?

Where is the engineering sense?
Anyone who has worked at building automated test harnesses knows that while many of the common target environments and UI components work with test automation tools, many still don't. And, if you go back just a few years, even fewer worked.

Building software is supposed to be an engineering practice and process. Software quality should be also. Why isn't an interface that is optimzed for automated testing a standard part of every product design? And why isn't the need for a testing interface part of every computing curriculum in the country, so that engineers are trained to include them as a matter of course?

Well, you might say, it's getting better. And, in fact, it is getting better. With Java, dotNet and Web computing ruling the day, all of the major automated software testing vendors have provided solutions for dealing with the standard UI objects in these environments (and most of the third-party objects too). (Never mind that some of them charge you extra for access to these standard environments.) For web applications, the best-supported browser is IE, which makes sense because it is the de-facto standard in the business world. There already is support for other up-and-coming browsers (IBM Rational's tool, Rational Functional Tester, also supports FireFox as of its most recent release, as does Hewlett-Packard's QTP), and if the browser market splits in a serious way, more automation tool vendors will support more browsers without a doubt. So things, you might say, don't look so bad.

Furthermore, for the middle tier and back ends, access has been around for a long time - you can hit your databases directly from your automated testing tool, and your SOA web services as well, and message queue products likewise have useful interfaces for most test tools. Automated data verification for all of these components has been attainable for a good long while.

So what's my problem? Well, I kind of feel like all these nice things that have been happening for test automation and software quality in the last few years are accidental. It never seems to be a deliberate part of the design strategy, which means that next week, or next year, the next new-new thing might be a step backwards for test automation and software quality engineering.

Even now, there are still many popular components out there where ease of automated testing is clearly not part of the design, not part of the object model, and not a strong interest of the vendor. So even if you can hook the object (and you often can), if the model doesn't expose the object's data, you're still in a tight spot for implementing test automation.

So, where is the engineering sense in all of this? Seems like a no-brainer that incorporating an automated testing interface should be part of how software is designed, and part of the decision when software tools are purchased. It should be a good selling/marketing point for component vendors (some do play up this side, but way too few...), and it shouldn't be too hard to implement. My guess: it won't happen until a virtuous cycle develops in the market - component vendors will have to come to view quality as a selling/marketing/competitive point, and purchasers will have to expect vendors to deliver features that enable and support quality engineering. Don't hold your breath.

Posted by Jeff Bocarsly on Thursday, June 07, 2007 2:39 PM EDT
Leave a comment | View Comments (0)

Houston, We've Had a Problem

2007-05-31T17:04:30.625-04:00

"Houston, we've had a problem." This is a well known quote from Tom Hanks (??); no - it was really James Lovell during the Apollo 13 Mission in 1970, which was supposed to be the 3rd manned lunar-landing mission under NASA's Project Apollo. Although the mission was catastrophic at the time and almost fatal, this particular quote has lived on in infamy to be used in humorous contexts or in my case when I am commercially using or testing an application that starts behaving badly, prompting me with cryptic error messages, taking what seems like eons to respond to a request, or even just refusing to respond to me at all whatsoever. And for some reason this has occurred to me quite often within this past year. This past Black Friday 2006, both Wal-Mart’s and Amazon's e-commerce web applications experienced significant outages and performance-related issues. Well, I didn't want that Microsoft Xbox anyway! On Christmas Day 2006 I could not even create a new account for my nephew on Apple iTunes, so that he could download songs for his brand new Apple iPod. Thankfully we were able to download songs from Green Day the following weekend. Even more recently, using Turbo Tax, 2006 tax returns could not be electronically submitted to the IRS starting several hours prior to the tax deadline. Well, I guess I deserved it for waiting until the last minute. That flat panel TV that I was going to buy using my tax refund would just have to wait a little longer. What's going on here?! Well, as it turns out, these companies just did not anticipate the large demand for their services and merchandise. Whether it was due to stellar marketing, an outstanding sales force, or drastic price slashing for 20-minute sales, their computing infrastructure was just not sized and/or configured properly to sustain the subsequent onslaught of their customer’s demands. So, why am I here? Well, hopefully moving forward we can start to use this Blog to start discussing issues related to the ones I've highlighted here and conversing about better ways to manage the life cycle of application performance. I'm looking forward to see where this is going to lead us.

Posted by Matthew Adcock on Thursday, May 31, 2007 5:04 PM EDT
Leave a comment | View Comments (0)

Travel Time

2007-05-31T14:12:04.812-04:00

Monday signifies the main thrust of my conference travel season. For those who travel frequently for business - my condolences. Between the 2 hour early arrival to the security people dumping out my after-shave because it was more than 2.5 ounces (and more money than I care to think about), travel has become a tiring affair. And what happened to food and movies on the planes? Now all we get are 1 bag of peanuts and "Everybody Loves Raymond". At least they still serve Bloody Marys :).

Two weeks ago we were sponsors at STAR East in Orlando. Monday kicks off our alliance partner tour. First stop - Detroit and a visit to the Compuware Partner Summit. While Detroit cannot be confused with Honolulu or Las Vegas, there are casinos to have some fun in and Compuware shows us a good time. Bob Donald will be our host and always makes for interesting happenings. Compuware has some new offerings and is creating a new marketing message, so we'll see how that all plays out. I was also hoping to catch a Pistons / Spurs NBA finals game, but, as of this writing, the Pistons don't seem to want to hold up their end of the deal. If they can't beat a Cleveland team with LeBron and a bunch of guys who couldn't make my high school team, then I guess I will have to watch Duncan, Ginobili and Eva Longoria's boy toy pound the Cavs into submission.

The following week brings us back to Orlando and our 13th trip to the IBM Rational User Conference (and about my 7th or 8th time at the Swan and Dolphin). We have been sponsors and attendees since IBM was Rational and Rational was SQA. The RUC gives us a chance to hook up with Jeff Schuster and the rest of the test tool dev team for a "look-see" at what they have been up to. We have been beta testing their SOA testing component and like what we see so far. It also means us taking Jeff out for a 54 ounce steak dinner at Shula's or other home of a gargantuan side of beef. Jeff is always on some kind of Atkins-like diet and prefers to clog his arteries with a half of a cow. IBM is spinning a more complete story, if not the most complete story of the entire application lifecycle management space. They are marching towards complete lifecycle traceability and should have some interesting stuff to show us.

The last leg is in Vegas and the HP Software Universe. As I mentioned in a previous blog, it should be interesting to see how HP "does software". Many of the senior Mercury guys are long gone, so this will truly be HP's show. We will have a booth and will be sure to speak with clients to see how the change affects them.

I'll post my thoughts and opinions when I have decompressed from the 3 conferences and can figure out where I am. Until then, does anyone have a bag of peanuts?

Posted by Bill Hayduk on Thursday, May 31, 2007 2:12 PM EDT
Leave a comment | View Comments (0)

Thank God You're Here

2007-05-31T03:12:22.812-04:00

Thought I'd start off with a little thought dump. I just watched a few episodes of "Thank God You're Here", a comedy show where actors are dressed up and thrown into a scene with absolutely no knowledge of what they are getting into and have to improvise in front of an audience and home viewers until a judge has had enough and stops the scene. Some of the actors do quite well, stealing the scene and making it work (mostly through comedy). Others get thrown for a loop and struggle through it (gets boring). This is not too far from my world. There are largely two situations I find myself in. First is when a company has performance testing as part of their pre-deployment in which case my job is to help them “make it work.” Second is when a company encounters a performance problem and calls upon RTTS to help them resolve the problem as quickly and efficiently as possible. That’s the “Thank God You’re Here!” This is the time when we (RTTS) are thrown into the scene, and find out if we have the knowledge and experience to make it work. I’m going to write about many things here, including experiences of mine, perplexing and philosophical questions, methodologies, interesting occurrences, issues and resolutions. I hope some of what I do here will elicit responses and participation or all to share.

Posted by Jonathan Harris on Thursday, May 31, 2007 3:12 AM EDT
Leave a comment | View Comments (0)

The STAR East Conference

2007-05-19T09:52:49.203-04:00

I attended STAR East last week in Orlando, which is sponsored by SQE, who are the publishers of Better Software Magazine and StickyMinds.com. What I realized is how international in flavor the show felt. It seems that after all of this time, software quality has finally become mainstream worldwide as part of the application lifecycle.

I had a thoroughly enjoyable dinner with Theresa Lanowitz of Voke, the premier analyst in the Application Lifecycle Management (ALM) space. We discussed the direction of the software quality industry and other topics, such as the great information that Theresa provides at Voke, along with RTTS’ future direction in the ALM space. I also realized that Theresa grew up in Pittsburgh and I am a life-long Cowboys fan! The horrible flashback of Roger Staubach and the men with the stars on their heads losing twice to Terry Bradshaw, Franco Harris, Mean Joe Greene and the rest of the Steelers. Those of you old enough to remember can understand the pain!

I had an interesting conversation with Yochai Hacohen and Rafi Benami of Radview. They have released a version of their WebLoad performance tool as open source and are hoping that it becomes the dominant opens source tool in its space. Our performance division has always liked the functionality and power of WebLoad, so we will take a look at it to see if it fits into our strategy.

I also met with Michelle Davidson, Site Editor of SearchSoftwareQuality.com (TechTarget) and Holly Bourquin, Director of Publishing of SQE. Both publications sport online resources that are excellent for information regarding software quality. Michelle and I spoke about the need for tools and documented strategies for testing Software Oriented Architectures (SOA) specifically web services. RTTS has been testing web services successfully for years and our resident guru Jeff Bocarsly wrote an excellent white paper on it documenting what SOA is and our approach to testing it.

In 2 weeks the RTTS team goes on the road for 3 weeks is a row to the Compuware Partner Conference (Detroit), the Rational Software Development Conference (Orlando again – our 13^th year there!) and then we finish with a trip to the HP Software Universe show (at the Venetian in Las Vegas – what goes on in Vegas…). We should all be ready to collapse by then. Well, I’m off to 2 little league games for my boys. I hope they faire better than the Yanks (lost to the Mets last night).

Posted by Bill Hayduk on Saturday, May 19, 2007 9:52 AM EDT
Leave a comment | View Comments (0)

The HP Mercury Transition

2007-05-07T09:48:34.390-04:00

Welcome to my new blog! The goal is to provide some insight into RTTS' vision as it relates to business and technology happenings while also delving into my personal day-to-day fun.

On to blogging....

We're all waiting to see what happens in the world of HP. Since the Mercury takeover, it's been pretty quiet. Customers, competitors and users are constantly asking us if we have heard anything. It will be interesting to see if there is a refocus regarding product direction, since Mercury (now HP) is clearly the market leader in test tools. When IBM purchased Rational way back in '03, there was a 2-year time frame where Rational employees were leaving due to the culture change from a mid-size to a giant firm and Big Blue was integrating itself into the day-to-day business. Rational products were almost frozen in time. This may happen again with Mercury. Competitors are definitely using this time to attack their existing base. We at RTTS will be heading out to HP Universe in Las Vegas to hear the new vision. It should be interesting.

Posted by Bill Hayduk on Monday, May 07, 2007 9:48 AM EDT
Leave a comment | View Comments (0)