So because I’m not a programmer, I won’t develop my own performance testing tool or add the features I want to an existing one, and I will have to use expensive commercial tools with or without their flaws.

What I’m trying to say is that even if there are hundreds of open source tools, for a regular testing effort they are, so often(at least for me), unsuitable.
I admit that for testing a web application open source is the best way to go, but in other areas… it is more to be done.
Also there are a lot of commercial tools out there, so finding a tool that suits your needs is not that dificult but it is time consuming.
Sadly, for me, the commercial tools are also stringly oriected on web, and only the most expensive ones cand be used for other tehnologies.

I agree that not all problems are solveable, but, when you are charging $10,000 plus per client and the software cannot perform the basic function that you are paying for, you expect that the problem will be solved.

Commercial tools definatley COULD exist in this space, however, IMHO, the current tool vendors lag behind the mainstream technology, have products that themselves are very buggy and do not justify the price that is charged for them.

Having to re-run 30% of your 1200 tests every day because of a tool bug that cannot be fixed by the vendor is completely unacceptable, and when the open-source replacement has worked flawlessly ever since, I would rather spend that $10,000 on a good testing services company to write tests in an open source tool instead.

But, as you point out, I am not an average tester, but I have different expectations than most of what a tester should be … http://www.teknologika.com/blog/the-tester-is-dead-long-live-the-sdet/

If that test tool would have been open source, and you had the posibility to debug and solve the problem yourself doesn’t automatically mean that you were necessarily able to do it. When I say you I don’t mean your person, I’m just reffering to a general programmer/tester from a company… and we are not all, geniuses.

And you didn’t said nothing about open source tools in non-development software companies, were I consider the commercial alternative the best one.

So again a I must say that the future is a) open source, b) use Microsoft’s tools, c)good commercial tools and services.

Regards, and sorry for my late answer(2009), I didn’t look at the date of your blog post before sending my previous message.

The thing about open source tools is that I can debug them and solve the problems myself. To date there is no issue that I have been unable to solve myself, and that includes building my own UI automation library from scratch to test windows applications.

The other thing that I didn’t say at the time when I wrote this post back in 2007 as I was under NDA, is the inclusion of the testing tools in Visual Studio 2010. So the way of the future is a) open source or b) use Microsoft’s tools.

Even if, I am also a big fan of opensource testing tools, I had the “chance” to participate in projects of implementing automated software testing in very large companies(telecom, banks) and I really understood their need of a comercial testing tool because of the suport, and reliability they ofer.

These companies had large testing departments but small or non existing development teams, so “simply open up the code and fix it or add the feature” its imposibile or in the best case scenario very expensive.

Till now I haven’t still meet this situation: “Most commercial vendors take 4-6 months or more to investigate, test and resolve issues.” and I think that there is no guarantee that a comunity of an open source tool could solve an issue faster.

So in conclusion I think that: <<commercial tools are dead in the long term, for software development companies which have the knowledge and the people to suport or to build their own open source testing tool ”

Regards,
John

The “total number of bugs” in this example is actually the difference between bugs found and bugs fixed. To get the key inputs into the system i.e. average and standard deviation, I pick a two week period on the project, either the last two weeks, or the last stabilization period.

However, this is a very good explanation of how to run the Monte Carlo simulation. Well done!