We’ve managed to source the first three reports on the State of Security from VeraCode, you can download these free of charge.  They provide a wealth of information on the current trends and is a good grounding for many aspects of software security  that you need to know as a software tester.

Free PDF Downloads:

The 4th Edition of State of Security can be downloaded from VeraCode.

I teach my boys about testing in a very informal sense.  They often comment on computers, games and design – their challenges, problems and bugs!  I help them out where possible and often have to explain that programs, like people, aren’t perfect.

But also, something they love doing is colouring in.  So here we present two fun freebies, great for kids (and adults).

The Tester Types Colouring Book - Download

The Testing Planet logo to colour in - Download

Enjoy!

Author Profile – Rosie Sherry

Rosie is the founder of Software Testing Club and loves to run the day to day activities of the community. You can follow her on Twitter @rosiesherry and very occasionally on her personal website – www.rosiesherry.com

This is your chance to ask a very Evil Tester a question and get some scrumptious and devious answers to your most pressing problems resolved.  Anything goes here!

Selected (great & challenging & humorous) questions will be answered in the next edition of The Testing Planet.

We recently came across a new test management tool, TestPad, which takes a different approach to test management.  Along our journey we met and spoke to the founder of TestPad, Stefan Butlin.

What is Testpad?

Testpad is an online tool to plan and execute manual testing, whether that’s making ad-hoc testing less ad-hoc, steering exploratory testing or maintaining detailed lists of test cases.

How is Testpad different?

Testpad is a new take on test management. It moves away from managing test cases one at a time, and instead towards a more flexible checklist approach that makes it much faster to create and maintain scripts.

Test scripts in Testpad are quite free-form and can be used in a wide variety of ways. For example:

• outline guides to steer exploratory testing
• charters for session-based testing
• traditional test cases with detailed steps
• regression tests to cover previous failures
• tests which should be automated but which haven’t been given the time yet
• even just lists of favourite things to check before making a release

Testpad has a unique user-friendly interface. It’s designed to feel and behave like a native keyboard-driven text editor with powerful outlining controls; nothing like a traditional test case manager with the typical one form at a time click-to-edit-click-to-save model. It also has a mobile UI so you can run your tests from the convenience of a touchscreen and with very separate context to the product under test.

Who is Testpad for?

At its most general, Testpad is for anyone who needs to maintain and run a list of tests or checks by hand.

With its script editing speed, it’s especially suitable for use in rapid (or agile) development environments where it’s otherwise hard to keep tests up to date with a continuously evolving product.

Testpad is great for adding control and measurability to exploratory testing without losing the value that an experienced tester can bring.

Currently, Testpad is aimed at smaller software companies that are big enough to have dedicated testers. Most customers to date are Test Managers at companies with 30-100 employees.

Who is Testpad not for?

Currently, Testpad is less suited for use in enterprise. Larger organisations (500+) tend to have the budgets, and emphasis on the mechanics of the process, to justify more expensive and ostensibly rigorous tools. Further, Testpad’s UI doesn’t support groups/departments of users and larger organisations are simply harder to sell to. Having said that, since starting Testpad, I have heard stories from inside two big companies of their having paid big money for the likes of HP’s Quality Center, only to hardly use it in favour of making do with Excel. So, never say never!

At the other end of the scale, Testpad’s subscription pricing model is less suited to tiny teams of 1 or 2 developers whose usage of a test tool will be intermittent. I’m still thinking about this one, so watch this space.

You’re a developer, why create a Test Tool?

My background is more than pure development. In my previous job, I was the CTO of a mobile web business and before that, a designer/manager of a platform for mobile user interfaces. I have therefore experienced first hand the problem of organising test teams and trying to procure the right tools. In every instance, we couldn’t find appropriate tooling and resorted to cobbling together a process built on top of spreadsheets; complete with custom formatting, archiving in wikis, faffing with emailed copies and writing custom VB to layer on some workflow.

For me these frustrations clearly highlighted the gap in the market for a simpler solution to manual testing, one that can keep up with agile product development, and the great opportunity to build a product to help make people’s testing lives easier.

What challenges have you faced creating the tool?

My biggest challenge by far is getting the word out about Testpad – a very different challenge to engineering a product. From talking to customers that have become fans there seems to be an aha! moment when evaluating Testpad. This typically comes when someone tries editing a script themselves and realises how fast it is just to keep typing (hitting enter for new cases, tab/shift-tab for outline structure) and further realising that expected outcomes don’t have to be separate to the steps to execute. The challenge is getting people to that point…

Testpad’s native-like interface is very javascript heavy and relies on a modern browser. During development, most day-to-day testing was on one of Chrome, Safari or Firefox. At least that was true until someone at the BBC who was helping evaluate Testpad first tried it on IE8 and reported it didn’t work at all – doh! Rather than try to support IE8 directly, which would have taken a lot of porting work, Testpad instead prompts for the install of ChromeFrame for users of IE8 and earlier. ChromeFrame is a great solution to get the features of a modern browser into older versions of IE. IE9 worked fine with very little to modify.

In fact, IE8 notwithstanding, after years of cross-browser headaches it is remarkable how consistent the latest crop of browsers are (IE9, Firefox, Chrome and Safari).

What have you learned about testing along the way?

Test practises are evolving all the time. There seems to be a lot of discussion currently about automated testing versus manual testing and the role of ‘traditional’ test teams in ‘modern’ product development. A common theme is the demise of manual testing in the face of increasing popularity and reliance on automation.

From my point of view, I don’t see why it has to be so either-or. Both automated testing and manual testing have a valuable role to play in most projects. Indeed, all of the teams I have talked to over the last year use a mixture of both, with increasing numbers embracing automation at the same time as moving away from fully-scripted manual testing in favour of exploratory testing.

However, whilst the quality of automated testing is rising with increasing sophistication, I think the claimed ‘exploratory testing’ is often closer to being just ad-hoc testing with very variable documentation or process to control it.

When asked by new teams how to go about their testing, my advice is to a) automate as much as possible/cost-effective and b) use exploratory testing for the rest – but where that exploratory testing is guided by a list of feature points and/or more specific scenarios if required.

Using a checklist of aspects/features/scenarios that you don’t want to forget is incredibly re-assuring to testers and managers alike. Teams can go all the way to Session Based Test Management if they like, however, I think the bulk of the value lies in the simple concept of just writing down a list of things that shouldn’t be forgotten.

Obviously I would say that as it’s an ideal use case for Testpad, however, even if teams don’t want to use Testpad, I’d still advocate steering exploratory testing with a simple checklist. I certainly wouldn’t recommend investing in the effort of creating and maintaining fully scripted test cases unless mandated by external requirements. Like automated testing, fully scripted testing will only uncover bugs you predicted might occur – Rumsfeld’s known unknowns if you like, whereas exploratory testing can, and often does, uncover unknown unknowns.

Further, having a simple checklist with a bunch of pass/fails (and bug reports) makes an effective report on the activities of exploratory testing. It gives you and your stakeholders confidence that everything you planned to check has been looked at.

Testpad looks simple, why would someone want to use it?

Well, thanks! It was a key design goal for Testpad to look (and be) simple to use, yet also flexible enough to adapt to the variety of ways people go about their testing; as a friend put it, Testpad is simple in function yet open in purpose. This is being borne out in practise too; I get a lot of feedback saying how much more productive people are finding their testing with Testpad.

Having said that, I do get asked quite often why wouldn’t someone just use a spreadsheet. Indeed, I’ve seen many companies do exactly that, even those that have already paid for expensive test case management tools.

Spreadsheets can be pressed into service for documenting test scripts, but they need a lot of customisation and convention to be useful (cell formatting, document structure, reporting, stats, test assignment, file locations, master copies, bug tracking etc).

In this light, Testpad is the ‘buy’ alternative to the spreadsheet ‘make’ option. Testpad is like a ready-to-go spreadsheet specialised for the purpose of testing. However, if you then consider features like recording test results via mobile touchscreens, delegating tests to other users and guests, multiple report formats and easy linkage to bug trackers, Testpad exceeds what can be achieved with a spreadsheet.

Do you use Testpad to test your own software?

Absolutely!

Testpad uses a mixture of automated and manual testing. The automated tests are based on Selenium and are great for catching regressions in the javascript and server-side code.

Testpad’s manual testing uses a collection of Testpad scripts as detailed guides to steer exploratory testing. If it’s a small change with low risk, just the section headers are used as the guide for a quick smoke test on one browser. If it’s a significant change with higher risk, the whole script is run on each of the four main browsers. An earlier version of one the Testpad self-test scripts is included in the website’s fledgling example library.

Testpad’s mobile UI comes in very handy here. I can load up the test script on an iPad and use it to prompt for tests to run on the browser running on my laptop. The iPad is logged into Testpad’s own account and the laptop browser is logged into a scratch account on the system under test – imagine how annoying it would be to try to test Testpad if the tests kept deleting and editing themselves! So the mobile UI makes for a very convenient way to separate these contexts and keep me sane.

Visit TestPad online – www.ontestpad.com

This is a follow up to the previous article - Documentation and The Tester.

• Proof Read for Spelling and Grammar - Some may sneer at the notion of a technical writer making simple errors. That’s silly. The hard part is organizing information and presenting it cogently. Give me that, and I am more than happy to look for typos.
• Spelling & Grammar Trick –  If you are reviewing documentation in a tool or viewer or format which does not offer native spell and grammar checking, considering importing the text into a tool which does. MS Word, Open Office Writer, Google Docs, a browser add-on – even cat the text file into “spell” on Unix. Take advantage of the tools at hand. But you still have to read it and check the rest manually.
• Check for Accuracy – errors of commission. Keep the Development Spec and your test notes handy.
• Check for Completeness – errors of omission. Again, use the Dev Spec and test notes for references.
• Consider Organization and Approach – compare with the Marketing Req. Spec.
• Test all Examples! – Do you (or your company) want to look dumb? Include an example in the doc that doesn’t work. I speak from experience here. So don’t just read the examples. Perform the exact steps documented.
• Look for changes in one area that might affect another - And make sure that screenshots are up-to-date.

Author Profile – Sean Morley

Sean Morley is a Principle Quality Engineer for Dassault Systemes. Having worked in a variety of roles on software tools for electronic design and test, he is currently focused on the testing of engineering software. Sean participates in test related forums and blogs at www.testyengineer.com in a concerted effort to improve his knowledge of the field and share his experiences. He appreciates all feedback, recommendations, and challenges as opportunities for growth, and can be contacted at seansync@gmail.com.

I’m sure we’ve all had days where we need a bit of help coming up with test ideas.  Even more so if the area of technology we are dealing with is new.

One way to get a feel of where things can go wrong is to check StackOverflow.  There is so much to be learned from here. So many developers hang out there and ask lots of questions.  It will also give you a better understanding of what challenges developers face, the more you understand the more respect you can get as a tester

So if you are testing a website in HTML5, then perhaps doing a search on testing + html5 would be a good start.  Not only will you get real life problems, but also real life solutions.

Of course this can apply to any related forum to technology or your industry, Stackoverflow is just an example.

Got a testing tip? Why  not submit it and get published?

Author Profile – Rosie Sherry

Rosie is the founder of Software Testing Club and loves to run the day to day activities of the community. You can follow her on Twitter @rosiesherry and very occasionally on her personal website – www.rosiesherry.com

The Development Perspective

Consider that Doc and Test both generally work from a Development spec. This document is written from the developers perspective – how they will implement these features that marketing requested. The problem is that this document often shifts the perspective away from using a tool toward breaking down its features – the deconstructed tool, if you will. Both Doc and Test need these details. But good testing and good documentation should come from a usage perspective. Both groups need to figure out how this information, and this feature, fit in the context of using the software, so they know how to describe it, and the logical usage scenarios to exercise it. (A marketing requirements doc might be of help.)

Synergies

Where is the synergy? The tasks are often coincident, but depending on scheduling one group might be ahead of the other. Share what you’ve got! A test plan or a set of test scenarios might be a great boon to a writer’s understanding of usage flow. And early draft documentation can help get a tester on the right track.

Sharing

What else can be shared? Maybe one has spent some time with the developer, and can pass on the brain dump. Are there any special setup instructions? This might be especially important for a new tool or a preproduction system. How about test data, or data generation scripts?

Strategy Sessions

Perhaps a strategy session can prove useful. I recall discussions on whether a new mode to an existing feature should be described in the section with the other existing modes, or whether it merited its own separate topic. The conclusion affected whether we would add incremental tests, or entire new scenarios.

Technical Writers

Technical writers tend to be pretty hands on, so they may well be the first to run the new code outside of the developer. Not surprisingly, they frequently find bugs. And not just early failures. In my experience writers seem particularly adept at catching consistency and interaction issues – probably because they often have a high-level perspective, wondering how the new feature can fit into the existing documentation, or how to structure a new document.

I have had the good fortune to work with some excellent technical writers. And they have had some darn good testing skills. In fact, I know several who have done tours in both roles. It can be a good thing to broaden your horizons.

Author Profile – Sean Morley

Sean Morley is a Principle Quality Engineer for Dassault Systemes. Having worked in a variety of roles on software tools for electronic design and test, he is currently focused on the testing of engineering software. Sean participates in test related forums and blogs at www.testyengineer.com in a concerted effort to improve his knowledge of the field and share his experiences. He appreciates all feedback, recommendations, and challenges as opportunities for growth, and can be contacted at seansync@gmail.com.

As it turns out, we didn’t have to look far.

Say hello to Simon Knight, Editor of The Testing Planet.  We are yet to meet face to face, but that’s pretty normal with our virtual set up.  He has been a very active member of the Software Testing Club community recently.  Perhaps most notably for bringing the STC Meetups to Birmingham.

You can find him on Twitter. Or perhaps why not drop him an email (simon[at]softwaretestingclub.com) to say hello