Making the Switch to Distributed Version Control

Many individuals, teams, and organizations are thinking about making the switch to distributed version control systems a la Git and Mercurial (Hg). This is the first post in a series of blog entries over the next several weeks that focus on using and understanding DVCS.

Let’s start off with the basics and explore what version control is in general. In this entry, we will discuss problems that any version control aims to solve, where version control came from and some of the basic concepts you’ll need to know in order to use it.

A Simple Example

It’s often helpful to have a concrete example when talking about editing code, so let’s use a simple personal web page:

We’ll use this simple HTML page as an example throughout this entry.

Code Changes Often

The code we write as programmers changes often. Bugs need to be fixed, features need to be added, and content needs to be changed.

Most code is stored as plain old text files, and we change the code by editing these files. Every time we save our changes, we overwrite the old version of the file with a new one.

Unfortunately, no programmer is perfect, and sometimes, we make mistakes. If you make a change to a file, save it, compile it, and find out that something went wrong, it’s often helpful to be able to go back to the old version or to get a report of what we actually changed, in order to focus on what we may have done wrong.

Suppose in our example, our fictional character John wants to update his “Contact Information” header to read “John’s Contact Information”. He might edit the file so that that section reads:

John saves the file, reloads the page, and notices something doesn’t look quite right. How can John figure out the problem?

In this simple example, it’s fairly easy to simply read the entire file and find the problem, but it can obviously get much more difficult when you’re editing many parts of a large file that all interact with each other.

One of the earliest methods that is still around for comparing versions of files is a pair of utilities called “diff” and “patch”. Modern version control systems still use the concepts (and even file formats) of these tools, so let’s take a look at how they work.

Diff

Diff was originally created in the early 1970s. Its purpose is to take two versions of a file as input, and output a hunk of text that tells you how to change the first file into the second file.

There are many formats of diff in existence, but we’ll just work with the most common format which is know as a “unified diff”. If you’re following along at home on a Linux or OS X machine, you’ll need to pass the

option when you run diff to get the format.

If our example user John wants to use diffs to help him visualize his changes, he’ll need to do some extra work beforehand because he needs to keep the original version of his web page. When John wants to edit his web page, he would do the following steps:

• Make a backup copy of his page by copying his index.html file to index-old.html.
• Make his changes, save them, and preview them.
• Use diff to compare index-old.html to index.html.

When he runs the files through diff, he’ll get output that looks like this:

Notice that there are two lines displayed to represent the line he modified. The first has a ‘-’ before it, which means: “this line was deleted”. The second has a ‘+’ before it, which means: “this line was added.”

This example demonstrates an important point about diffs: even though John only changed a few parts of the line, diff treats the entire line as added and removed. Standard diffs only deal with entire lines.

Now that John can see exactly what he changed, it’s quite easy to see the problem – he’s changed the second level header to a first level header. He can fix the problem, save his changes, and delete the index-old.html file when he’s satisfied.

The ability to easily see what changed in a file is one of the biggest advantages of using diffs, but they can also be used to transform files with the patch utility.

Patch

Patch is a utility used to read hunks of text produced by diff and apply them to files, in order to transform the old files into the new versions.

Patch is often used to share changes with other people. For example, let’s say John asks his friend Mary for some advice on his web page. Mary looks over the code and changes it a bit to make the important sections stand out. She then runs diff to produce a hunk of text describing her changes, which looks like this:

She saves that text to a file and emails it to John.

When John receives the file, he could simply retype all of Mary’s changes and save the file, but that’s a lot of work and he might make a typo. Instead, he could use the patch utility to “apply” the output of Mary’s diff to his own copy of the file.

When he feeds his version of the file and Mary’s diff to the patch utility with patch index.html index-from-mary.patch, it modifies his copy by replaying the changes Mary made. The result is that his copy of the file now looks like Mary’s, and he can simply upload it to his web server without any more work.

Core Concepts of Diff and Patch

Diff’s purpose can be summed up as: “representing changes to files as hunks of text”. These hunks of text can be read by a human to determine what changed, and can also be saved as files and emailed to other people.

Patch’s purpose can be summed up as: “taking hunks of text produced by diff and applying them to the old versions of files in order to transform them into the new versions.”

These utilities give us an efficient way to share changes. If one line in a 10,000-line file changes, the diff for that file is only a few bytes. If we transferred the entire file instead, the file would be 10kb. When you work with multiple files, this difference can add up quickly.

Managing Versions of a File Without Version Control

A large disadvantage of the diff and patch utilities is that they work with two, and only two, versions of a file: “old” and “new”. Rare indeed is the programming project that only ever needs a single update in its lifetime.

It is often useful to see how a code file has evolved over time. To do that, we need to store “versions” of the file, so we can compare them later. One way to do this is to save many copies of a file with some numbering scheme, like this:

The disadvantages to this method are many:

• We need to save a full copy of the file even if only a single line changed.
• The numbering scheme becomes more complicated if we need to store two separate versions for the same date.
• Two people may edit the file on the same date.
• Many versions of the file are stored, which clutters our project folder.
• If our hard drive fails, the entire history of the file disappears.
• If we want to tell a coworker to “look at the changes between X and Y,” we need to send them those versions.

Managing Versions of a Group of Files Without Version Control

Another disadvantage of diff and patch is that they only work on a single file at a time.

In the real world, most programming projects consist of many files, which means we need to save copies of each file we change.

Worse still, changes to one file might affect other files. If we change a C header file, we almost certainly need to change the corresponding .c file, which means we need to make sure that those versions are kept together as a group.

Managing versions yourself by making copies of files quickly becomes a nightmare. Version control systems are programs designed to handle this for us, so we can stop worrying about copying files and get back to coding.

Centralized Version Control vs. Distributed Version Control

In our next entry we will discuss the advantages and disadvantages of centralized version control. Thinking about moving from Subversion to Git? Want to learn about the key concepts of DVCS? Stay tuned for the rest of our Switch To DVCS blog series.

This guest blog post is part of an Atlassian blog series raising awareness about testing innovation within the QA community. You can find the other posts in this series under the QA Innovation tag. 

This is a guest blog post by Paul Gerrard, a Principal of Gerrard Consulting Limited and is the host of the UK Test Management Forum. It is Part 2 of a two-part blog series on the future of QA testing.

In Part I of this article on the Redistribution of Testing, I suggested there were four forces that were pushing testers out of the door of software projects (and into the real world, perhaps). In this post, I want to highlight the industry changes that seem to be on the way, that impact on development and delivery and hence on testing and testers. After the negative push, here’s the pull. These changes offer new opportunities and improve testers’ prospects.

Recent reports (IBM’s ‘The Essential CIO’ 2011 study and Forrester’s ‘Top 10 Technology Trends to Watch’) put Business Intelligence, adoption of cloud platforms and mobile computing as the top three areas for change and increased business value (whatever that means).

Once more, the industry is in upheaval and is set for a period of dramatic change. I will focus on adoption of the cloud for platforms in general and for Software as a Service (SaaS) in particular and the stampede towards mobile computing. I’m going to talk about internet- (not just web-) based systems rather than high integrity or embedded systems, of course.

The Industry Changes its Mind – Again

The obvious reason for moving to the cloud for Infrastructure as a Service (IaaS) and regardless of the subtleties of capex v opex costs, the cost advantage of moving to cloud-based platforms is clear. “Some of this advantage is due to purchasing power through volume, some through more efficient management practices, and, dare one say it, because these businesses are managed as profitable enterprises with a strong attention to cost.” So, it looks like it’s going to happen.

Moving towards IaaS will save some money. The IT Director can glory in the permanent cost savings for a year – and then what? Business will want to take advantage of the flexibility that the move to the cloud offers.

The drift from desktop to laptop to mobile devices gathers pace. Mobile devices coupled with cloud-based services have been called the ‘App Internet’. It seems that many websites will cease to be and might be replaced by dedicated low-cost or free Apps that provide simple user interfaces. New businesses with new business models focusing on mobile are springing up all the time. These businesses are agile by nature and Agile by method. The pull of the App internet and Agile approaches is irresistible.

The Move to SaaS and Mobile (App) Internet

I’m not the biggest fan of blue sky forecasters, and I’m never entirely sure how they build their forecasts with an accuracy of more than one significant digit, but according to Forrester’s report Sizing the Cloud, the market for SaaS will grow from $21bn in 2011 to $93bn in 2016 and represent 26% of all packaged software.

Now 26% of all packaged software doesn’t sound so dramatic, but wait a minute. To re-architect an installed base of software and create new applications from scratch to make that percentage will be a monumental effort. A lot of this software will be used by corporates who have systems spanning the (probably private) cloud and legacy systems and the challenges of integration, security, performance and reliability will be daunting.

The Impact on Development, Delivery and Testing

Much of the software development activity in the next five years or so will be driven by the need for system users and service vendors to move to new business models based on new architectures. One reason SaaS is attractive to software vendors is that the marketing channel and the service channel are virtually same and the route to market is so simple that tiny boutique software shops can compete on the same playing field as the huge ISVs. The ISVs need to move pretty darned quick not to be left with expensive, inflexible, unmarketable on-premise products so are scrambling to make their products cloud-ready. Expect there to be some consolidation in some market sectors.

SaaS works as an enabler for very rapid deployment of new functionality and deployment onto a range of devices. A bright idea in marketing being deployed as new functionality in the afternoon seems to be feasible and some companies seem to be succeeding with ‘continuous delivery’. This is the promise of SaaS.

Many small companies (and switched-on business units in large companies) have worked with continuous delivery for years however. The emergence of the cloud and SaaS and of maturing Agile, specification by example, continuous integration, automated testing and continuous delivery methods means that many more companies can take this approach.

What Does this Mean for Software Practitioners?

Businesses like Amazon and Google and the like have operated a continuous delivery model for years. The ‘Testing is Dead’ meme can be traced to an Alberto Savoia talk at Google’s GTAC conference. Developers who test (with tools) ship code to thousands of internal users who ‘test’ and then the software goes live (as a Beta, often). Some products take off; some, like Wave, don’t. The focus of Alberto’s talk is that software development and testing is often about testing ideas in the market.

Google may have a unique approach, I don’t know. But most organisations will have to come to terms with the new architectures and a more streamlined approach to development. The push and pull of these forces are forcing a rethink of how software available through the internet is created, delivered and managed. The impacts on testing are significant. Perhaps testing and the role of testers can at last can mature to what they should be?

Some Predictions

Well, after the whirlwind tour of what hot and what’s not in the testing game, what exactly is going to happen? People like predictions so I’ve consulted my magic bones and here are my ten predictions for the next five years. As predictions go, some are quite dramatic. But in some companies in some contexts, these predictions will come true. I’m just offering some food for through.

Our vision, captured in our Pocketbook is that requirements will be captured as epic stories, and implementable stories will example and test those requirements to become ‘trusted’, with a behaviour-driven development approach and an emphasis on fully and always automated checking. It seems to us that this approach could span (and satisfy) the purist Agilists but allow many more companies used to structured approaches to adopt Agile methods whilst satisfying their need to have up-front requirements.

Here are my predictions:

1. 50% of in-house testers will be reassigned, possibly let go. The industry is over staffed by unqualified testers using unsystematic, manual methods. Lay them off and/or replace them with cheaper resource is an easy call for a CIO to make.
2. Business test planning will become part of up-front analysis. It seems obvious, but why for all these years has one team captured requirements and another team planned the test to demonstrate they are met. Make one (possibly hybrid) group responsible.
3. Specification by example will become the new buzzword on people’s CV. For no other reason that SBE incorporates so many buzzwordy Agile practices – Test-First, Test-Driven, Behaviour-Driven, Acceptance-Test Driven, Story-Testing, Agile Acceptance testing) – it will be attractive to employers and practitioners. With care, it might actually work too.
4. Developers will adopt behaviour-driven-development and new tools. The promise of test code being automatically generated and executed compared to writing one’s own tests is so attractive to developers they’ll try it – and like it. Who writes the tests though?
5. Some system tests and most acceptance tests will be business-model driven. If Business Stories, with scenarios to example the functionality, supported by models of user workflows are created by business analysts, those stories can drive both developer tests and end to end system and acceptance tests. So why not?
6. Business models plus stories will increasingly become ‘contractual’. For too long, suppliers have used the wriggle-room of sloppy requirements to excuse their poor performance and high charges for late, inevitable changes to specification. Customers will write more focused compact requirements, validated and illustrated with concrete examples to improve the target and reduce the room for error. Contract plus requirements plus stories and examples will provide the ‘trusted specification’.
7. System tests will be generated from stories or be outsourced. Business story scenarios provide the basic blocks for system test cases. Test detailing to create automated or manual test procedures is a mechanical activity that can be outsourced.
8. Manual scripted system test execution will be outsourced (in the cloud). The cloud is here. Testers are everywhere. At some point, customers will lose their inhibition and take advantage of the cloud+crowd. So, plain old scripted functional testers are under threat. What about those folk who focus more on exploratory testing? Well, I think they are under threat too. If most exploration is done in the cloud, then why not give some testing to the crowd too?
9. 50% of acceptance tests will be automated in a CI environment for all time. Acceptance moves from a cumbersome, large manual test at the end to a front-end requirements validation exercise with stories plus automated execution of those stories. Some manual tests, overseen by business analysts will always remain.
10. Tools that manage requirements, stories, workflows, prototyping, behaviour-driven development, system and acceptance testing emerge.

Where do testers fit? You will have to pick your way through the changes above to find your niche. Needless to say you will need more than basic ‘certification level’ skills. Expect to move towards a specialism or be reassigned and/or outsourced. Business analysis, test automation, test assurance, non-functional testing or test leadership beckon.

Whither the Test Manager?

You are test manager or a test lead now. Where will you be in five years? In six months? It seems to me there are five broad choices for you to take (other than getting out of testing and IT altogether).

1. Providing testing and assurance skills to business: moving up the food chain towards your stakeholders, your role could be to provide advice to business leaders wishing to take control of their IT projects. As an independent agent, you understand business concerns and communicate them to projects. You advise and cajole project leadership, review their performance and achievement and interpret outputs and advise your stakeholders.
2. Managing Requirements knowledge: In this role, you take control of the knowledge required to define and build systems. Your critical skills demand clarity and precision in requirements and the examples that illustrate features in use. You help business and developers to decide when requirements can be trusted to the degree that software can reasonably be built and tested. You manage the requirements and glossary and dictionary of usage of business concepts and data items. You provide a business impact analysis service.
3. Testmaster – Providing an assurance function to teams, projects and stakeholders: A similar role to 1 above – but for more Agile-oriented environments. You are a specialist test and assurance practitioner that keeps Agile projects honest. You work closely with on-site customers and product owners. You help projects to recognise and react to risk, coach and mentor the team and manage their testing activities and maybe do some testing yourself.
4. Managing the information flow to/from the CI process: in a Specification by Example environment, if requirements are validated with business stories and these stories are used directly to generate automated tests which are run on a CI environment, the information flows between analysts, developers, testers and the CI system is critical. You define and oversee the processes used to manage the information flow between these key groups and the CI system that provides the control mechanism for change, testing and delivery.
5. Managing outsourced/offshore teams: In this case, you relinquish your onsite test team and manage the transfer of work to an outsourced or offshore supplier. You are expert in the management of information flow to/from your software and testing suppliers. You manage the relationship with the outsourced test team, monitor their performance and assure the outputs and analyses from them.

Moving Forward

The recent history and the current state of the testing business, the pressures that drive the testers out of testing and the pull of testing into development and analysis will force a dramatic re-distribution of test activity in some or perhaps most organisations.

But don’t forget, these pressures on testing and predictions are generalisations based on personal experiences and views. Consider these ideas and think about them – your job might depend on it. Use them at your own risk.

Paul Gerrard is a consultant, teacher, author, webmaster, programmer, tester, conference speaker, rowing coach and most recently a publisher. He has conducted consulting assignments in all aspects of software testing and quality assurance, specialising in test assurance. He has presented keynote talks and tutorials at testing conferences across Europe, the USA, Australia, South Africa and occasionally won awards for them. Find him on Twitter at @paul_gerrard or his site, Gerrard Consulting.

This guest blog post is part of an Atlassian blog series raising awareness about testing innovation within the QA community. You can find the other posts in this series under the QA Innovation tag.

The post is written by Lanette Creamer, an Agile testing consultant involved in software testing since 1999.

Testers are prone to argue about anything. Just last night, at the Selenium Meetup, a fellow tester, Gary Masnica said, “There is nothing worse than listening to two testers argue. Before long, it will even get down to semantics, and I’m reminded of the Bill Clinton trial as each tester ponders the meaning of the word ‘the’ or something equally as silly.” Considering the small size of the testing community in relation to the number of developers, what gives? Are these testers just a cranky and cantankerous lot of malcontents? If so, surely not 4 out of 5 of them? Well, let’s first say that the title is a joke, because I’ve done no research, so those Dataphiles among you, please don’t go looking for the pie chart and research attached. Secondly, there IS something to it.

Reasons for Disparity

Critical Thinking

First, to be effective as a software testers, you develop a different kind of critical thinking in order to find the exceptions to a given rule. It can be said that if you hear hoofbeats, think horses, not zebras. Testers know you’ve considered horses, but can you handle a herd of zebras? This is one reason why our arguments may sound strange to you, but when your code is on Safari, will you be less sorry?

Varying Cultural Norms

Secondly, testing is a HUGE profession than spans the entire globe. The education for testers is far from consistent, and there are these much debated “schools of thought” that some subscribe to. On top of that, we have testers who work in different stages of Agile transitions across the industry now. Call someone a “resource” in some company and you’d be better off to have cursed at them. Let’s just say that cultural norms vary. Even this combined with cultural differences across the globe don’t explain all that is going on.

Making the Argument

The third reason why testers often do not agree, even on things that seem simple to other people, is that testers need to convince other people in order to see changes happen. For many of us, the number of bugs we FIND isn’t important. It is only the number of bugs important enough to be fixed or that we prevented that adds value to the end user. We don’t intend to argue for bugs being fixed or prevented unless they ARE important, but when that bug is important, a good tester will be as convincing as a pitbull lawyer. There will be research, evidence, and convincing testimony from bystanders. By the time you’ve reached many years experience as a professional software tester, you are as adept at explaining risk and sharing information with others as a good insurance agent, and that is a good thing, unless you happen to be debating against one.

Don’t Take It Personally

The fourth reason I have for you to consider is that often debates between testers aren’t personal. As we work in different areas of software testing, what sounds like disagreement and discord may just be two people who like to verbally spar and consider alternatives. We’re nerds, generally, just like other computer people. Star Wars vs Star Trek and Pirates vs Ninjas still can become religious issues. I’ve recently learned if I ever want to be the Crazy Cat Lady of Software I’ve got a loooong way to go. Did you know that Sandra Lerner of Cisco Fame has 9 cats and just bought the first feline knee replacement for one of them? That is awesome! She’s also into hosting full costume ball dances at her mansion.

 QA Drama

Now, for reason 5 out of 5, consider that maybe 4/5 testers do agree, but we never see them. The ones who don’t agree are just very loud and active. Also, drama draws focus, especially on the internet. Sometimes testers do feel the need to speak up, and indeed it is a key responsibility to point out flaws and possible big risks. To let important issues quietly go by wouldn’t be a helpful trait as a tester, but consider that all of the times we quietly agree, you are less likely to hear it. The memehappy testers at Mozilla appreciated my many cat photos, but the same photos with a different audience led to accusations that I’m making testers look unprofessional. Just like the word “resource”, even personal style can be considered a risk to testers.  The level of risk that is normal at a web based company would be considered irresponsible in some regulated environments. Testing will be far different when working on medical equipment than when working on a free video game, and rightly so. While 4/5 testers may simply say, “I work in a different part of testing than you do,” you are likely to hear and remember the few who disagree loudly in public.

Where do we go from here?

While disagreement between testers has been the status quo for so long that it is still noted today, cooperation and teamwork are broadening testing in new ways.

As automation becomes more common, developers and testers find that their skills are merging. They may have different areas of focus, but pairing between testers and developers is one of the faster growing areas in the industry, especially on teams adopting agile practices. As developers learn to do more testing, such as unit testing, and even TDD, the testers are delving further into tool assisted exploratory testing, using innovations in visual display of data to guide them to changes and areas of possible bug regression based on changes.

Testers are participating in creation of automation, early code reviews, and also in testing requirements before the software is even created, basically preventing bugs before they can be written. In addition, due to advances in Developer technology, testers can test branches in Jenkins or Hudson continous integration servers from anywhere! This makes a new type of build, where checking in to the main branch may be gated by a person AND automated acceptance tests and multiple levels, giving teams control over how many tests a build needs to pass before going out to a client, or the public vs. how good is good enough for internal testing.

This can also be done faster than ever before. Free tools such as TestFlight allow developers from all over the world to specify and release to just the beta testers they trust, giving them control over in the wild testing, where paid services, such as uTest can literally provide an army of testers in a certain location at the time named by the team! We are only starting to see which combinations are the most useful in which situations, but one thing is certain, the need for talented testers isn’t going away. It is diversifying, growing, and many of the skills needed may have aspects in common with that of good developers, but the mindset is slightly different.

Most testers now feel that they are part of the development team, where in the past testers were more siloed. As this change has happened, testers still seek out advice and help from other testers. Strongly identifying with a team is positive, as is shared responsibility for quality. There may not be a testing “team” in your company, but the testing community is growing, and now, more than ever, it includes more enlightened developers who need to test their own code, as well as the team members who take the testing of the entire product, beyond the code, further than they have been able to do before as tools enable earlier testing.

 Lanette Creamer a.k.a. Testy Redhead is a consulting software tester who is passionate about collaborative testing and is a practicing student of the context driven school. Lanette is a frequent industry speaker in the software testing community, and also enjoys agile and lean peer conferences. She has presented at StarEast, CAST, STPCon, PNSQC, and as a guest speaker at Mozilla in the past year. She also writes articles occasionally in addition to her blog, Testy Redhead. Find her on Twitter at @lanettecream.

REST APIs provide for an easy and lightweight way to integrate with other apps using basic HTTP and JSON libraries. However, some developers might find it more useful to work on a client library that has a tighter interface to these REST APIs. Our friends at Trineo decided to build a RubyGem that does just that.

Trineo’s jira-ruby gem was built just for JIRA 5’s REST API. It abstracts much of the boilerplate code that every developer has to deal with including OAuth. Here’s a sample of how to use the gem:

It’s a great gem for working with JIRA 5. The best part is it’s open source, so feel free to fork it and send Trineo some new features.

It’s written by Richard Hale, a Senior Consultant & Director of Quality Assurance with Go2Group. As an Atlassian Platinum Expert, Go2Group provides its clients with world-class ALM and Testing services, with expertise in integration solutions for Atlassian, HP, Perforce, MuleSoft, IBM, and SalesForce.com. 

As a Director of Quality Assurance, I’ve been involved in many different software development environments and cultures. Many have been good and productive and some have been dysfunctional, for lack of a better word. Penny Wyatt touches briefly on the importance of fostering a team culture that encourages testers to investigate and report bugs in her post as well. In my experience, I’ve seen a team or certain sub-groups within the team reticent to report too many defects, or certain types of defects because they are sensitive topically, or hot points in the project, or “managing down” the defect count because of management sensitivity to the total defect count.

This is done for various reasons – usually for a perception management approach so that upper managers, who may not understand the value of a found defect, do not get a bad impression of the progress of their project. The practice, as I’ve observed it, is never a formality but always a covert attempt at re-working the total defect count (and sometimes other numbers) along the project’s procession.  At times, this can also often be at the behest of a middle manager who has a desire to manage the perception of his/her uppers. Sometimes this is at the request of a developer.

What I’d like to accomplish in this blog entry, is explore some of the examples I’ve seen, how they came about, and their real value for the project. These are just some case studies that we can hopefully learn a little from.

Typical scenarios I’ve observed are:

• The Lame Bug
• “It’s Not a Requirement”
• Developer is already fixing that – e.g. The Dev “knows about it”

 “The Lame Bug”

Sometimes QA is faced with complex environment, data or configuration issues that cause what seem to be defects. In this case, the ‘defect’ may not be in the code or functionality itself, but due to these issues, the result of test cases can be failures.

For example, say the tester was expecting to see certain data samples as part of their test environment, which includes pricing for a certain region, but their test cases fail because the proper pricing data is not present. It may be the responsibility of the Data or Development Team to ensure environments are setup properly. The failure of the test cases impact the timeline of testing, and waste QA time until the situation is resolved.

When they encounter this situation and enter defects, development teams may say, “Don’t enter that as a defect because it’s not a bug” – or that it’s a ‘lame bug’. However, this is a missed opportunity, if not entered and tracked, to correct a problem in the configuration management (CM) process or environment & data setup process, application deployment methodology, etc.  The defect also captures the project effort and resources used to resolve the issues and can provide a valuable benchmark to ensure the issues do not arise again.

QA teams should be diligent about entering these types of defects, properly categorizing them, and working with the teams involved to pinpoint the root cause. Once the cause is identified, it could also change the way the defect is described & characterized with attributes so that is an appropriate reference artifact. This will help the project’s key processes – those which actually support the life cycle– continually improve throughout the software development life cycle (SDLC).  Again, we capture the work involved for all teams in resolving these issues.

“It’s Not a Requirement”

This is a case where the requirement artifact(s) should have a defect entered against them.

The tester finds what seems to be a defect, although it’s not necessarily a test case failure. Sometimes this is found through ad-hoc style testing. When this happens, the tester must check with requirements team to confirm whether the issue they encountered is a gap in the requirement. What may normally happen here is that the requirement is quickly updated to include the intended functionality, and development and test teams continue with addressing this new requirement.

However, at this point, a better way of handling this situation would be entering the defect against the requirement artifact with the missing requirement, and assigning the Requirement Team resource. With proper issue types, this practice can help to track scenarios where testing uncovers necessary functionalities not originally included in the requirements documentation and capture this effort on the part of QA teams when it occurs. Otherwise, these incidents can be a hidden cost of the project’s development.

“The Developer is Already Fixing That”

Say QA is testing per requirements docs or test cases and the tester finds an issue in functionality: either it doesn’t exactly line up with requirements or there are unclear requirements surrounding a particular functionality, showing a gap in the documentation or expectations of the actual functionality of the product. The tester is fairly certain about the issue, but tries to confirm with developer. Then, the developer states, “Oh yes, I found this, it’s already fixed and will be in your next build.”

The result is that the tester never enters the bug, and the requirement documentation doesn’t get corrected. From there, a test case may not be specifically written for this feature and that’s the end of the conversation.

The problem here is that if QA does not document the defect properly and this type of scenario begins to happen with greater frequency, the project metrics and measure of progress will be skewed. As QA often spends effort interacting with the requirements team, business teams, and developers, the true value of effort is lost in this process is not accounted for. It’s always best to just enter the defect, and make sure to account for the new requirement. In this case, it may be necessary to enter a defect against the requirements artifact as well, so that any effort required in that activity can be tracked. This can help management isolate the issue of incomplete requirements if that problem becomes more frequent.

Enter the Defect . . . with conviction!

In summary, entering the defect is not a ‘bad’ thing.  Defects or issues – with proper typing across the various sorts of incidents – are a necessity to the entire project in:

1. Helping improve processes and fostering continual improvement in team and individual disciplines
2. Capturing and quantifying the real cost of the project (e.g. identifying potential hidden costs)
3. Helping to isolate the root cause of certain project support issues, and resolving them (e.g. CM, environment, data)
4. Supporting more helpful project metrics

The extremely important tasks of quantifying our work, tracking our progress, and continuous improvement are dependent upon the proper and good practice of entering issues.

Rich Hale has been involved in the software development industry for over 15 years.  He’s worked as a Developer, QA Analyst, Test Automation Architect and SDLC Tools & Process Specialist. 

What are you waiting for? Register today… and if you’re coming, make sure to add yourself to our Lanyrd page.

Atlassian’s plugin framework is a powerful tool for developers looking to build on top of our products. It allows you to apply one of our core values, “be the change you seek,” directly to our software. However, the power this framework provides is often challenging to harness without proper guidance.

Matt Doar is a great candidate for providing that guidance. He has worked on Atlassian products and have developed JIRA plugins for well over five years. Now he is the “Chief Toolsmith” at CustomWare.

First Impressions

Matt Doar’s “Practical JIRA Plugins” provides new plugin developers guidance without the downside of feeling overwhelmed. He introduces the framework in a way that allows developers to get immersed in the most common type of plugins: the introduction of custom field types.

Practical JIRA Plugins starts by guiding the user through an overview of Atlassian’s Plugin SDK (the command line tool, generated files, what plugins can do, etc.). It’s a clear and concise introduction for those who haven’t used the framework before.

After the overview, Matt guides the reader in building a plugin that introduces a custom field type to JIRA. Through this plugin, the reader is introduced to Velocity templates, logging, the configuration system, WebWork actions, searchers, workflow, and data storage.

After this exercise, Matt introduces the reader to the process of publishing a plugin on the Atlassian Plugin Exchange (plugins.atlassian.com). He also touches on how to handle upgrading a plugin for newer versions of JIRA.

Summary

Practical JIRA Plugins is a great introduction to JIRA’s plugin framework. I’d recommend it to anyone who’s just starting out with the Atlassian plugin framework on JIRA. The current version of the book targets JIRA 4.2.4.

Book Details

Practical JIRA Plugins, by Matthew B. Doar, published by O’Reilly Media in July 2011. ISBN 978-1-449-30827-8. You can get it at Amazon.com.

Congratulations and great job, Matt!

This guest blog post is part of an Atlassian blog series raising awareness about testing innovation within the QA community. You can find the other posts in this series under the QA Innovation tag.

The post is written by Paul Gerrard, a Principal of Gerrard Consulting Limited and is the host of the UK Test Management Forum. It is Part 1 of a two-part blog series on the future of QA testing.

Recently, there has been a spate of predictions of doom and gloom in our business.  Conference talks have had titles such as ‘Test is Dead’ and ‘Death to the Testing Phase’. ‘Testing has contributed little to quality improvement in the last ten years’, and even being a tester is a ‘bad thing’ – are all keynote themes that circulated at conferences, blogs and YouTube in late 2011.

My own company has been predicting the demise of the ‘plain old functional tester’ for years and we’ve predicted both good and bad outcomes of the technology and economic change that is going on right now. In July, I posted a blog, ‘Testing is in a Mess’, where I suggested that there’s complacency, self-delusion and over capacity in the testing business; there is too little agreement about what testing is, what it’s for or how it should be done.

There are some significant forces at play in the IT industry and I think the testing community, at least the testers in what might be called the more developed ‘testing nations’ will be coming under extreme pressure.

The Forces and Factors that will Squeeze Testers

The growth of testing as a career

Over the last twenty years or so there has been a dramatic growth in the number of people who test and call themselves testers and test managers. When I started in the testing business in 1992 in the UK, few people called themselves a tester, let alone thought of themselves as having a career in testing. Now, there are tens of thousands in the UK alone. Twenty years ago, there were perhaps five companies offering testing services. There must be ten times that number now and there are hundreds, perhaps thousands of freelancers who specialise and make a living in testing. Beyond this of course, the large system integration and outsourcing firms have significant testing practices with hundreds or even thousands of staff, many offshored of course.

It’s not that more testing happens. I think it’s because the people who do it are now recruited into teams, having managers who plan, resource and control sizable budgets in software projects to perform project test stages. Many ‘career testers’ have never done anything else.

Lack of advance in the discipline

The sources and sheer volume of testing knowledge have exploded. There are countless papers, articles, blogs and books available now, and there are many conferences, forums, meet-ups and training courses available too. But, even though the volume of information is huge, most of it is not new. As a frequent conference goer over 20 years, it depresses me that the innovation one sees in conferences, for example,  tends to be focused on the testers’ struggle to keep pace with and test new technologies rather than insights and inventions that move the tester’s discipline forward.

Nowadays, much more attention is paid to the management of testing, testers and stakeholders expectations and decision-making. But if you consider the argument that perhaps test management is a non-discipline, that is  there is no such thing as test management, there’s just management, and you take the management away – what’s left? Mostly challenges in test logistics – or just logistics – another management discipline?

Advances(?) in Automation

What about the fantastic advances in automation? Let’s look at the two biggest types ot test automation.

Test execution robots are still, well, just robots. The advances in these have traced the increased complexity of the products used to build and deliver functionality. From green-screen to client/server to GUI to Web, to SOA, the test automation engineer of 1970 (once they got over the shock of reincarnation) would quickly recognise the patterns of test automation used today. Of course, the automation frameworks are helping to make test automation somewhat more productive, but one could argue that people have been building their own custom frameworks for years and years and they should have been mainstream many years ago.

The test management tools that are out there are fantastic. Integrated test case management, scheduling, logging and incident management and reporting. Except that the fundamental purpose of these tools is basic record-keeping and collaboration. Big deal. The number of companies who continue to use Excel as their prime test management tools shows just how limited the test management tools are in what they do. Most organisations get away without test management products altogether because these products support the clerical test activities and logistics, but do little or nothing to support the intellectual effort of testers.

The Emergence/Dominance of Certification

The test certification schemes have gone global it seems. Dorothy Graham and I had an idea for a ‘Foundation’ certification in 1997 and we presented a one page syllabus proposal to an ad-hoc meeting at the Star WEST conference in San Jose to gauge interest. There wasn’t much. So we came back to the UK, engaged with ISEB (not part of BCS in those days) and I became the founding Chair of the initial ISEB Testing Board. About ten or so UK folk kicked off the development of the Foundation scheme which had its first outing in late 1998.

As Dorothy says on her blog, the Foundation met its main objective of “removing the bottom layer of ignorance” about software testing. Fourteen years and 150,000 certificate awards later, it does the same. Except that for many people it’s all they need (and may ever need) to get a job in the industry.

The Agile Juggernaut

Agile is here to stay. Increasingly, developers seem to take test, Test-Driven and Behaviour-Driven Development and Specification by Example more seriously. Continuous Integration and Delivery is the heartbeat, test, life-support and early warning system. The demands for better testing in development are being met. A growing number of developers have known no other way.

It seems likely that if this trend continues, we’ll get better, stable software sooner and much of the checking done late by system testers will not be required. Will this reduce the need for system testers? You bet.

Some Agile projects don’t use testers – the testers perform a ‘test assurance’ role instead. The demand for unskilled testers reduces and the need for a smaller number of smarter testers with an involvement spread over multiple projects increases.

What is the Squeeze?

The forces above are squeezing testers from the ‘low-value’ unskilled, downstream role to upstream, business-savvy, workflow-oriented, UX (user experience)-aware testing specialists with new tools. Developers are absorbing a lot of checking that is automated. Some business analysts are taking their chance and absorbing test disciplines into analysis and are taking over the acceptance process.

If a 3 day certification is all you need to be a professional tester, no wonder employers think testing is a commodity, so will outsource it when they can.

Stakeholders know that avoiding defects is better than finding them. Old-style testing is effective but happens at the end. Stakeholders will say, “Let’s take requirements more seriously; force developers to test and outsource the paperwork”.

Smart testers need to understand they are in the information business, that testing is being re-distributed in projects and if they are not alert, agile even, they will be squeezed out. Needless to say, the under-skilled testers, relying on clerical skills to get by will be squeezed out.

A Methodological Shift

There seems to be a methodological shift from staged, structured projects to iterative and Agile and now, towards ‘continuous delivery’. Just as companies seem to be coming to terms with Agile – it’s all going to change again. They are now being invited to consider continuous ‘Specification by Example’ approaches. Specification by example promotes a continual process of specification, exampling, test-first, and continuous integration.

But where does the tester fit in this environment?

I’ll make some suggestions on what might happen and what you might do about it in the second part of this article.

Paul Gerrard is a consultant, teacher, author, webmaster, programmer, tester, conference speaker, rowing coach and most recently a publisher. He has conducted consulting assignments in all aspects of software testing and quality assurance, specialising in test assurance. He has presented keynote talks and tutorials at testing conferences across Europe, the USA, Australia, South Africa and occasionally won awards for them. Find him on Twitter at @paul_gerrard or his site, Gerrard Consulting.

This guest blog post is part of an Atlassian blog series raising awareness about testing innovation within the QA community. You can find the other posts in this series under the QA Innovation tag.

The post is written by Catherine Powell, a principal at Abakas, a software consulting company. At Abakas, she provides engineering management, development, testing, and process consulting services. 

Ahh, the glories of software creation. We have an idea, and an idea turns into a list, and a list turns into code plus more lists, which turns into more code and new machines in production ready to share our great new software with the world. And there are a few caffeine-fueled late nights, and there are a few company-sponsored lunches and demos. And then we’re ready… just as soon as we run one last regression test cycle. And it’s going to take way too long. We’re all excited and ready to go, and we have to stop to wait for test. It’s the last thing to finish, and there are never enough testers during those end crunch times, and we keep having to cut corners to make releases happen on time. Plus, it frequently points out that we made the same mistakes we always make. So then we have to patch things and fix configurations and tweak it. That darn testing!

So how do we fix it?

How do we keep testing from being a bottleneck?

Simple. We do less of it. We minimize that end-of-cycle testing holdup. And we do THAT by building trust in our development and in our software creation, deployment, and maintenance process. Test cycles, particularly regression test cycles, are ultimately driven by fear. We do a “test pass at the end” because we do not trust that the activities that lead up to this point have resulted in robust, deployable software that does what we meant for it to do. Fix that lack of trust, and we can skip the extended test cycle.

There’s an old saying about starting off on the right foot. We don’t always have the luxury of starting (or restarting), but we can find the right foot, even on an ongoing project. To improve our testing, to make it more efficient, we have to address the whole software development life cycle. We must find our underlying fear and fix it instead of hiding behind an ever-larger test cycle.

The tricky thing about the fear driving the “test pass at the end” is that it’s frequently justified. We keep finding bugs, or configuration errors, or data problems, or deployment failures. We can’t stop testing when it keeps finding things that we really do need to know! Step one to reducing this test phase is finding the true underlying problems causing our fear, and fixing those. In other words, to make testing more efficient, we have to fix everything else.

The Underlying Problems

Let’s look at some of the more common problems that cause fear, and what we can do about it:

• We fear that our deployment didn’t work or wasn’t complete, so we test. Instead, we can automate deployment so code gets into QA the same way it gets into production. That way we only have to test in one environment (QA), not two.
• We fear that a developer forgot to check something in or didn’t make the build properly, so we test. Instead, we can set up a build system and only test – or deploy – from there. Again, it means we only test a build once, not once per environment.
• We fear that a new feature accidentally broke the old way things worked, and that customers want the old way and the new way to both work, so we test. Instead, we can create a regression test suite and automate it, so the build system can run it every time we check in code. There’s no need to wait until the end to find out that something regressed.
• We fear that some underlying change – a browser version or OS patch – broke our software, so we test. Instead, we can schedule that testing at a different point in the cycle. Tie it to the release of the software we’re worried about, not to our own release schedule. Let our regression suite (see above) help us find major problems, watch the release patterns of the software with which we interact, and don’t test just because something might have changed. Reserve this for when something actually did change.
• We fear that several features coming together at the very end will break each other, so we test. This one we still need to test! We can – and should – try to bring in features serially over time, but if the development team is larger than about two people, there’s going to be a bit of a rush trying to get things in before some deadline, whether that deadline is a sprint, a release, or just Friday. Minimize it where possible, and then move everything else out of the way so only this much smaller task is left.

Test smarter, not harder

Fear causes us to take on large test cycles near the end of a release (or sprint or story or whatever the “chunks” are in a given project). We perform slow, repetitive tasks with a relatively low value, just because of that fear. Eliminate the source of the fear, automate the repetitive and low value tasks, and make that end testing cycle less of a bottleneck.  Get the software on the right foot, and you’ll free testers to do the activities that not only assuage fear, but that truly provide value.

Catherine Powell has worked with a broad range of software, including an enterprise storage system, a web-based healthcare system, data synchronization applications on mobile devices, and webapps of various flavors. She is an author, speaker and a mentor to engineers and technical managers. Catherine focuses primarily on the realities of shipping software in small and mid-size companies. Catherine’s published works, blog, and contact information can be found at www.abakas.com.

At Atlassian, we work hard to ship great products to our amazing customers. Another thing we like to do is set aside some time to let our brains run wild. Every last employee has some idea in the back of their mind about a product improvement and we think it’s critical to give them time to make those ideas a reality. FedEx day is a day, once a quarter, where we all set aside our day to day and run a 24 hour gauntlet to see what we can deliver in that time frame.

It’s also a great kick in the pants to finally build that thing you’ve been saying would make customers lives easier if it could just land on the backlog. We encourage people to form teams at the end get together, demo the final output and vote on a winner. The chance at glory gives you the push you’ve needed to finally prototype that idea that’s been rolling around.

For a bit more information on Atlassian FedEx days take a look on the About Us section of Atlassian.com.

Winning FedEx

Clear Customer Value

The most important part of a FedEx day project that has aspirations of winning is to create something that shows clear customer value. Atlassian is full of engineers, so we value engineering quality and beautiful code but at the end of the day we’re all about customers. So while we enjoy a presentation on this cool new thing a computer can do, if we can’t figure out how it’s going to make a customers life easier it’s probably not going to win FedEx.

Our most recent FedEx day was October 27/28 and several great things came out of it. The winner this time around was a Confluence Plugin called Autoconvert and we think it’s going to make quickly creating great content in Confluence even easier.

Autoconvert solves the problem of sharing links to other content within a Confluence page. Right now you might be thinking to yourself that there is no problem with linking to content. Creating a link is simple in Confluence. If you just want to share a plain old link, you can copy and paste it into the editor and autoformat will take care of making it into a clickable link and if you want to share a link to content within another Atlassian product you can even use macros so that the content is intelligently linked too, providing hover information about what’s behind the link.

The Autoconvert team decided that this way of creating links was broken. You shouldn’t have to know which button to press, or which macro to use when you want to create these `intelligent links` Confluence should know what content it can link to intelligently and just do the work for you.

That’s exactly what Autoconvert does. It enables Confluence to make decisions about the links you paste in. If it understands what you’re linking to, it will transform the link into the best possible experience for a user reading the page. If you paste a link to a YouTube video, the link will get transformed into an embedded player to show the video at that location. If the link goes to another Confluence page, the link text changes to the title of that page and will update should the page title ever change in the future.

Watch the Autoconvert demo video below to get a better idea.

Leverage Existing Tech

24 hours is not much time to churn out something interesting from scratch. In fact it’s nearly impossible. Instead a FedEx project should leverage existing technology that at least one person on the team has a solid understanding of. This doesn’t have to be Atlassian specific code or product, we’re huge fans of open source and we love it when we discover a new project that’s going to help us ship something amazing.

With the recent launch of Confluence 4.0, Atlassian has been investing in our new Rich Text Editor for around a year with nearly every member of the Confluence team contributing to the editor in some way by the wrap up of 4.0 development. This makes the RTE a great launch pad for a FedEx day project. This is how the Autoconvert team was able to get so many intelligent link transformations in by the end of the 24 hour window. The core of Autoconvert is a simple editor plugin that extends the already solid autoformatting of links on paste.

Make It Extensible

This is a bit of social engineering that helps one to win an Atlassian FedEx Day. As mentioned, we’ve got a very strong engineering culture and as such we love anything we can extend and customize to serve our own purposes. So making sure that you FedEx project has extension or plugin points and talking about those when the final presentations roll around can help get you those few extra votes that count.

From a developers perspective Autoconvert is less about how intelligent it makes Confluence, but about how easy it makes adding more of this intelligence. The core plugin is actually just an extension point through which plugin developers can tell Confluence how it should treat links of certain types.

In 9 lines of code we’ve added handling for YouTube’s URL shortener youtu.be.

The excellent plugin tutorial Extending Autoconvert gives an explanation of each step in creating a Confluence extension that makes use of Autoconvert.

Learn Something

You’re going to need fans among the voters. People who when voting will mention casually to their friends that your team is the obvious winner, and keep in mind that these fans probably pitched their own FedEx project just a few minutes ago. These fans come along pretty naturally when you do all points mentioned above.

• Create Customer Value
• Leverage Existing Tech
• Demo The New Shiny
• Make it Extensible

But it’s really not about winning at all. Autoconvert won FedEx and when presented with the opportunity to give a speech every member of the team said thanks into the mic and passed it along. We’re all pretty happy about the fact that we work with people who want to create new shiny stuff that will actually have an impact on users.

With that in mind it’s critically important to do something on FedEx day that you can actually learn from. With Autoconvert some of us learned a bit about how easy it is to write an extension for the Confluence RTE. It’s also the first that that some of us have used a style of programming called continuation passing. We actually thought that these two points were interesting enough that we open sourced the code for Autoconvert it’s up on GitHub here.

Join Us For FedEx Day

Could you win FedEx day without breaking a sweat? You should come work for Atlassian. We’re on the lookout for several different types of awesome.