Sara's Testing World :: ' i COMPLETE QUALiTY & TESTiNG '

SecurityTesting Tools

noreply@blogger.com (The BASS) — Thu, 23 Feb 2017 14:12:00 +0000

Security Softwares

LinkScanner Lite

https://www.metasploit.com/

http://www.acunetix.com/

http://www.veracode.com/

http://www.arachni-scanner.com/

http://w3af.org/

Site Development Process

noreply@blogger.com (The BASS) — Thu, 21 Jan 2016 04:26:00 +0000

The Site Development Process

Every significant web project poses unique challenges, but the overall process of developing a complex web site generally follows six major stages that you should think through before crafting your final project planning and proposal documents:

Site definition and planning
Information architecture
Site design
Site construction
Site marketing
Tracking, evaluation, and maintenance

Developing a large web site is a process that may have far-reaching budgetary, personnel, and public relations consequences for an organization, both during the development of the site and long after its deployment. Too many web sites begin life as ad hoc efforts, created by small interest groups working in isolation from their peers elsewhere in the organization and without fully considering the site’s goals within the context of the organization’s overall mission. The result of poorly planned, hasty development efforts often is an “orphan site,” starved of resources and attention.

As you consider the development process outlined below, note that the construction of the pages that make up the web site is one of the last things that takes place in a well-designed project. Consider each step in the process and its impact on your developing project charter plan (see Developing a Project Charter, below). Think before you act, and make sure you have the organizational backing, budget, and personnel resources you’ll need to make the project a success (fig. 1.8).

Figure 1.8 — Another concept of web development projects, here emphasizing the necessary tightening of focus from broad input and participation in the beginning, to a narrow focused team at the end. Necessary, that is, if you ever want to finish your site on time and on budget.

Site definition and planning

This initial stage is where you define your goals and objectives for the web site and begin to collect and analyze the information you’ll need to justify the budget and resources required. This is also the time to define the scope of the site content, the interactive functionality and technology support required, and the depth and breadth of information resources that you will need to fill out the site and meet your users’ expectations. If you are contracting out the production of the web site, you will also need to interview and select a site design firm. Ideally, your site designers should be involved as soon as possible in the planning discussions.

Site production checklist

Not every site will require detailed consideration of every item on the lists below. Web teams within corporations or other large enterprises can often count on substantial in-house technology support when creating web sites. If you are on your own as an individual or small business, you may need to contract with various technology and design vendors to assemble everything you’ll need to create a substantial content site or small e-commerce site.

Production

Will your site production team be composed of in-house people, outside contractors, or a mix of the two?
Who will manage the process?
Who are your primary content experts?
Who will be the liaison to any outside contractors?
Who will function long-term as the webmaster or site editor?

Technology

What operating systems and browsers should your site support?
- Windows, Macintosh, unix, Linux
- Firefox, Internet Explorer, Safari, Opera; minimum version supported
What is the network bandwidth of the average site visitor?
- Internal audience or largely external audience
- Ethernet or high-speed connections typical of corporate offices
- isdn, or dsl medium-speed connections typical of suburban homes
- Modem connections for rural audiences
Will the site have dynamic html and advanced features?
- JavaScript required
- Java applets required
- Third-party browser plug-ins required
- Special features of the Linux, unix, or Microsoft iis server environments required
- Special security or confidentiality features required
How will users reach support personnel?
- Email messages from users
- Chat rooms, forums, help desks, or phone support
Does the site require database support?
- User logins required to enter any site areas
- Questionnaires required
- Search and retrieval from databases needed
Will the site have audiovisual content?
- Video or audio productions

Web server support

Will the site reside on an in-house web server or be outsourced to an Internet service provider for web hosting?
- Disk space limitations, site traffic limitations, extra costs
- Adequate capacity to meet site traffic demands
- Twenty-four-hour, seven-day-a-week support and maintenance
- Statistics on users and site traffic
- Server log analysis: in-house or outsourced
- Search engine suitable for your content
- cgi, programming, database middleware support
- Database support or coordination with in-house staff

Budgeting

What are staffing costs?
- Short-term development staff
- Long-term editorial and support staff (site editor or webmaster)
- Ongoing server and technical support
- Database maintenance and support
- New content development and updating
What hardware and software are needed for in-house development team members?
What are staff training costs?
- Web use, database, web marketing, web design
What are the outsourcing fees?
- Site design and development
- Technical consulting
- Database development
- Site marketing

Appoint a site editor to become the “process manager”

Every successful new web site makes a transition from a development project to an ongoing editorial process that keeps the site alive and fresh over time. You’ll need a project manager to get your new site launched, but you’ll also need to hand the site over to a process manager (read: “editor”) after the site is launched. A site that is “everyone’s responsibility” can quickly become an orphan. For current content and consistent editorial, graphic design, and management policies you’ll need one person to act as the editor of the overall web site. The site editor’s duties will vary according to how you choose to maintain your site. Some editors do all the work of maintaining site content, relieving their coworkers of the need to deal directly with web page editing. Other editors coordinate and edit the work of many contributors who work directly on the site pages, aided by a maintenance plan that specifies who is responsible for the content of each section of the site. When multiple people contribute to site maintenance, the site editor may choose to edit pages after they are created and posted to avoid becoming a bottleneck in the communications process. However, high-profile public pages or pages that contain important information should be vetted by the editor before posting. A site editor will also typically bear the primary responsibility for keeping the site content as visible as possible in local enterprise or general Internet search engines. Broken links and scrambled content organization schemes can harm your search engine rankings and make your content harder for users to locate. The site editor is also the logical person to handle the collection and analysis of web site analytics and to produce periodic reports on the usage of the site.

In addition to ensuring editorial quality, a site editor must also make certain that the content of the site reflects the policies of the enterprise, is consistent with local appropriate use policies, and does not contain material that violates copyright laws. Many people who post pictures, cartoons, audiovisual files, or written material copied from other sites on their own sites do not understand copyrights and the legal risks in using copyrighted materials inappropriately. A site editor is often an institution’s first line of defense against an expensive lawsuit over the misuse of protected material.

Information architecture

At this stage you need to detail the content and organization of the web site. The team should inventory all existing content, describe what new content is required, and define the organizational structure of the site. Once a content architecture has been sketched out, you should build small prototypes of parts of the site to test what it feels like to move around within the design. Site prototypes are useful for two reasons. First, they are the best way to test site navigation and develop the user interface. The prototypes should incorporate enough pages to assess accurately what it’s like to move from menus to content pages. These prototypes can be used to test the information architecture with users. Second, creating a prototype allows the graphic designers to develop relations between how the site looks and how the navigation interface supports the information design. The key to good prototyping is flexibility early on: the site prototypes should not be so complex or elaborate that the team becomes too invested in one design at the expense of exploring better alternatives.

Typical results or contract deliverables at the end of this stage include:

Detailed site design specification
Detailed description of site content
- Site maps, thumbnails, outlines, tables of contents
User-tested wireframes and prototypes demonstrating site architecture
Multiple graphic design and interface design sketches
Detailed technical support specification
- Browser technology supported
- Connection speed supported
- Web server and server resources
Proposals to create programming or technology to support specific features of the site
A schedule for implementing the site design and construction

Site design

At this stage the project acquires its look and feel, as the page grid, page design, and overall graphic design standards are created and approved. Now the illustrations, photography, and other graphic or audiovisual content for the site need to be commissioned and created. Research, writing, organizing, assembling, and editing the site’s text content is also performed at this stage. Any programming, database design and data entry, and search engine design should be well under way by now. The goal is to produce all the content components and functional programming and have them ready for the final production stage: the construction of the actual web site pages.

Typical products or deliverables at the end of this stage include:

Content components, detailed organization and assembly

Text, edited and proofread
Graphic design specifications for all page types
- Finished interface graphics for page templates
- Header and footer graphics, logos, buttons, backgrounds
Detailed page comps or finished examples of key pages
- Site graphic standards manual for large, complex sites
Interface design and master page grid templates completed
- Finished html template pages
Illustrations
Photography

Functional and logic components

JavaScript scripts, Java applets designed
Database tables and programming, interaction prototypes completed
Search engine designed and tested

Templates

Whether you develop your site on your own or hire a professional web developer, you should develop page templates for your new web site. It’s much easier to add new pages when you can start from a page that already contains basic navigation and site graphics. If you have a team working on page development, you will want to share templates, along with standards on how to handle page text and content graphics. Popular web site development software such as Adobe Dreamweaver offer powerful templates and standard reusable libraries of site graphics and html that make it easy to create new pages and maintain consistency in your site.

Accessibility

In most large enterprises, providing universal access to web pages is long-established institutional policy, and in many instances it is required by state or federal regulations. It is critical, therefore, that you validate your designs and page templates and the content of your site throughout the development process to ensure that your pages are accessible to all users. Use the guidelines and techniques developed and maintained by the Web Accessibility Initiative (wai) as a measure against which to test the accessibility of your pages.

Sidebar: Web Accessibility

Making sure that web sites are accessible and usable by people with disabilities is the focus of web accessibility. The key efforts in this area arise out of the World Wide Web Consortium (W3C) in the form of the Web Accessibility Initiative. WAI efforts are focused on developing tools and best practices that promote the development of universally accessible web sites. Most validation tools use the WAI guidelines as the standard against which to measure sites.
The following links may be helpful in learning what makes a site accessible, how to promote accessibility within an organization, what’s required for accessible designs, and how to evaluate designs for accessibility:

Site construction

Only at this mature stage of the project are the bulk of the site’s web pages constructed and filled out with content. By waiting until you have a detailed site architecture, mature content components, fully tested wireframes and prototypes, and a polished page design specification you will minimize the content churning, redundant development efforts, and wasted energy that inevitably result from rushing to create pages too soon. Of course, you will always learn new things about your overall design as the prototype matures into the full-blown web site. Be prepared to refine your designs as you and your users navigate through the growing web site and discover both weak spots and opportunities to improve navigation or content.

Once the site has been constructed, with all pages completed and all database and programming components linked, it is ready for user testing. Testing should be done primarily by people outside your site development team who are willing to supply informed criticism and report programming bugs, note typographic errors, and critique the overall design and effectiveness of the site. Fresh users will inevitably notice things that you and your development team have overlooked. Only after the site has been thoroughly tested and refined should you begin to publicize the url of the site to a larger audience.

Typical products or deliverables at the end of this stage should include:

Finished html for all web pages, all page content in place
Finished navigation link structure
All programming in place and linked to pages, ready for user testing
All database components in place and linked to site pages
All graphic design, illustration, and photography in place
Final proofreading of all site content
Detailed testing of database and programming functionality
Testing and verification of database reporting features
Testing of site user support procedures, answering email, etc.
Archives of all site content components, html code, programming code, and any other site development materials

Maintainable code

Most businesses or departments in larger enterprises will contract with a web development group to create the initial site design and to build all the pages in the first version of the web site. They then assume responsibility for the site, doing some or all of the daily maintenance and updating content as needed to keep the site current.

Often not until the practicalities of site maintenance arise do customers realize the importance of understanding the details of how the web developer generated the html and other code that makes up the web site. Although all html and css markup is much the same to web browsing software, how the html and css is formatted and what web authoring tool the developer used can make a huge difference in how the code looks to a human reader.

Consider the two code examples below:

Example 1

<table summary=”HR Committee Schedule, FY 2008”>
<tr>
<th>Meeting Dates 2008</th>
<th>Agenda Item Submission Deadline</th>
</tr>
<tr>
<td>Monday, Oct 6, 2008</td>
<td>Friday, Oct 3, 2008</td>
</tr>
</table>

Example 2

<table summary="HR Committee Schedule, FY 2008"> <tr> <th>Meeting Dates 2008</th> <th>Agenda Item Submission Deadline</th> </tr> <tr> <td>Monday, Oct 6, 2008</td> <td>Friday, Oct 3, 2008</td> </tr> </table>

Which example do you find easier to understand? These code examples are exactly equivalent to a web browser, but most people would find Example 1 significantly easier to read and understand. If you contract with a developer to build your site, it is important to understand how the developer writes code, what state the code will be in when the site is delivered, and whether the software used by the developer is compatible with what you will be using to maintain the site after delivery. Some web development software produces html code that is nearly impossible for a human to read without significant (and expensive) reformatting. Other programs (such as Adobe Dreamweaver) produce html code that is easy for web programmers to read, which can make a huge difference if you decide to change web developers or if you decide to edit html directly when maintaining your site.

If you hire someone to create your web site or components of your site, such as database or dynamic elements, be sure to ask what tools they will use to write the html and any other code. Ask to see examples of code written for other clients. Have your technology lead examine the code to be sure the developer inserts explanatory comments and dividers for legibility in the code. Be sure to find out whether there will be problems or conflicts if you use your favorite tools to edit the code the developer produces. Make sure the developer understands what editing tools you prefer to use and develops the code for maximum compatibility with your maintenance tools.

HTML and CSS code validation

Also ask for representative sites the developer has created, and choose pages to test for code validity, using the free online tools available from thew3c (see below). Many perfectly functional pages will fail w3c validity tests either for relatively minor code mistakes or for complex links to database or application urls that use problematic characters like ampersands (&). Ignore minor failures in the code validation, since they are unlikely to cause major functional problems. But if representative pages come back from testing with long lists of html code problems and css mistakes, beware of that developer’s work, and thoroughly discuss and put in writing your expectations around code validation as part of any contract.

html and css code validation tools from the w3c:

html validation: validator.w3.org
css validation: jigsaw.w3.org/css-validator

Today’s web pages are much more complex than pages in the past, and many new mobile and other devices can now display web pages. Search visibility is crucial to successful web sites, and web accessibility is a legal requirement with a growing set of case law behind it. Using carefully validated html and css code is one of your best strategies for getting maximum flexibility and value from your web development dollars. Be extremely wary of a web developer who tells you, “Validation isn’t important.”

Site marketing

Your web site should be an integral part of all marketing campaigns and corporate communications programs, and the url for your site should appear on every piece of correspondence and marketing collateral your organization generates.

If your web site is aimed primarily at local audiences you must look beyond getting listed in standard web indexes, such as Yahoo! and Google, and publicize your url where local residents or businesses will encounter it. Local libraries, newspapers, and schools are often the key to publicizing a new web site within a specific locale.

You may also find opportunities to cross-promote your site with affiliated businesses, professional organizations, broadcast or print media, visitor or local information agencies, real estate and relocation services, Internet access providers, and local city or town directory sites. Your organization could also feature local nonprofit charitable or school events on your web site. The cost in server space is usually trivial, and highly publicized local events featuring a web page hosted within your site will boost local awareness of your web presence. Site sponsorship might also interest local broadcast media as an interesting story angle.

Your home page url should appear in all:

Print advertisements
Radio and television advertisements
Lobby kiosks in high-traffic areas of your enterprise or in local libraries, schools, or other suitable venues
Direct mail campaigns
Business cards
Stationery
Bills and statements
Product manuals and product packaging
Response cards and warrantee cards
Publications and promotional materials
Press releases
Posters and billboards

Tracking, evaluation, and maintenance

Your web server software can record an abundance of information about visitors to your site. Even the simplest site logs track how many people (unique visitors) saw your site over a given time, how many pages were requested for viewing, and many other variables. By analyzing the server logs for your web site you can develop quantitative data on the success of your site. The logs will tell you which pages were the most popular and what brands and versions of web browser people used to view your site. Server logs can also give you information on the geographic location of your site users. Detailed logs are the key to quantifying the success of a web site. Your webmaster should archive all site logs for long-term analysis and should be prepared to add or change the information categories being logged as your needs and interests change.

A number of popular software packages are designed to produce easily readable site traffic reports, complete with data graphics and charts to aid in data analysis. As a service to customers, site hosting companies often offer reports from popular site analysis programs like Google Analytics for no additional charge. Before contracting with an Internet service provider for site hosting services, always ask about site analysis services. If your isp(Internet service provider) or corporate web site does not offer a good site traffic analysis package, ask whether the webmaster can give you access to a monthly server log of your account. Basic versions of traffic analysis programs like WebTrends are inexpensive and you can run them on a personal computer if you can gain access to the raw web server log from your isp or corporate webmaster (fig. 1.9).

Figure 1.9 — Web statistics are much more than just raw measures of traffic. They can tell you what content people looked at, where your visitors are coming from, and provide a rich set of technical information on what technology your typical readers are using. From Google Analytics.

Maintaining the site

Don’t abandon your site once the production “goes live” and the launch parties are over. The aesthetic and functional aspects of a large web site need constant attention and grooming, particularly if a group of individuals shares responsibility for updating content. Your site editor will need to be responsible for coordinating and vetting the new content stream, maintaining the graphic and editorial standards, and ensuring that the programming and linkages of all pages remain intact and functional. Links on the web are perishable, and you’ll need to check periodically that links to pages outside your immediate site are still working. Don’t let your site go stale by starving it of resources just as you begin to develop an audience—if you disappoint them by not following through, it will be doubly difficult to attract your audience back to the site.

Backups and site archives

The site editor should be sure that the web site is regularly backed up onto a secure and reliable storage medium to ensure that a catastrophic hardware failure in your web server does not wipe out your web site. Most web servers maintained by it professionals or commercial web service providers are backed up at least once a day. If you don’t know what your backup schedule is, ask your webmaster or web hosting provider. Human error is the most common reason you may need quick access to a backup copy of your web site. Unfortunately, it’s easy to overwrite an old file (or a whole directory of files) accidentally over a newer version on the web server, to delete something important in error, or to wipe out someone else’s work by mistake when updating a web site. A recent backup (ideally no more than twenty-four hours old) can often be a lifesaver.

If your site is successful, it will quickly become an important record of your enterprise’s work, your accomplishments, and a valuable record of the “state of things” as the site evolves over time (fig. 1.10). Unfortunately, too little attention is paid to this aspect of web sites, and we are collectively losing huge pieces of our hig because no one thinks about preserving permanent records of a web site. Unless your web site is prohibitively large, your web site editor should arrange to collect and store the files of the site periodically or contract with your web service provider to set aside a backup version at regular intervals as a long-term archive. We take for granted the “paper trail” of history left by conventional business and work practices. Without a plan for preserving our digital works, our collective history may vanish without a trace.

Figure 1.10 — Much as older buildings grow through additions and adaptation over time, sites grow and change in response to changing needs and ideas. (Adapted from Stewart Brand’s How Buildings Learn.)

Testing & Deployment

noreply@blogger.com (The BASS) — Thu, 21 Jan 2016 04:21:00 +0000

Testing and deployment

Testing: it’s one of those potentially dull and yet highly contentious topics. I’ll try to avoid the dull, but there may be some contention brewing on the horizon.

Most programmers work on software that others have created and that others will develop after them. Sharing the code creates a community in which the team of developers live and interact. Preserving code quality not only has a direct impact on how well a developer can do their job, it becomes a moral obligation to their colleagues in the community.

Add to this the responsibility to deliver quality to the paying customer, the direct relationship between app uptime and revenue, plus the difficulty of testing in a complex web ecosystem, and it’s easy to see why web developers have a love/hate relationship with testing.

What to test

Web app tests fall into five main categories:

Functional tests: does the app work?
Compatibility tests: does the app work consistently for everyone?
Performance tests: does the app respond quickly and how does traffic affect performance?
Security tests: is the app secure against attacks?
Usability tests: is the app easy to use and does it respond to interaction as expected?

All these tests add up to a lot of time, but do you really need them all? As usual, it depends. Let common sense prevail. If your app is used in a hospital to prescribe medication doses, or it’s a critical financial component in a large enterprise, don’t skimp on the tests. On the other hand, I suspect that most of you are building a spanking new web app that doesn’t impact human safety or hundreds of jobs. The critical thing to remember for new ideas is that your app will probably change.

If you’re following this book’s advice to build a minimum viable product, you will do the bare minimum necessary to get something out the door as quickly as possible to test the waters. Once you know more about your market, you can refine your app and try again, each time inching closer to a product that your customers want.

Focus your tests around the MVP process and apply the same minimal approach. Test what you need to ensure that your product is given a fair chance in the market, but don’t worry about scaling to 100,000 customers or about the long-term testability of your code: it is likely to change significantly in the first few iterations. Your biggest problems right now are identifying desirable features and getting people to use the app.

Some types of test require greater investment than others before they pay off.

Some tests yield improvements quickly (the left graph) whereas others require more investment before they start to pay off (the right graph). Be lazy and target the quick wins for your MVP. Invest in medium-term tests when your app finds a foothold in the market, and as your codebase undergoes less change between revisions.

Functional testing

The lazy version of functional testing is more accurately called system testing, which confirms that the app works as a whole but doesn’t validate individual functions of the code.

First, you create a simple test plan listing the primary and secondary app features and paths to test before each release. Next, you or your team should personally use the app to run through the test plan and check for major problems. Finally, ask some friends or beta testers to use the app and report any problems.

That’s it. Any bugs that aren’t discovered through regular activity can probably be disregarded for the time being. It’s not comprehensive or re-usable, but for an MVP app with minimal features it should cover the basics. When your app gains traction and the codebase begins to settle between iterations, it’s time to progress to a medium-term investment in functional testing: automated unit and interface tests that enable efficient regression tests of changes between versions – don’t worry, I’ll decipher what these mean in the next section.

Unit tests

A unit test is a piece of code designed to verify the correctness of an individual function (or unit) of the codebase. It does this through one or more assertions: statements of conditions and their expected results.

To test a function that calculates the season for a given date and location, a unit test assertion could be, “I expect the answer ‘summer’ for 16 July in New York”. A developer normally creates multiple test assertions for non-trivial functions, grouped into a single test case. In the previous example, additional assertions in the test case should check a variety of locations, dates and expected seasons.

Not all functions are as straightforward to test. Many rely on data from a database or interactions with other pieces of code, which makes them difficult to test in isolation. Solutions exist for these scenarios, such as mock objects and dependency injection, but they create a steep initial learning curve for those new to unit testing.

It also takes some investment to pay off. Even a simple web app can contain hundreds of individual functions. Not all require a unit test, but even if the initial tests are focused solely on functions that contain critical logic, a significant number may be required to catch all of the important bugs.

The effort does eventually pay off, however. Commonly stated benefits of unit testing include:

Greater ease in making sizeable changes to the code. Unit tests are automated and isolated to individual functions, allowing you to quickly check whether changes break existing functionality.
Better designed, re-usable code. Developers familiar with unit testing tend to create code that is easier to test (known as the code’s testability). This inherently encourages best practices: independent, decoupled functions with clear logical responsibilities.
Better documentation. A unit test provides a developer with an easy way to understand a function’s behaviour, and is more likely to be updated than a document.

Some developers find unit testing so beneficial that they make it the initial scaffolding from which the app is developed. Under this test-driven development approach, each test is written prior to the functional code, to define the expectations of the function. When it is first run, the test should fail. The developer then writes the minimum amount of code necessary to satisfy the test. Confidence in the successful test enables the developer to iteratively re-factor improvements to the code.

Whether or not you decide to adopt test-driven development, hundreds of frameworks are available to ease your implementation of automated unit testing. The Wikipedia list¹ is a great place to start.

Automated interface testing

Web app logic is shifting from the server to the browser. As your app progresses from a simple MVP to a more mature product, the interface code will become more elaborate, and manual tests more cumbersome.

Automated interface tests share the same assertion principle as unit tests: conditions are set and the results are checked for validity. In the case of interface tests, the conditions are established through a number of virtual mouse clicks, form interactions and keystrokes that simulate a user’s interaction with the app. The result is usually confirmed by checking a page element for a word, such as a success message following a form submission.

In terms of automated interface test frameworks, a sole developer or small team of developers who are intimately familiar with the interface code may prefer the strictly code-oriented approach of a tool such as Watir². For larger teams or apps with particularly dynamic interfaces, the graphical test recording of Selenium³ may be better suited. Both tools support automated tests on multiple platforms and browsers.

Once you’ve started automated interface testing it’s easy to get sucked in, as you try to cover every permutation of user journey and data input. For the sake of practicality, it’s best to ignore business logic in the early days: tests for valid shipping and tax values are better served in unit tests. Instead, create tests for critical workflows like user registration and user login – verify paths through the interface rather than value-based logic.

Compatibility testing

Print designers and television companies enjoy a luxury unknown to web developers: the limits of their media. We must contend with the rapid proliferation of devices and software with differing capabilities, and must squeeze as much compatibility as we can out of our apps to attract and retain the largest possible audience.

The main causes of web app incompatibility are:

Web browsers. Internet Explorer, Firefox, Google Chrome and Safari each enjoy a non-trivial market share, with variations in rendering and JavaScript engines.
Versions. Web browsers regularly update their layout engines and technology support. Internet Explorer 9 renders a webpage very differently from Internet Explorer 6. Google Chrome progressed from version 0.2 to version 10 in less than three years⁴, a major release every three months on average.
Operating systems. Web browsers are not consistent across operating systems, even if the differences are limited to native interface components like form buttons. For example, Safari displays slight differences between Windows, Mac OS X and iOS.
Devices. Desktop computer, laptop, tablet, smartphone, LCD television and digital projector: each has a hardware and software profile that influences the display and experience of your web app.
Display profiles. Screen resolution, pixel density and colour management variations alter the perception of your app interface.
Configurations and preferences: Window sizes, plug-ins, fonts, zoom magnification and browser privacy settings vary from user to user.
Personal capabilities. Each of us has a unique set of physical and mental capabilities that changes throughout our lives, from our ability to see, hear and move, to the language and words that we understand. Addressing these accessibility needs is not only commercially shrewd, it is a legal requirement in many countries⁵.
Environments. The local environment may override the capabilities of the device and user. For example, audio, display or device input may be restricted if the user is browsing in a coffee shop, in direct sunlight or in a noisy classroom.

That seems like an awful lot to think about, but the extent to which the technical compatibility factors affect the success of your app will depend on your target market.

Markets with high and low technical sophistication tend to exhibit less diversity in web software and devices

If your app is targeted at web developers or other technology-savvy users (AKA geeks), you can presume that a majority will have recent web browser versions and sophisticated hardware. Alternatively, for an app designed for a traditional financial enterprise, you may be able to assume a majority of users with Microsoft operating systems and Internet Explorer. Only a mass-market app (Google, Facebook and the rest) must consider the widest possible variation in hardware, software and user capabilities from the get-go.

If you don’t trust your gut audience stereotypes to refine the range of your compatibility tests, use analytics data from your teaser page or MVP advertising campaign. Build a quantified profile of your target market and aim to provide compatibility for at least 90% of the users based on the largest share of browser/operating system/version permutations.

Worst case: if you can’t assume or acquire any personalised statistics, test compatibility for browsers listed in the Yahoo! A-grade browser support chart⁶.

It’s important to realise that not all market segments offer equal value. To use a sweeping generalisation as an example, you may find that Mac Safari users constitute a slightly smaller share of your visitors than Windows Firefox users, but they convert to paid customers at twice the rate. It’s important that you measure conversion as quickly as possible in your app lifecycle (you can start with users who sign up for email alerts on the teaser page) and prioritise compatibility tests accordingly.

You can overcome cross-browser inconsistencies if you take advantage of mature front-end JavaScript libraries, CSS frameworks and a CSS reset style sheet. Some additional problems may be resolved by validating your HTML and CSS; use the W3C online validator⁷ or install a browser validation plug-in to detect and correct mistakes.

To achieve accurate cross-browser compatibility you’ll need regular access to a variety of browsers, versions and operating systems. You may find that online services like Browsershots⁸ or Spoon⁹ suit your needs, but to regularly test dynamic web interfaces, nothing beats having a fast local install of the browser, either as a native installation or in a local virtual environment, such as VMWare¹⁰ or Parallels¹¹. Microsoft handily makes virtual images of IE6, 7 and 8 available¹². If you opt for virtualisation, upgrade your computer’s memory to appreciably improve performance.

Aiming for compatibility on all four major browsers, many developers favour a particular testing order:

Google Chrome. Excellent modern standards support, efficient rendering, a speedy JavaScript engine and integrated debugging tools make Chrome a great baseline for standards-based browser compatibility.
Safari. As the internal WebKit rendering engine is shared with Chrome, achieving Safari compatibility should be a straightforward second step.
Firefox. Firefox feels a little slower to regularly tweak-and-refresh, but a high adherence to standards and mature debugging tools ease compatibility from Chrome and Safari to Firefox.
Internet Explorer. Finally, once the standards-based browsers have been satisfied without too many tweaks to the code, it’s time to slog through the browser-specific workarounds for the several popular but often standards-averse versions of Internet Explorer.

Like cross-browser tests, accessibility tests are offered by a number of free online web services, (WAVE¹³, for example). Automatic tests can only detect a subset of the full spectrum of accessibility issues, but many of the better services behave as guided evaluations that walk you through the manual tests. For apps in development that aren’t live on the web, or for a faster test-fix-test workflow, you may prefer to use a browser plug-in, like the Firefox Accessibility Extension¹⁴.

Some of the most important accessibility issues¹⁵ to test include:

Do images, videos and audio files have accurate text alternatives?
Do form controls have relevant labels or titles?
Is the content marked up with the most appropriate semantic HTML, including form and table elements?
Does the app reset the focus to the new content after an Ajax content update?
Do colour and contrast choices allow the text to be read easily?
If colour is used to convey information (in required form fields, for instance), is the same information also available in text format?
Can all of the app functionality be accessed using the keyboard alone?

Performance testing

Web app responsiveness can be evaluated through performance tests, load tests and stress tests. For the sake of practicality, you may want to consider starting with simple performance tests and hold off on the more exhaustive load and stress tests until you’ve gained some customers. Luckily for us, scalable cloud hosting platforms enable us to be slightly lazy about performance optimisation.

Performance tests

Performance tests measure typical response times for the app: how long do the key pages and actions take to load for a single user? This is an easy but essential test. You’ll first need to configure your database server and web application server with profilers to capture timing information, for example with Microsoft SQL Server Profiler (SQL Server), MySQL Slow Query Log (MySQL), dotTrace¹⁶ (.NET), or XDebug¹⁷ with Webgrind¹⁸ (PHP). You can then use the app, visiting the most important pages and performing the most common actions. The resultant profiler data will highlight major bottlenecks in the code, such as badly constructed SQL queries or inefficient functions. The Yahoo! YSlow¹⁹ profiler highlights similar problems in front-end code.

Of course, this isn’t an accurate indication of the final production performance, with only one or two people accessing the app on a local development server, but it’s valuable nonetheless. You can eliminate the most serious obstructions and establish a baseline response time, which can be used to configure a timed performance test as an automated unit or interface test.

Yahoo! YSlow

Load tests

Load tests simulate the expected load on the app by automatically creating virtual users with concurrent requests to the app. Load is normally incremented up to the maximum expected value to identify the point at which the application becomes unresponsive.

For example, if an app is expected to serve 100 users simultaneously, the load test might begin at 10 users, each of whom make 500 requests to the app. The performance will be measured and recorded before increasing to 20 users, who each make 500 requests, and so forth.

New bottlenecks may appear in your web app profiling results that highlight a need for caching, better use of file locking, or other issues that didn’t surface in the simpler single-user performance test. Additionally, load tests can identify hard limits or problems with server resources, like memory, disk space and so on, so be sure to additionally profile your web server with something like top²⁰ (Linux) or Performance Monitor²¹ (Windows).

Because load tests evaluate the server environment as well as the web app code, they should be run against the live production server(s) or representative development server(s) with similar configurations. As such, the response times will more accurately reflect what the user will experience. Which brings us to an interesting question: what is an acceptable response time?

It all depends on the value of the action. A user is more prepared to wait for a complex financial calculation that could save them hundreds of dollars than to wait for the second page of a news item to load. All things considered, you should aim to keep response times to less than one second²² to avoid interrupting the user’s flow.

Free load testing software packages include ApacheBench²³, Siege²⁴, httperf²⁵, and the more graphical JMeter²⁶ and The Grinder²⁷.

Stress tests

A stress test evaluates the graceful recovery of an app when placed under abnormal conditions. To apply a stress test, deliberately remove resources from the environment or overwhelm the application while it is in use:

Use a load test tool to simulate an unsupportable volume of traffic.
Create a large temporary file that fills the available disk space.
Restart the database server.
Run a processor-intensive application on the web server.

When the resources are reinstated the application should recover and serve visitors normally. More importantly, the forced fail should not cause any detrimental data corruption or data loss, which may include:

Incomplete cache files that are mistakenly processed or displayed when the app recovers.
An incomplete financial transaction, where payment is taken from the customer but their order is not recorded.

Because a stress test tackles infrequent edge cases, it is another task that you can choose to defer until your app begins to see some success, even if you do have to deal with the occasional consequence, such as manual refunds and cache re-builds.

Security testing

The discouraging reality is that it’s impossible to be fully secure from all the varieties of attacks that can be launched against your app. To get the best security coverage, it is vital to test the security of your app at the lowest level possible. A single poor cryptography choice in the code may expose dozens of vulnerabilities in the user interface.

There really is no substitute for your team having sufficient knowledge of secure development practices (see chapter 18) at the start of the project. “An ounce of prevention is worth a pound of cure”, as the internet assures me Benjamin Franklin once said, albeit about firefighting rather than web app security.

The next best thing is to instigate manual code reviews. If you’re the sole developer, put time aside to review the code with the intention of checking only for potential security vulnerabilities. If you’re working in a team, schedule regular team or peer reviews of the code security and ensure that all developers are aware of the common attack vectors: unescaped input, unescaped output, weak cryptography, overly trusted cookies and HTTP headers, and so on.

If your app handles particularly sensitive information – financial, health or personal – you should consider paying for a security audit by an accredited security consultant as soon as you can afford to. Web app security changes by the week and you almost certainly don’t have the time to dedicate to the issue.

You should frequently run automated penetration tests, which are not a silver bullet but are useful for identifying obvious vulnerabilities. Many attackers are amateurs who rely on similar automated security test software to indiscriminately scattergun attack thousands of websites. By running the software first, you’re guarding against all but the most targeted of attacks against your app. Skipfish²⁸, ratprox²⁹⁷ and the joyously named Burp Intruder³⁰ are three such tools that can be used in conjunction with data from the attack pattern database fuzzdb³¹.

For a fuller understanding of web app security testing, put some time aside to read through the comprehensive OWASP Testing Guide³².

Usability testing

See chapter 15 for an in-depth look at usability testing.

Deployment

In the early stages of your web app’s development you’ll probably manually copy files from your local development computer to your online server. While you have no customers and a simple single server, it’s more valuable to devote your time to iterations of your MVP features than to a sophisticated deployment process.

As the customer base, technical complexity and hosting requirements of your app grow, the inefficiency and fragility of manual SFTP sessions will quickly become apparent.

Automated deployment isn’t only about enforcing the quality of code upgrades and reducing downtime. A solid deployment tool gives you the confidence to push changes to your users more easily, speed up the feedback loop on new features, isolate bugs quicker, iterate faster and build a profitable product sooner.

The simplest form of automated deployment will script the replication of changed files from your local environment to the live server. You can do this through your version control software³³ or rsync³⁴, but eventually you’ll run into problems with choreographing file and database changes, differences in local/live configuration, and any number of other technical intricacies.

A better solution assumes that the app deployment has three parts: the local preparation (or build), the transfer of files, and the post-upload remote configuration.

A build is normally associated with the compilation of code into executable files, but the term can also apply to popular interpreted web languages that don’t require compilation, such as PHP or Ruby. For the purposes of deployment, the release build process normally includes:

A fresh checkout of the code to a test environment.
Preparation of files for the live environment, which may include minimisation of JavaScript and CSS files, and bundling individual images into single sprite files.
A run through all automated tests where the deployment will halt on a failed test.
Configuration of files for the live environment (database settings, for instance).
Automatic creation of release notes or updated documentation.

With the build prepared, the files can be transferred to one or more live servers with rsync or a similar utility. The files for each release should be copied to a new time-stamped directory, not directly over the live files.

In the final stage of the deployment, the equivalent of an install script is run on the server to switch from the current release to the newly uploaded release:

The database is backed up.
Database migration scripts are run on the database to modify or add the appropriate tables and columns for the latest code³⁵. In most situations, this is safe to run on the live database because changes tend to be backwards compatible with the current live code, or should be designed to be so. In the worst case, the live site may need to be put into maintenance mode to prevent data modification while the live database is copied, major changes are applied, and the remainder of the install process is carried out.
Automated tests are run on the new live install. If a test fails, the install halts and all updates, including database migrations, are rolled back.
Cache and session files are reset.
The new version is made live by pointing the live server directory at the new release directory with a symlink. Using this system, no app files ever exist under the live server directory, which always points to a specific release directory.

Use a symlink to quickly move your website between different versions

Automated build and deployment tools are readily available in most popular web languages: Jenkins³⁶ and CruiseControl³⁷ (Java), Phing³⁸ (PHP) and Capistrano³⁹ (Ruby) are among those frequently used. Note that except for familiarity, there’s no reason why your deployment tool has to be the same language as your web app. Just because your app is in PHP, it doesn’t mean you should rule out the excellent Jenkins or Capistrano tools from your process.

For an extra layer of confidence in your release process, you should incorporate an intermediate deployment to a staging server, where you test database migrations on a copy of the live database and perform manual acceptance tests. If your team uses your app internally, you can even make the staging server the primary version of the app that you use, so that you ‘dogfood’ the new candidate version for a couple of days before pushing it out to the live server.

Updates

How often you release an update will depend on a number of factors: how rapidly you develop features; how much manual testing is required; how much time you have available for testing; and how easy or automated the deployment process is.

You should schedule releases so that you don’t build up a long backlog of changes. Each change adds risk to the release, and feedback is easier to measure when new features are released independently. Some companies like Etsy make dozens of releases a day⁴⁰, but this continuous deployment approach relies on a serious investment in deployment automation and comprehensive automated test coverage. A more reasonable schedule, as adopted by Facebook and others, is to aim for a release once a week.

Summary

Tests and deployment options come in many shapes and sizes; start with critical checks to your core features and gradually expand your test infrastructure as your app features stabilise and your user base grows.

Create a test plan of primary and secondary features and paths.
Run through the test plan with your team.
Implement unit tests for critical functions and complex business logic.
Implement automated interface tests to test paths through the interface.
Build a profile of your target market’s browser usage.
Use local virtualised browsers for compatibility testing.
Test for accessibility issues, including alternative text, appropriate semantic markup, use of colour and keyboard controls.
Profile your database and web server during normal use.
Load test to identify resource issues and bottlenecks.
Stress test to assess graceful recoverability.
Perform manual code reviews.
Run automated penetration tests.
Develop an automated build and deployment process.

CMMI - SQA Audit

noreply@blogger.com (The BASS) — Mon, 30 Mar 2015 07:15:00 +0000

SQA audit is the audit conducted by the SQA of the project / function.

From the CMMi point of view below are the four goals:

1. SQA activities are planned.

2. Adherence of software products and activities to the applicable and defined standards, procedures, and requirements are verified objectively.

3. All affected groups and individuals are informed of software quality assurance activities and results.

4. Non-compliance issues that cannot be resolved within the software project are addressed by senior management.

Commitment needed: The project follows a written organizational policy for implementing software quality assurance (SQA). This policy typically specifies that:

1. The SQA function is in place on all software projects and functions.

2. The SQA group has a reporting channel to senior management that is independent of

- Project manager
- Software engineering group
- And other software-related groups

3. Senior management should periodically reviews the SQA activities and results.

Four Abilities:
1. A group that is responsible for coordinating and implementing SQA for the project (i.e., the SQA group) exists.

2. Adequate resources and funding are provided for performing the SQA activities:

- A manager is assigned specific responsibilities for the project’s SQA activities.
- A senior manager, who is knowledgeable in the SQA role and has the authority to take appropriate oversight actions, is designated to receive and act on software non-compliance items.
- Tools to support the SQA activities are made available.

3. Members of the SQA group are trained to perform their SQA activities.

4. The members of the software project receive orientation on the role, responsibilities, authority, and value of the SQA group.

Eight Activities:
1. An SQA plan is prepared for the software project according to a documented procedure. This procedure typically specifies that:

- The SQA plan is developed in the early stages of, and in parallel with, the overall project planning.
- The SQA plan is reviewed by the affected groups and individuals.
- The SQA plan is managed and controlled.

2. The SQA group’s activities are performed in accordance with the SQA plan. The plan covers:

- Responsibilities and authority of the SQA group.
- Resource requirements for the SQA group (including staff, tools, and facilities).
- Schedule and funding of the project’s SQA group activities.
- The SQA group’s participation in establishing the software development plan, standards, and procedures for the project.
- Evaluations to be performed by the SQA group.
- Audits and reviews to be conducted by the SQA group.
- Project standards and procedures to be used as the basis for the SQA group’s reviews and audits.
- Procedures for documenting and tracking non-compliance issues to closure.
- Documentation that the SQA group is required to produce.
- Method and frequency of providing feedback to the software group and other software-related groups on SQA activities.

3. The SQA group participates in the preparation and review of the project’s software development plan, standards, and procedures.

- The SQA group provides consultation and review of the plans, standards, and procedures with regard to compliance to organizational policy, compliance to externally imposed standards and requirements (e.g., standards required by the statement of work), standards that are appropriate for use by the project, topics that should be addressed in the software development plan, and other areas as assigned by the project.
- The SQA group verifies that plans, standards, and procedures are in place and can be used to review and audit the software project.

4. The SQA group reviews the software engineering activities to verify compliance:

- The activities are evaluated against the software development plan and the designated software standards and procedures.
- Deviations are identified, documented, and tracked to closure.
- Corrections are verified.

5. The SQA group audits designated software work products to verify compliance.

- The deliverable software products are evaluated before they are delivered to the customer.
- The software work products are evaluated against the designated software standards, procedures, and contractual requirements.
- Deviations are identified, documented, and tracked to closure
- Corrections are verified.

6. The SQA group periodically reports the results of its activities to the software engineering group.

7. Deviations identified in the software activities and software work products are documented and handled according to a documented procedure. This procedure typically specifies that:

- Deviations from the software development plan and the designated project standards and procedures are documented and resolved with the appropriate software task leaders, software managers, or project manager, where possible.
- Deviations from the software development plan and the designated project standards and procedures not resolvable with the software task leaders, software managers, or project manager are documented and presented to the senior manager designated to receive non-compliance items.
- Non-compliance items presented to the senior manager are periodically reviewed until they are resolved.
- The documentation of non-compliance items is managed and controlled

8. The SQA group conducts periodic reviews of its activities and findings with the customer’s SQA personnel, as appropriate.

Measurement: Measurements are made and used to determine the cost and schedule status of the SQA activities.

Three Verifications:
1. The SQA activities are reviewed with senior management on a periodic basis.

2. The SQA activities are reviewed with the project manager on both a periodic and event-driven basis.

3. Experts independent of the SQA group periodically review the activities and software work products of the project’s SQA group.

Sample Checklist of SQA audit:

1. Is the Work order Amended for the change in PM or extension of Project?

2. Are the latest versions of Development Plans and Estimates Approved?

3. Is the schedule updated?

4. Are the planned efforts for the current phase (from schedule), consistent with the effort distribution on SQAP and the latest approved estimate?

5. Is the actual start and end dates for the activities completed since last SQA audit?

6. Are the Internal Review meetings held and Minutes of Meeting prepared?

7. Are the minutes of meeting circulated to affected support groups?

8. Are the Efforts captured in the effort capturing tool?

9. Is there any evidence for implementing the corrective action identified on the previous metrics sheets?

10. Are the metrics tracked and the metrics sheet updated, as per the frequency of analysis in Software Quality Assurance Plan?

11. Is the Master List of Project Document updated?

12. Are Change Control Forms available for the changes in Configurable Items?

13. Are staffing Plan, Risk Plan updated?

14. Are weekly status reports being sent to Sr. Management?

15. Are status reports being sent to customer as per Software Project Management Plan?

16. Is project back up taken? Is any Back up restoration check conducted?

17. Has DAR (Decision Analysis and Resolution) been performed as planned?

18. Has CAR (Casual Analysis and Resolution) been performed at end pf phases?

19. Is Requirements Trace ability report updated?

20. Are the relevant audits conducted for completed activities?

21. Are peer review records available for the work products developed since last audit? Are the peer review findings tracked to closure?

22. Are updated test cases with actual results available?

23. Are the tools, standards and templates used for the current development activity, identified in the relevant plans?

24. Are the Non compliances for the audits last conducted closed?

10 Open Source Mobile Test Automation Tools

noreply@blogger.com (The BASS) — Tue, 17 Feb 2015 10:57:00 +0000

Here is a list of top ten open source mobile test automation tools to allow you to automate mobile apps and mobile web. Some open source tools for mobile automation can be used for both Android and iOS as well as Native, Web and Hybrid.

Appium(Android and iOS)

Appium is an open-source tool for automating native, mobile web, and hybrid applications on iOS and Android platforms.Native apps are those written using the iOS or Android SDKs.Mobile web apps are web apps accessed using a mobile browser (Appium supports Safari on iOS and Chrome or the built-in ‘Browser’ app on Android). Hybrid apps have a wrapper around a “webview” — a native control that enables interaction with web content. Download Appium

Calabash (Android and iOS)

Calabash is an automated acceptance testing framework for mobile apps. Calabash could be compared to Selenium WebDriver. However, it is important to realize that interacting with a web app from a desktop computer is vastly different than interacting with a native app using a touch screen. Calabash provides APIs that are specialized to native apps running on touch screen devices.Download Calabash

Frank (iOS)

Frank allows you to write structured text test/acceptance tests/requirements (using Cucumber) and have them execute against your iOS application. Frank also includes a powerful “app inspector” (called Symbiote) that you can use to get detailed information on your running app. Download Frank

MonkeyTalk (Android and iOS)

MonkeyTalk automates real, functional interactive tests for iOS and Android apps – everything from simple “smoke tests” to sophisticated data-driven test suites. Native, mobile, and hybrid app, real devices or simulators. Download MonkeyTalk

iOS UI Automation (iOS)

Use the Automation instrument to automate user interface tests for your iOS app through test scripts that you write. These scripts simulate user actions by calling UI Automation, a JavaScript programming interface that specifies actions to be performed in your app as it runs. During the tests, the system returns log information to you.
When you automate tests of UI interactions, you free critical staff and resources for other work. In this way you minimize procedural errors and shorten the amount of time needed to develop product updates. More information

Robotium (Android)
Robotium is an Android test automation framework that has full support for native and hybrid applications. Robotium makes it easy to write powerful and robust automatic black-box UI tests for Android applications. With the support of Robotium, test case developers can write function, system and user acceptance test scenarios, spanning multiple Android activities. Download Robotium

iOS-driver (iOS)
Automate any IOS native, hybrid, or mobile web application using the Selenium / WebDriver API. ios-driver is fully compatible with the Selenium / Webdriver API. IOS automation is therefore as easy as automation for a browser.
ios-driver fully integrates with Selenium Grid so you can reuse your existing web automation infrastructure including your helper and utility classes (i.e. data creation, page objects etc.)
Download iOS-driver

Ui Automator (Android)

The uiautomator testing framework lets you test your user interface (UI) efficiently by creating automated functional UI testcases that can be run against your app on one or more devices.
The uiautomator API is bundled in the uiautomator.jar file under the /platforms/ directory. The API includes these key classes, interfaces, and exceptions that allow you to capture and manipulate UI components on the target app. More information

KeepItFunctional (iOS)

KIF, which stands for Keep It Functional, is an iOS integration test framework. It allows for easy automation of iOS apps by leveraging the accessibility attributes that the OS makes available for those with visual disabilities.
KIF builds and performs the tests using a standard XCTest testing target. Testing is conducted synchronously in the main thread (running the run loop to force the passage of time) allowing for more complex logic and composition. Download KIF

Selendroid (Android)

Selendroid is a test automation framework which drives off the UI of Android native and hybrid applications (apps) and the mobile web. Tests are written using the Selenium 2 client API.Download Selendroid

Automated Mobile Testing Tools

noreply@blogger.com (The BASS) — Tue, 17 Feb 2015 10:49:00 +0000

Automating the testing of mobile applications is especially challenging as mobile devices are less accessible and less open than standard desktop environments or web-based applications. Vendors of automated testing tools usually offer specialized modules or additional tools as part of their products designed for mobile applications. There are also a couple of free open source tools that you can use to automate your mobile application tests.

Commercial

DeviceAnywhere

DeviceAnywhere is available in multiple editions and allows you to test your apps and websites on different devices and automate your mobile tests.

Commercial

Ranorex

Ranorex can be used to automatically test mobile applications on iOS, Android and Windows 8 devices. Ranorex allows you to directly record your tests on your device to build your tests.

Commercial

eggPlant

eggPlant is another test automation tool that also supports various mobile development platforms such as iOS, Android and Windows Mobile/Phone.

Commercial

Silk Mobile

Silk Mobile is a test automation tool for web, iOS, Android, Blackberry and Windows Phone devices, which allows you to build and run your automated tests without having to jailbreak your devices.

Commercial

SeeTest

SeeTest is a test automation tool for iOS, Android, Blackberry and Windows Phone. It allows you to record your tests on real devices and can be used to build cross-platform test automation suites.

Open Source

MonkeyTalk

MonkeyTalk is an open source test automation tool for mobile devices. It can be used to automate real, functional interactive tests for native, mobile, and hybrid iOS and Android apps, everything from simple smoke tests to more complex data-driven test suites.

Open Source

NativeDriver

NativeDriver is an implementation of the WebDriver API which drives of the UI of a native application rather than a web application. The project is working on drivers for Android, iOS and Windows/Windows Phone devices and applications.

OPEN SOURCE! THE TOP 10 IOS AND ANDROID AUTOMATION TOOLS

noreply@blogger.com (The BASS) — Tue, 17 Feb 2015 10:41:00 +0000

Appium (iOS/Android) is a splendid open source tool used for automation of native and web as well as hybrid apps. What I mean by that is the fact that it is good for automation of apps written on original Android or iOS SDKs (native), web applications that are accessible via a mobile browser (web) and applications that are wrapped around the ‘webview’- the native control feature which enables various interactions with web contents (hybrid).
Calabash (iOS/Android) is an acceptance testing automation framework. It may be compared with the famous Selenium Web Driver. Yet every tester has to realize the difference between testing the app on a desktop machine and on a touchscreen devise. Especially when dealing with a native application. Calabash may provide the necessary API full of ‘touchscreening’ experiences.
Frank (iOS) actually allows you to write structure text tests as well as acceptance tests and requirements (via Cucumber) and all that fun will be executed against your iOS app. You can also get all the required info about your running app via Frank’s Symbiote, the app inspector in other words.
MonkeyTalk (iOS/Android) can automate all possible real and functional interactive tests. Everything from little “Smoke Tests” to the tough data-driven test suits. It does bot real device and simulation tests with native and hybrid as well as mobile apps.
iOS UI Automation (for iOS) This one is an instrument. The one you can use for automating your user interface tests through some written test-scripts.
Robotium (Android) A great tool for both native and hybrid apps. Robotium makes black box testing fairly easier. With it test case developers are free to write function and system as well as user acceptance test scenarios.
iOS Driver (iOS) is capable of complete integration with the Selenium grid. Thus you will have the possibility of reusing the already existing wen automation infrastructure. This also includes the helper as well as the utility cases.
UI Automator (Android) This UI Automation framework allows you to test your UI via creation of automated functional test cases. They will be able of running against your app on a single or multiple devices.
KeepItFunctional (iOS) KIF is an integration test framework. KIF is building and performing cases with a standard XCTest testing target.
Selendroid (Android) is but one more test automation framework. It is driven of the UI of both native and hybrid apps as well as the mobile web. So the tests will be written via Selenium 2 client API.

15 Best Test Management Tools for Software Testers

noreply@blogger.com (The BASS) — Tue, 17 Feb 2015 04:04:00 +0000

Test Management encompasses anything and everything that we do as testers. Our day-to-day activities include:

Creating and maintaining release/project cycle/component information
Creating and maintaining the test artifacts specific to each release/cycle that we have- requirements, test cases, etc.
Establishing traceability and coverage between the test assets
Test execution support – test suite creation, test execution status capture, etc.
Metric collection/report-graph generation for analysis
Bug tracking/defect management

1) QMetry:

QMetry is a total test management tool that you can use to create requirements, test cases (test suites) that can be run on multiple platforms and defects. It integrates seamlessly with many leading bug-tracking and automation environments making a good candidate for use in most situations. It is a commercial product, with a 30 day free trial available. Visit the site for more information at:http://www.qmetry.com/

2) TestRail:

TestRail is a centralized test case management tool – you can use it to create test cases and test suites, track execution and report metrics. Additionally, it integrates with many issue tracking tools that makes requirements from external systems to be linked to test cases in TestRail; bugs can also be created in the external systems and links can be established to the corresponding test case. It comes with a HTTP-based API to integrate with the automated test results. One most common integration is with Gemini, which is a incident/ticket management system(supports agile too). It is a commercial product with a free trial available at:http://www.gurock.com/testrail/

3) JIRA:

JIRA is tool that makes an appearance anytime there is a discussion on any-management process- for all the right reasons. JIRA has 2 add-ons that support the test management process.

a) Zephyr: All the aspects that you would expect of a typical tool of this type are supported. You can create tests/test suites/test cycles/bugs/reports and so on. You could have an additional add-on, ZAPI for automation integration. Along with the initial JIRA license you would have to pay for Zephyr to use it. ($10 for 10 users a month). There is a free trail available too. Check out information about it at:https://marketplace.atlassian.com/plugins/com.thed.zephyr.je

b) Go2Group SynapseRT: This tool has all the test management features but the primary focus is on requirement based testing. It can be used for projects where it makes more sense to track your progress in terms of the completion and/or success-failure of a certain requirement as opposed to test cases. Traceability is of higher priority with this add-on. Along with the initial JIRA license you would have to pay for this add-on too. ($10 for 10 users a month). There is a free trail available at:https://marketplace.atlassian.com/plugins/com.go2group.jira.plugin.synapse

4) XQual

XQual delivers one of the most advanced Test Management solution. From this unique tool, you’ll be able to manage your releases, requirements (incl. risks), specifications, tests, campaigns and bugs.

It can be integrated with all the continuous integration platforms and it can drive ANY kind of test: 5 different interfaces for manual testing and 40+ connectors with all the best automation frameworks on the market: Selenium, QTP, AutoIt, JMeter, Ranorex, TestComplete, JUnit, TestPartner, Sahi and many more. Integration with many third-party requirement or bug-tracking systems is also possible. There is also a free community edition available here:http://www.xqual.com/support/download.html

4) qTest:

Developed by QASymphony, qTest is one of the cloud-based test management tools that has all the typical key features. With the help of qTest Connector, it can integrate with JIRA for an entire end-to-end QA solution – but that is not all, it also integrates with other tools like Bugzilla, FogBugz, Rally etc. It is not open sourced but very affordable. Check out information and pricing at:http://www.qasymphony.com/qtest.html

5) TestLodge:

This is a comprehensive test case management tool that has 4 key aspects – Test Plan, requirements, Test suites/cases and test runs. So, as you can see, it has everything it needs to be able to manage test cases for you. For all the other operations, it integrates with the many main stream incident/issue management tools to provide a comprehensive solution. It is a commercial product, for a free trial visit: http://www.testlodge.com/

6) HP ALM/Quality center:

HP QC has been one of the most used test management softwarefor many years. It has all the features necessary and in many ways, it is the standard against which the other tools are measured. Even though it is one of the high-end tools, economically, it still remains to be very popular. Check out http://www8.hp.com/in/en/software-solutions/software.html?compURI=1172141 for trial and other information.

7) Zephyr:

This is the same Zephyr that we discussed earlier as being an add-on to JIRA. The reason why it deserves an independent discussion is because it can be used as an individual product also. For free trial and more info: http://www.getzephyr.com/

8) Testuff:

Testuff is Saas Test Management tool that has many cool features. The typical test case management features are a given. Besides that, it has a cool video upload capability for a defect. Integrates with a huge list of bug-trackers, some of which are, Bugzilla, JIRA, YouTrack, Mantis among others. It has an API that supports automation tools like QTP, Rational Robot, Selenium, TestComplete etc. Best of all, it is very affordable. Check out the features and pricing information at: http://www.testuff.com/

9) Test Collab:

A web based test case management tool that states its speed to be one of its key features due to its ‘Ajax’ified interface(in the exact words of the software makers). It is simple to use, integrates with all major bug/issue/incident trackers. Customizable and has a good reporting facility. It is a commercial product and information is available at: http://testcollab.com/

10) Gemini:

One of the key components of this tool is supporting ‘Testing & QA’ along with the other aspects like Project Planning, issue tracking etc. Using this tool you could create test plans, test cases, test runs, traceability, test run reports etc. There are also various integrations and extensions available. It is a commercial product with a free starter pack available at:http://www.countersoft.com/solutions/testing/

11) PractiTest:

------------

It is an end-to-end QA and Test management software. You can organize your requirements, create tests, run tests, tracks bugs etc, using this tool. It integrates very well with three of the leading incident management tools like JIRA, Bugzilla and RedMine. It is not open sourced but is quite affordable. For more feature and pricing information, check: http://www.practitest.com/

12) TestLink:

This is one of the very few open source test management toolsavailable for use in the market. It is a web- based tool with typical features like, requirement management, test case creation and maintenance, test runs, tracking bugs, reports, integration with common issue trackers etc. To download, visit:http://sourceforge.net/projects/testlink/

13) QAComplete:

QAComplete is one of the most powerful test management tools that we have. This suits agile/traditional, manual/automation projects excellently. You could use it in integration to QTP and TestComplete. For automation projects, you could schedule the test runs and run them remotely on any registered hosts. There is also a detailed release management feature that provides for better analysis. It works along with major bug-trackers and with source control tools – Subversion, Perforce and CVS. Given all the features, it is a little pricey. There is a free trial available though. Check out all its features at: http://smartbear.com/products/qa-tools/test-management/

14) Silk Central:

This is a test management product by Borland. Once you have Silk Central with you there is a nothing left for you to ask for anything else. It is robustness personified, though it is pricey. All the features for coverage, traceability, reporting, test creation, running – it has got them. It integrates with many source control and issue tracking system. There are additional plug-ins to extend its capabilities to automation testing using QTP, WinRunner etc. It comes with a video capture feature and supports SAP testing. This product is really cool. Try it at: http://www.borland.com/products/silkcentral/

15) IBM Rational Quality Manager:

A test Management product that has all the typical features – Test planning, test design, test execution, tracking and reporting. It integrates with many of the rational products for automation, source control and bug tracking activities. It is a commercial product. Check out its features, pricing and other information at: http://www-03.ibm.com/software/products/en/ratiqualmana

Additional tools:

The below are some more tools that are worth mentioning:

16) VersionOne: A commercial product primarily catering to agile projects, this has a test management module along with planning, reporting and others, with all the typical features. Check out:http://www.versionone.com/product/agile-test-management/

17) TestPad: This tool’s motto is “Spend more time actually testing”. The primary concept of this tool is – checklists. In your test plan you could have a series of checklists (tests) that can be as detailed or as short as possible. It is perfect for exploratory testing. It is commercial and you can try it at: https://ontestpad.com/

18) Aptest: a web based, commercial product that has all the typical features you would expect along with the typical integrations with issue trackers. http://www.aptest.com/atm2/

19) SpiraTest: A complete QA solution is what this product is. Instead of having your requirements, tests, defects in separate systems, this tool has it all in one place. It integrates with unit and automation testing frameworks among other things. It is commercial, though not very expensive.https://www.inflectra.com/SpiraTest/Default.aspx

20) Meliora TestLab: This is a simple to use product for requirement management, test-creating & running, defect management, workflow optimization, and integration with JIRA etc. Commercial again, check it out at: https://www.melioratestlab.com/

21) SmarteQM: A complete life cycle management tool that provides the complete end-to-end test process support and integrates with other SmartSoft functional test tools. It is web based and commercial.http://www.smartesoft.com/products_smarteQM.php

22) Test Run: a web based commercial test management process that is easy and simple to use. It has all that you need to create test plans, execute them successfully and report. Integrates with JIRA and LightHouse. http://runtestrun.com/

23) Test Wave: A test management tool that needs no installation, is web based, simple and commercial (affordable). This tool provides you a facility to import your already existing requirements/test assets from excel sheets. It also comes with an inbuilt defect trackerhttp://www.testwave.co.uk/

24) Enterprise Tester: This test management tool supports both agile and traditional projects. Integrates with JIRA and also works great for automation testing with QTP, Selenium, RFT etc. This is a really cool commercial tool. Check it out at:http://enterprisetester.com/

25) TestLog: A very comprehensive end-to-end test management tool that is easy to install and configure due to its XML database. It allows documentation of both automation and manual test cases. It also comes with a web interface for remote access. This product is commercial too. http://www.testlog.com/

26) QaTraq: An open source test process control tool that can be used to create test cases, running them, recording results etc.http://sourceforge.net/projects/qatraq/

Roles and responsibilities of SQA in CMMI

noreply@blogger.com (The BASS) — Tue, 10 Feb 2015 07:38:00 +0000

SQA (Software Quality Assurance) Engineer in an Organization going for CMMI Assessment has various responsibilities. SQA Team Structure depends on the overall strength of the organization and number of people involved with process implementations (in Software Development or Services Organization). SQA is the mainly responsible for leading the Quality Initiative in the CMMI organization and is responsible for conducting many associated process improvement activities in the organization.

SQA is involved with these CMMI Process Improvement activities:

Process Improvement Planning
Conducting Audit of products, projects and services
Development, modification and updating of a CMMI Compliant QMS
Implementation Checks and Audits and the Follow-up Audits
Conducting Training for Process Implementers according to their roles
Training to Project Team Members on core activities like Project Management, Estimation, Reviews etc.
Metrics Analysis Report preparation
Preparation for SCAMPI Assessment Documentation and Training

All these activities are targeted towards the Continuous Process Improvement in the organization using CMMI Framework.

SEPG/QAG qualities required for CMMI

noreply@blogger.com (The BASS) — Tue, 10 Feb 2015 07:37:00 +0000

SEPG/QAG Group, here are some of the qualities required in a SEPG Team Member in a CMMI Organization:

CMMI Knowledge is an added advantage. Knowledge of other Quality Framework and Standards makes you a pro.
Subject Matter Expertise – you should have good knowledge and skill set in your field. It may in Software Development, Requirements Gathering, Software Design, Testing, Quality or any other field. But should understand Software Development Life Cycle models.
Should have worked with good number of projects and people.
Understand Processes and know about Process Improvement. Should have experience in assisting project teams in process implementation.
Excellent communication and interpersonal skills.
Good Team Player.
Should be self motivated and capable of motivating others.
Should be enthusiast about the process improvement and the organization.
Should take part in SEPG meetings actively.
Should complete the tasks on time, assigned by SEPG Team.

CMMi Appraisals

noreply@blogger.com (The BASS) — Wed, 07 Jan 2015 11:27:00 +0000

Appraisals

Broadsword makes appraisals easier for you. We’re a CMMI Institute Partner licensed to perform all classes of SCAMPI Appraisals for CMMI-DEV and CMMI-SVC.

We’ve been there – right where you are. Every one of our consultants has been an engineer, a software developer, or a project manager and knows what it’s like to adopt processes and be appraised. Our approach to appraisals is just enough – not too much.

Though proud of our advanced degrees and training from prestigious universities, the Broadsword team takes a practical, not academic, approach to problem solving. Because we’ve seen engineering work done so many different ways over the years, we understand what you go through every day. As a result, we don’t try to fit you into the CMMI model. We make the CMMI fit into you.

And, if you are an agile organization, we can help you understand how well your company’s implementation of agile methods and Scrum is working. Our unique appraisal approach gets rid of the guesswork and provides a detailed, quantitative assessment.

Make it easier. Contact Broadsword.

Our Appraisal offerings include:

SCAMPI A, B, and C Appraisals
CMMI-DEV or CMMI-SVC Appraisals
CMMI Mini-Appraisal
Process Diagnostics
Change Readiness Assessment
Agile Appraisals
Scrum Appraisals

SCAMPI Appraisals

Appraisals are designed to help your organization identify operational strengths and weaknesses. The formal method used to evaluate an organization’s performance is the Standard CMMI Appraisal Method for Process Improvement (SCAMPI).

There are three “classes” of SCAMPI Appraisals – SCAMPI A, SCAMPI B, and SCAMPI C. Broadsword is licensed to deliver all three.

Most of our clients “cycle” through multiple appraisals, starting with SCAMPI C, as a way to mitigate risks and identify gaps, followed by a SCAMPI B “test run,” and then a SCAMPI A – the most rigorous class of appraisal and the only one that results in a formal CMMI Maturity Level or Capability Level Rating.

SCAMPI A

A SCAMPI A is the only class of appraisal that results in a “maturity” or “capability” rating.

Organizations wishing to attain a CMMI Rating (“Level”) must conduct a SCAMPI A Appraisal. A SCAMPI A is a team-based event which is led by an external “Certified SCAMPI Lead Appraiser” who has been certified by the CMMI Institute and Software Engineering Institute (SEI) to perform SCAMPI Appraisals.

For more information, go to SCAMPI A.

SCAMPI B

A SCAMPI B is also a formal appraisal, though it does not result in a rating (or “level”). This appraisal serves as a rigorous risk management tool to give you the information you need to completely understand your current state relative to the CMMI. In particular, the SCAMPI Bappraisal will provide you with enough data to determine whether you are likely to succeed in the formal SCAMPI A.

SCAMPI C

A SCAMPI C is primarily used as a “gap analysis” tool and does not result in a rating. It is a tremendously useful tool regardless of your desire to achieve a CMMI rating. The SCAMPI C is first and foremost a way for your company to find out how you stack up against the CMMI model and other companies that are using it.

CMMI Mini-Appraisal

A CMMI Mini-Appraisal is a low cost alternative when you are just looking for solid information about the process performance of your organization. It will give you the information you need to start improving – right away.

CMMI Micro-Assessment

Take our “CMMI Micro-Assessment” and find out where you stand!

Appraisal Sequence

Each of our clients has unique goals, and we will help you customize your own plan and approach. The most typical sequence of events includes:

SCAMPI C Appraisal
Process Improvement Program Planning
CMMI Training
Design, Develop, and Deploy process architecture and components
Training in “techniques” (estimating, planning, code reviews, etc.)
Appraisal Planning
Readiness Review with go/no-go decision
SCAMPI Appraisal Team Training
SCAMPI B Appraisal
Performance Improvement Planning
Process “Tune-up”
Readiness Review
SCAMPI A Appraisal
Celebrate an awesome accomplishment!

CMMI appraisal method by SEI - SCAMPISM A, SCAMPISM B, SCAMPISM C

noreply@blogger.com (The BASS) — Wed, 07 Jan 2015 04:26:00 +0000

CMMI appraisal method by SEI

What is SCAMPISM A, SCAMPISM B, SCAMPISM C? Are these related to certification?

SCAMPI A (Standard CMMI Appraisal method for process improvements) is CMMI appraisal method. SCAMPI is developed by SEI and there are three variants of method (SCAMPI A, SCAMPI B and SCAMPI C). Through SCAMPI A the organization’s maturity or capability levels could be determined (along with process improvement opportunities) and SCAMPI B and SCAMPI C are less rigorous and primarily focus on finding out process improvement opportunities or to find gaps in the present processes.SCAMPI methods are presently being used globally for performing internal appraisals for process improvements. SEI also maintains an appraiser program, and through extensive training authorizes individuals to perform SCAMPI across through SEI Transition partners. SEI also maintains a list of active lead appraisers and the list is accessible to all from SEI website.

CMMI - Maturity Levels

noreply@blogger.com (The BASS) — Tue, 06 Jan 2015 14:07:00 +0000

A maturity level is a well-defined evolutionary plateau toward achieving a mature software process. Each maturity level provides a layer in the foundation for continuous process improvement.

CMMI models with staged representation, have five maturity levels designated by the numbers 1 through 5. They are:

Initial
Managed
Defined
Quantitatively Managed
Optimizing

CMMI Staged Representation Maturity Levels

The following image shows the maturity levels in a CMMI staged representation.

Now we will learn the details about each maturity level. Next section will list down all the process areas related to these maturity levels.

Maturity Level Details

Maturity levels consist of a predefined set of process areas. The maturity levels are measured by the achievement of the specific and generic goals that apply to each predefined set of process areas. The following sections describe the characteristics of each maturity level in detail.

Maturity Level 1 Initial

At maturity level 1, processes are usually ad hoc and chaotic. The organization usually does not provide a stable environment. Success in these organizations depend on the competence and heroics of the people in the organization and not on the use of proven processes.

Maturity level 1 organizations often produce products and services that work; however, they frequently exceed the budget and schedule of their projects.

Maturity level 1 organizations are characterized by a tendency to over commit, abandon processes in the time of crisis, and not be able to repeat their past successes.

Maturity Level 2 Managed

At maturity level 2, an organization has achieved all the specific and generic goals of the maturity level 2 process areas. In other words, the projects of the organization have ensured that requirements are managed and that processes are planned, performed, measured, and controlled.

The process discipline reflected by maturity level 2 helps to ensure that existing practices are retained during times of stress. When these practices are in place, projects are performed and managed according to their documented plans.

At maturity level 2, requirements, processes, work products, and services are managed. The status of the work products and the delivery of services are visible to management at defined points.

Commitments are established among relevant stakeholders and are revised as needed. Work products are reviewed with stakeholders and are controlled.

The work products and services satisfy their specified requirements, standards, and objectives.

Maturity Level 3 Defined

At maturity level 3, an organization has achieved all the specific and generic goals of the process areas assigned to maturity levels 2 and 3.

At maturity level 3, processes are well characterized and understood, and are described in standards, procedures, tools, and methods.

At maturity level 3, the standards, process descriptions, and procedures for a project are tailored from the organization's set of standard processes to suit a particular project or organizational unit. The organization's set of standard processes includes the processes addressed at maturity level 2 and maturity level 3. As a result, the processes that are performed across the organization are consistent except for the differences allowed by the tailoring guidelines.

Another critical distinction is that at maturity level 3, processes are typically described in more detail and more rigorously than at maturity level 2. At maturity level 3, processes are managed more proactively using an understanding of the interrelationships of the process activities and detailed measures of the process, its work products, and its services.

Maturity Level 4 Quantitatively Managed

At maturity level 4, an organization has achieved all the specific goals of the process areas assigned to maturity levels 2, 3, and 4 and the generic goals assigned to maturity levels 2 and 3.

At maturity level 4, sub-processes are selected that significantly contribute to the overall process performance. These selected sub-processes are controlled using statistical and other quantitative techniques.

Quantitative objectives for quality and process performance are established and used as criteria in managing the processes. Quantitative objectives are based on the needs of the customer, end users, organization, and process implementers. Quality and process performance are understood in statistical terms and are managed throughout the life of the processes.

For these processes, detailed measures of process performance are collected and statistically analyzed. Special causes of process variation are identified and, where appropriate, the sources of special causes are corrected to prevent future occurrences.

Quality and process performance measures are incorporated into the organization's measurement repository to support fact-based decision making in the future.

A critical distinction between maturity level 3 and maturity level 4 is the predictability of process performance. At maturity level 4, the performance of processes is controlled using statistical and other quantitative techniques, and is quantitatively predictable. At maturity level 3, processes are only qualitatively predictable.

Maturity Level 5 Optimizing

At maturity level 5, an organization has achieved all the specific goalsof the process areas assigned to maturity levels 2, 3, 4, and 5 and the generic goals assigned to maturity levels 2 and 3.

Processes are continually improved based on a quantitative understanding of the common causes of variation inherent in processes.

This level focuses on continually improving process performance through both incremental and innovative technological improvements.

The quantitative process-improvement objectives for the organization are established, continually revised to reflect changing business objectives, and used as criteria in managing process improvement.

The effects of deployed process improvements are measured and evaluated against the quantitative process-improvement objectives. Both the defined processes and the organization's set of standard processes are targets of measurable improvement activities.

Optimizing processes that are agile and innovative, depends on the participation of an empowered workforce aligned with the business values and objectives of the organization. The organization's ability to rapidly respond to changes and opportunities is enhanced by finding ways to accelerate and share learning. Improvement of the processes is inherently a role that everybody has to play, resulting in a cycle of continual improvement.

A critical distinction between maturity level 4 and maturity level 5 is the type of process variation addressed. At maturity level 4, processes are concerned with addressing special causes of process variation and providing statistical predictability of the results. Though processes may produce predictable results, the results may be insufficient to achieve the established objectives. At maturity level 5, processes are concerned with addressing common causes of process variation and changing the process (that is, shifting the means of the process performance) to improve process performance (while maintaining statistical predictability) to achieve the established quantitative process-improvement objectives.

Maturity Levels Should Not be Skipped

Each maturity level provides a necessary foundation for effective implementation of processes at the next level.

Higher level processes have less chance of success without the discipline provided by lower levels.
The effect of innovation can be obscured in a noisy process.

Higher maturity level processes may be performed by organizations at lower maturity levels, with the risk of not being consistently applied in a crisis.

Maturity Levels and Process Areas

Here is a list of all the corresponding process areas defined for a software organization. These process areas may be different for different organization.

This section provides the names of the related process areas. For more details about these Process Areas go through the CMMI Process Areas Chapter.

Level	Focus	Key Process Area	Result
5 Optimizing	Continuous Process Improvement	Organizational Innovation and Deployment Causal Analysis and Resolution	Highest Quality / Lowest Risk
4 Quantitatively Managed	Quantitatively Managed	Organizational Process Performance Quantitative Project Management	Higher Quality / Lower Risk
3 Defined	Process Standardization	Requirements Development Technical Solution Product Integration Verification Validation Organizational Process Focus Organizational Process Definition Organizational Training Integrated Project Mgmt (with IPPD extras) Risk Management Decision Analysis and Resolution Integrated Teaming (IPPD only) Org. Environment for Integration (IPPD only) Integrated Supplier Management (SS only)	Medium Quality / Medium Risk
2 Managed	Basic Project Management	Requirements Management Project Planning Project Monitoring and Control Supplier Agreement Management Measurement and Analysis Process and Product Quality Assurance Configuration Management	Low Quality / High Risk
1 Initial	Process is informal and Adhoc		Lowest Quality / Highest Risk

SEI CMMI - Representations

noreply@blogger.com (The BASS) — Tue, 06 Jan 2015 14:03:00 +0000

A representation allows an organization to pursue different improvement objectives. An organization can go for one of the following two improvement paths.

Staged Representation

The staged representation is the approach used in the Software CMM. It is an approach that uses predefined sets of process areas to define an improvement path for an organization. This improvement path is described by a model component called a Maturity Level. A maturity level is a well-defined evolutionary plateau towards achieving improved organizational processes.

CMMI Staged Representation:

Provides a proven sequence of improvements, each serving as a foundation for the next.
Permits comparisons across and among organizations by the use of maturity levels.
Provides an easy migration from the SW-CMM to CMMI.
Provides a single rating that summarizes appraisal results and allows comparisons among organizations.

Thus Staged Representation provides a pre-defined roadmap for organizational improvement based on proven grouping and ordering of processes and associated organizational relationships. You cannot divert from the sequence of steps.

CMMI Staged Structure

Following picture illustrates CMMI Staged Model Structure.

Continuous Representation

Continuous representation is the approach used in the SECM and the IPD-CMM. This approach allows an organization to select a specific process area and make improvements based on it. The continuous representation uses Capability Levels to characterize improvement relative to an individual process area.

CMMI Continuous Representation:

Allows you to select the order of improvement that best meets your organization's business objectives and mitigates your organization's areas of risk.
Enables comparisons across and among organizations on a process-area-by-process-area basis.
Provides an easy migration from EIA 731 (and other models with a continuous representation) to CMMI.

Thus Continuous Representation provides flexibility to organizations to choose the processes for improvement, as well as the amount of improvement required.

CMMI Continuous Structure

The following picture illustrates the CMMI Continuous Model Structure.

Continuous vs Staged Representations

Continuous Representation	Staged Representation
Process areas are organized by process area categories.	Process areas are organized by maturity levels.
Improvement is measured using capability levels. Capability levels measure the maturity of a particular process across an organization; it ranges from 0 through 5.	Improvement is measured using maturity levels. Maturity levels measure the maturity of a set of processes across an organization: it ranges from 1 through 5.
There are two types of specific practices: base and advanced. All specific practices appear in the continuous representation.	There is only one type of specific practice. The concepts of base and advanced practices are not used. All specific practices appear in the staged representation except when a related base-advanced pair of practices appears in the continuous representation, in which case only the advanced practice appears in the staged representation.
Capability levels are used to organize the generic practices.	Common features are used to organize generic practices.
All generic practices are included in each process area.	Only the level 2 and level 3 generic practices are included.
Equivalent staging allows determination of a maturity level from an organization's achievement profile.	There is no need for an equivalence mechanism to back the continuous representation because each organization can choose what to improve and how much to improve using the staged representation.

Which Representation is Better?

Each representation has its advantages over the other, some organizations use both representations to address particular requirements at various times in their improvement programs.

Organizational maturity is the focus of the staged representation, whereas process area capability is the focus of the continuous representation.

Organizational maturity and process area capability are similar concepts. The difference between them is that organizational maturity pertains to a set of process areas across an organization, while process area capability deals with a set of processes relating to a single process area or specific practice.

The following diagram depicts both the presentations. In this diagram, ML indicates Maturity Level and PA Indicates Process Area.

CMMI – 5 Levels of Maturity

noreply@blogger.com (The BASS) — Tue, 06 Jan 2015 14:00:00 +0000

CMMI – 5 Levels of Maturity

CMMI Level 1
Level 1 is an immature state. The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort. Here there is no objective basis for judging product quality or for solving product or process problems. Therefore product quality is difficult to predict. Activities intended to enhance quality such as reviews and testing are often curtailed or eliminated when projects fall behind schedule. Below points are the characteristics of this level.

The processes within the organization are highly unstable and unpredictable
During crisis, all the processes and procedures are either abandoned or skipped
The projects are purely person dependant. i.e., when the persons involved leave the project or the company, things come to a halt. Also, the performance depends on the capabilities of the individuals rather then the organizational capability.
Even if some processes remain, they are constantly changed or modified as the work progresses. (i.e…, the process is ad hoc).
There are no process areas at Level 1. Level 1 is just a denotation of ad-hoc behavior.

CMMI Level 2
Level 2 in CMM Integrated – Managed Level. Managed, as the word means, the project management is defined at this level. Basic project management principles are established to track cost, schedule and functionality. The necessary process discipline is in place to repeat earlier success on projects. An effective process can be characterized as practiced, documented, enforced, trained, measured, and able to improve. Projects in Level 2 organizations have installed basic software management controls. Some of the important points are:

Realistic project commitments are based on the results observed on previous projects and on the requirements of the current project.
The project managers for a project track software costs, schedules, defects and functionality; problems in meeting commitments are identified when they arise.
The project’s process is under the effective control of a project management system, following realistic plans based on the performance of previous projects.
Level 2 has seven process areas. Each process area has certain goals which need to be met by the organization.

CMMI Level 3
Level 3 which is The Defined Level, looks at building engineering processes and organizational level processes using the strong base set at Level 2. Level 3 is, when an organization will have processes for Requirements gathering (Test Requirements Gathering), Design & Build (Strategize and Prepare Test Cases / Scripts), Reviews, Testing (Execute Test Cases / Scripts) etc, defined at organization level. Information and artifacts of previous projects is available for re-use within the organization through mechanisms of knowledge sharing.

There are 13 process areas in Level 3 of CMMI. Apart from engineering processes and organizational level processes, there are a few process areas focused on certain critical aspects like risk management, decision making, team work etc..

CMMI Level 4
Level 4 in CMMI is a very critical step. It is called “Quantitatively Managed” level. At this level, the organization has achieved all the goals of PAs in Level 2, 3 and 4. They key attribute of Level 4 is sub process performance. The selected sub processes are controlled using statistical and other quantitative techniques

At Level 4, process and project management happens through Quantitative techniques. Quantitative objectives are based on the needs of the client, end users, organization and process improvement. Quality and process performances are understood in statistical terms and are managed throughout the life of the processes.For the various processes, measures of process performance are collected and statistically analyzed. Special Causes of process performance are identified and corrected to prevent future occurrences.

The crucial difference between Level 3 and 4 is Predictability. At Level 4, performance of process is quantitatively predictable.

CMMI Level 5
Level 5 – The Optimizing Level, focuses on continually improving process performance through both incremental and innovative improvements. The effects of deployed process improvements are measured and evaluated. A critical distinction between Level 4 and 5, is the type of process variation that is addressed. At maturity Level 4, we look at Special cause of variation (Through Statistical Process Control or other statistical techniques).

At Level 5, we are concerned with addressing common causes of variation and changing the process (Ex – Defect & Problem Prevention)

CMMI Maturity Levels

noreply@blogger.com (The BASS) — Tue, 06 Jan 2015 13:57:00 +0000

A maturity level is a well-defined evolutionary plateau toward achieving a mature software process. Each maturity level provides a layer in the foundation for continuous process improvement.

In CMMI models with a staged representation, there are five maturity levels designated by the numbers 1 through 5

Initial
Managed
Defined
Quantitatively Managed
Optimizing

CMMI Staged Represenation- Maturity Levels

Now we will give more detail about each maturity level. Next section will list down all the process areas related to these maturity levels.

Maturity Level Details:

Maturity Level 1 - Initial

At maturity level 1, processes are usually ad hoc and chaotic. The organization usually does not provide a stable environment. Success in these organizations depends on the competence and heroics of the people in the organization and not on the use of proven processes.

Maturity level 1 organizations often produce products and services that work; however, they frequently exceed the budget and schedule of their projects.

Maturity level 1 organizations are characterized by a tendency to over commit, abandon processes in the time of crisis, and not be able to repeat their past successes.

Maturity Level 2 - Managed

At maturity level 2, requirements, processes, work products, and services are managed. The status of the work products and the delivery of services are visible to management at defined points.

Commitments are established among relevant stakeholders and are revised as needed. Work products are reviewed with stakeholders and are controlled.

The work products and services satisfy their specified requirements, standards, and objectives.

Maturity Level 3 - Defined

At maturity level 3, an organization has achieved all the specific and generic goals of the process areas assigned to maturity levels 2 and 3.

At maturity level 3, processes are well characterized and understood, and are described in standards, procedures, tools, and methods.

A critical distinction between maturity level 2 and maturity level 3 is the scope of standards, process descriptions, and procedures. At maturity level 2, the standards, process descriptions, and procedures may be quite different in each specific instance of the process (for example, on a particular project). At maturity level 3, the standards, process descriptions, and procedures for a project are tailored from the organization's set of standard processes to suit a particular project or organizational unit. The organization's set of standard processes includes the processes addressed at maturity level 2 and maturity level 3. As a result, the processes that are performed across the organization are consistent except for the differences allowed by the tailoring guidelines.

Maturity Level 4 - Quantitatively Managed

At maturity level 4 Subprocesses are selected that significantly contribute to overall process performance. These selected subprocesses are controlled using statistical and other quantitative techniques.

Quantitative objectives for quality and process performance are established and used as criteria in managing processes. Quantitative objectives are based on the needs of the customer, end users, organization, and process implementers. Quality and process performance are understood in statistical terms and are managed throughout the life of the processes.

Quality and process performance measures are incorporated into the organization.s measurement repository to support fact-based decision making in the future.

Maturity Level 5 - Optimizing

At maturity level 5, an organization has achieved all the specific goals of the process areas assigned to maturity levels 2, 3, 4, and 5 and the generic goals assigned to maturity levels 2 and 3.

Processes are continually improved based on a quantitative understanding of the common causes of variation inherent in processes.

Maturity level 5 focuses on continually improving process performance through both incremental and innovative technological improvements.

Quantitative process-improvement objectives for the organization are established, continually revised to reflect changing business objectives, and used as criteria in managing process improvement.

Optimizing processes that are agile and innovative depends on the participation of an empowered workforce aligned with the business values and objectives of the organization. The organization's ability to rapidly respond to changes and opportunities is enhanced by finding ways to accelerate and share learning. Improvement of the processes is inherently part of everybody's role, resulting in a cycle of continual improvement.

A critical distinction between maturity level 4 and maturity level 5 is the type of process variation addressed. At maturity level 4, processes are concerned with addressing special causes of process variation and providing statistical predictability of the results. Though processes may produce predictable results, the results may be insufficient to achieve the established objectives. At maturity level 5, processes are concerned with addressing common causes of process variation and changing the process (that is, shifting the mean of the process performance) to improve process performance (while maintaining statistical predictability) to achieve the established quantitative process-improvement objectives.

Maturity Levels Should Not be Skipped:

Each maturity level provides a necessary foundation for effective implementation of processes at the next level.

Higher level processes have less chance of success without the discipline provided by lower levels.
The effect of innovation can be obscured in a noisy process.

Higher maturity level processes may be performed by organizations at lower maturity levels, with the risk of not being consistently applied in a crisis.

Maturity Levels and Process Areas:

Here is a list of all the corresponding process areas defined for a S/W organization. These process areas may be different for different organization.

This section is just giving names of the related process areas, for more detail about these Process Areas go through CMMI Process Areas Chapter.

Level	Focus	Key Process Area	Result
5 Optimizing	Continuous Process Improvement	Organizational Innovation and Deployment Causal Analysis and Resolution	Highest Quality / Lowest Risk
4 Quantitatively Managed	Quantitatively Managed	Organizational Process Performance Quantitative Project Management	Higher Quality / Lower Risk
3 Defined	Process Standardization	Requirements Development Technical Solution Product Integration Verification Validation Organizational Process Focus Organizational Process Definition Organizational Training Integrated Project Mgmt (with IPPD extras) Risk Management Decision Analysis and Resolution Integrated Teaming (IPPD only) Org. Environment for Integration (IPPD only) Integrated Supplier Management (SS only)	Medium Quality / Medium Risk
2 Managed	Basic Project Management	Requirements Management Project Planning Project Monitoring and Control Supplier Agreement Management Measurement and Analysis Process and Product Quality Assurance Configuration Management	Low Quality / High Risk
1 Initial	Process is informal and Adhoc		Lowest Quality / Highest Risk

TOP 6 BENEFITS OF ADOPTING CAPABILITY MATURITY MODEL – CMMI

noreply@blogger.com (The BASS) — Mon, 22 Dec 2014 07:10:00 +0000

The quality of a software product is only as good as the process used to develop and maintain it. Whether a software organization is competing in the marketplace or trying to satisfy internal requirements, its software process is a critical success factor. Well thought out improvements to the process will significantly contribute to the organization’s performance.

Capability Maturity Model Integration (CMMI) is a process improvement training and certification program and service administered and marketed by Carnegie Mellon University and required by manyDOD and Government programs for government contracts, especially software development. Carnegie Mellon University claims CMMI can be used to guide process improvement across a project, division, or an entire organization.

SOME INTERESTING STATS

There are over 5,000 businesses that use CMMI models from over 70 countries, including the U.S., China, Germany, Italy, Australia, India, Turkey, Pakistan, Egypt,Chile, and Russia.

A research by University of Missouri – St. Louis IS6840 – Fall 2008

TOP 6 BENEFITS OF ADOPTING CMMI QUALITY FRAMEWORK?

CMMI not only rates the maturity of companies’ process, it gives a level of assurance that the company being given the work will be able to complete the job in the time and price quoted for the project.

In order for the local software development industry to become more competitive on a global scale, it will need to fall into line with international standards, so that local companies seeking international contracts will be able to meet the CMMI level specified by international companies.

Having originated in the US defense sector, CMMI is now being adopted increasingly widely to drive Business Improvement in diverse organisations.

1 – CONSISTENCY

CMMI provides a proven approach that has enabled diverse organisations to drive out real benefits in terms of dramatically improved project predictability and consistency. Whilst any or all of the above factors may drive an organisation’s initial interest in CMMI, the key benefit from implementing the model that executives focus on is consistency in delivery.

2 – COST SAVING

CMMI driven process improvement also delivers real cost savings such as earlier and more effective error detection, and hence reduced cost of remediation, more effective management of change so you spend less on re-work, reductions in schedule variability and increased cost predictability.

3 – SELF IMPROVEMENT

There is also the aspect of self improvement. Companies will be able to use CMMI as a way of differentiating themselves locally and by achieving a level of CMMI will have naturally improved their processes which will make them more competitive. The heat of competition is now driving significant interest in CMMI. Development, Service Provider and Acquisition/Outsourcing organisations are all adopting CMMI both as a differentiator and as an enabler to enhanced performance.

4 – MARKET DEMAND

Competing companies are utilizing CMMI for industry best practices and reaping the benefit of it. Companies have adopted this approach to best meet the customer demands and competition. Growing popularity of CMMI in the market is also a big reason of adoption of this practice by companies.

Subcontractors providing custom software to companies creating solutions for the federal government must either themselves be following CMMI, or be covered by their client. All the parts of the software product delivered to the government must be following CMMI somewhere in the supply chain.

5 – PERFORMANCE DEMAND

The purpose of CMMI is to improve upon the performance of the existing organizational standards, processes and procedures and NOT to redefine or them. CMMI is meant to help organizations improve on their “capability” to consistently and predictably deliver the products, services, and sourced goods their customers want, when they want them and at a price they’re willing to pay.

CMMI can be applied to create a process improvement solution appropriate to the context of each unique organization and can provide a path for an organization to achieve its performance goals.

6 – PROCESS IMPROVEMENT

A CMMI driven improvement project will deliver a framework to standardize your processes, ensuring that your business’s best practices are captured, shared and adopted so that you can move staff around your organization and leavers won’t take business critical information away with them.

CMMI includes 25 different process areas to cater comprehensive business process improvement solution. Each process area is made of two kinds of goals, two kinds of practices, and a whole lot of informative information helping management to make strategies.

WHY USE A MODEL?

Without a model of how an organization work, which functions it need, and how those functions interact, it is difficult to lead efforts to improve. A model gives us an understanding of discrete elements in an organization and helps to formulate language and discussion of what needs to be improved and how such improvement might be achieved. A model offers the following benefits:

provides a common framework and language to help communicate
leverages years of experience
helps users keep the big picture in mind while focusing specifically on improvement
is often supported by trainers and consultants
can provide a standard to help solve disagreements

CHARACTERISTICS OF THE MATURITY LEVELS

Under the CMMI methodology, processes are rated according to their maturity levels, which are defined as: Initial, Repeatable, Defined, Quantitatively Managed, Optimizing. Currently supported is CMMI Version 1.3. CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.

CMMI MODEL FRAMEWORK

3 Constellations are defined to help improve a given business need. Currently there are three (3) constellations:

Development: For improving the development of solutions.
Acquisition: For improving the purchasing of products, services and/or solutions.
Services: For improving delivery of services and creation of service systems (say, to operate a solution but not buy it or build it in the first place).

MATURITY LEVELS IN CMMI FOR DEVELOPMENT

There are five maturity levels. However, maturity level ratings are awarded for levels 2 through 5. The process areas below and their maturity levels are listed for the CMMI for Development model:

Maturity Level 2 – Repeatable
Maturity Level 3 – Defined
Maturity Level 4 – Quantitatively Managed
Maturity Level 5 – Optimizing

MATURITY LEVELS IN CMMI FOR SERVICES

The process areas below and their maturity levels are listed for the CMMI for Services model:

Maturity Level 2 – Managed
Maturity Level 3 – Defined
Maturity Level 4 – Quantitatively Managed
Maturity Level 5 – Optimizing

MATURITY LEVELS IN CMMI FOR ACQUISITION

The process areas below and their maturity levels are listed for the CMMI for Acquisition model:

Maturity Level 2 – Managed
Maturity Level 3 – Defined
Maturity Level 4 – Quantitatively Managed
Maturity Level 5 – Optimizing

SUMMARY

A high maturity organization where all individuals recognize their role and responsibility for business success is an organization that is more likely to achieve success.

What an organization gets out of process deployment and CMMI appraisals is a reflection of what the organization puts into it. Organizations that focus on a just wining the CMMI certification hardly find the real benefits of adopting the quality framework.

High maturity, with its focus on quality and process performance objectives, puts organizations and projects in a position to succeed.

Why is CMMI Appraisal Important for Software Development Companies?

noreply@blogger.com (The BASS) — Mon, 22 Dec 2014 07:05:00 +0000

As a software development professional that has been in the IT industry for over 30 years, I think it is fair to say I have a qualified perspective on the effect that process and standardization have on producing software products and services. Back in the day, however, I didn’t always understand the need for these formalities. I would ask questions like, “Is it really necessary to ‘document’ my code?” or “Do we really need to do ‘that’ on ‘all’ of our deliverables?” or say, “I can get so much more done if I didn’t need to worry about ‘dotting the I’s and crossing the T’s.’” I make sure my code is documented and tested; why do I have to consider other approaches?” It all just seemed to be a waste of time and money to me.

Well, that was me as a coder, many moons ago, and I’m sure that these types of concerns still resonate with others today. Now I have a different perspective as an owner and partner ofSegue Technologies, and the current Director of Software Engineering. There are a whole host of challenges that arise with answering those core questions while still maintaining productivity and quality across the enterprise of services and development platforms. As a business owner, I am fully cognizant of the acronym ROI (Return on Investment) and how most decisions, if not all, are based on what the ROI expectations tell us. This both applies to the operations and success of Segue, and to the value and quality of our services for our customers.

Can a balance be achieved between administration and management costs and just “cranking out the code?” Should a formal analysis be conducted and investments made with the hope of the coveted ROI? Or was I right in my junior years? As an organization, you can start to find this balance by delving into process organizations whose whole mission in life is centered on identifying everything under the sun coupled to “best practices.” You know who they are: ISO,ITIL, Six Sigma, IEEE, and a host of others. They all have white-papers and checklists that are truly inspirational and offer practical guidance in possibly achieving that necessary “balance.”

The Importance of a CMMI Appraisal

The Software Engineering Institute (SEI), at Carnegie-Melon University, was initiated by the U.S. Department of Defense. SEI has long been the organization which not only researches and recommends development improvements, processes and practices, but provides an independent and objective “appraisal” as to whether an organization is actively practicing these standards. This last fact alone justifies a CMMI appraisal from a purely compliance perspective. It has become increasingly more common to see the requirement for a development company to have had a successful CMMI appraisal conducted in order to even qualify for Federal Government contracts. Bottom line: if you do work for the Federal government, you better get started with a CMMI Appraisal, if you haven’t already!

The Benefits of CMMI Accreditation

Through experience and (dare I say) wisdom, I have seen first-hand the benefits that CMMI accreditation has brought to our organization, to include:

1. Expectation of Sustainment for the long haul

2. A balance between productivity and process by aligning our internal processes, with the Process Areas called out in the CMMI guidance.

Repeatability is the key for process success, and as they have now become the norm and not painful, everyone is in synch with the expectations.

Without an appraisal, it is doubtful that our organization would have implemented a sustainable and measurable process that everyone is aware of and that drives operations. We have seen ROI through optimization of tasks, estimation techniques, ownership/accountability, management, and how all corporate components play a role in quality through a consistent work-flow, just to list a few!

What to Expect During a CMMI Appraisal

This is where the bulk of your investment takes place. It involves all facets of your organization and requires training and management. You will also need to pay for the independent appraisal itself, from a qualified SEI appraiser. In addition, there are options to work with certified CMMI organizations that can consult and guide you through the appraisal preparation. There are also a series of tools and guides available from SEI, at no cost, that allow you to conduct the preparation without need of consultation.

In our journey toward CMMI appraisal, our organizational “pains” occurred when we embarked on the inventory and mapping of what we were currently doing, as compared to the CMMI process areas and generic and specific practices. Gaps were identified and addressed and all of those were found to add value to our organization. It was well worth it though, as we did realize ROI and can measure it routinely now!

The beauty of CMMI is its different levels of maturity. These help guide an organization to continually improve and implement practices in chunks rather than having to deal with everything at once. We can now progress and refine improvement as we mature by building on the CMMI foundation we have established. The more mature your organization becomes, following the CMMI progression/capability levels, the more you become competitive through optimization and quality. Most important to my business and to the value I can provide my customers, is continually improving ROI.

How to Achieve Level 5 Maturity for QA and Testing Process

noreply@blogger.com (The BASS) — Mon, 08 Dec 2014 07:26:00 +0000

For any process whether it is a QA process, development process or any non-technical process, there are levels of its maturity. By levels of maturity we mean that the level of formality and processes improvement, like ad-hoc processes – to formally defined steps – to managed result metrics – to optimization of the processes.

CMM (Capability Maturity Model) is process based model which is used to assess the maturity of an organization for different domains. Although this model is normally termed as the software development model but eventually it was used for other processes as well like QA and testing.

It has 5 different levels of maturity from 1 to 5. As we go towards level 5 from 1, variability and inconsistency reduces. Below are the details of 5 levels. Here we will go through the 5 CMM levels with respect to QA process and what all output/result is expected for each level to mature a QA/testing process and reach up to level 5.

Level 1 – Ad-Hoc: Unplanned, unsystematic, and inconsistent

As the word ‘Ad-Hoc’ states: unplanned, unprepared, at this level significance is not given to planning, following processes, guidelines and standards. There is no standardized & consistent way of doing any task. The only thing which is important at this level is meeting the timelines, irrespective of the quality of the end product and deliverables.

As there are no pre-defined standards and processes, same task is done in different ways by different people.

And this becomes even more unsystematic and inconsistent if same task is done differently next time.

Example -
QA – The example would be that in an organization although QA is 1 of the phases in a product life cycle but there are not any standard & no process defined, no templates for QA deliverables like plan, strategy, scenarios, and cases are standardized. Even if these are documented then all team members have their own way of doing it and not consistent at all.

Level 2 – Control: initiate defining processes at high level:

Solution to the problem which we saw at Level 1 of unavailability of QA processes, methodology & standards would be to have all these in place. The standards and processes are not only finalized but also are well documented, so that those can be re-used by any one for similar task.

Example -
QA – Define overall QA process and methodology for different types of testing like functional, data, performance etc. Define the role of a QA engineer in project’s life cycle and prepare templates for deliverables in each phase. Not only define and prepare rather share within team

Level 3 – Core Competency: Come up with a generalized process for wider audience and domains:

At this level 3, people are motivated to follow the standards and processes defined at level 2. For this first of all the processes need to be conveyed to all people and need to identify what all skills are needed to use those effectively and efficiently and also if any training is required for that and then motivated and supported to follow those standards and processes. Here people having more experience share their knowledge with others.

Example -
QA – Conduct webinars and training sessions to let people get acquainted about the newly defined QA process and standards and motivate them to make use of those during their day to day project’s life

Level 4 – Predictable: Measure the processes

At this level processes defined at level 3 are measured quantitatively. This is done to control the effort required on any task. Based on this quantitative analysis, processes can be adjusted if needed, and that to without degrading the quality of the end product. Analysis is done by dividing complete process into smaller sub-processes and then quantitative techniques are applied on these sub-processes and as per the result, sub-processes are adjusted if needed. This level is called predictable as based on prior experience; we can predict the process quantitatively and make use of that for the upcoming processes.

Example -
QA – Performing regular audits would be a good idea here. This can include to check if teams are actually following the processes defined, using the standard templates, adhere to methodology or not.

Level 5 – Innovative: Continuous Improvement

At this level, innovative ways are identified to further improve the pre-defined processes and standards. This is a continuous process. For this our own processes are watched and re-engineered continuously by adding new tools technologies, by continuous studies and by keeping ourselves updated with new information in the market. This can also be achieved by benchmarking other organizations and learn from them and try to improve our process by adding new innovations to it.

Example -
QA – Keep on improving the methodology, processes defined based on prior audit results.

Based on some studies it has been concluded that the organizations at level 1 may spend $1000 for any particular task then for the same task organization at level 5 needs to spend $10.

After going though all 5 levels mentioned above, looks like reaching up to level 3 is difficult. Once it achieved then next levels are not too far and difficult to achieve

What is SEI? CMM? ISO? IEEE? ANSI?

noreply@blogger.com (The BASS) — Mon, 08 Dec 2014 06:08:00 +0000

· SEI = ‘Software Engineering Institute’ at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.

· CMM = ‘Capability Maturity Model’, developed by the SEI. It’s a model of 5 levels of organizational ‘maturity’ that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMM ratings by undergoing assessments by qualified auditors.

Level 1 - characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if any processes in place; successes may not be repeatable.

Level 2 – software project tracking, requirements management, realistic planning, and configuration management processes are in place; successful practices can be repeated.

Level 3 – standard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is in place to oversee software processes, and training programs are used to ensure understanding and compliance.

Level 4 – metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.

Level 5 – the focus is on continuous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.

· ISO

= ‘International Organization for Standards’ – The ISO 9001, 9002, and 9003 standards concern quality systems that are assessed by outside auditors, and they apply to many kinds of production and manufacturing organizations, not just software. The most comprehensive is 9001, and this is the one most often used by software development organizations. It covers documentation, design, development, production, testing, installation, servicing, and other processes. ISO 9000-3 (not the same as 9003) is a guideline for applying ISO 9001 to software development organizations. The U.S. version of the ISO 9000 series standards is exactly the same as the international version, and is called the ANSI/ASQ Q9000 series. The U.S. version can be purchased directly from the ASQ (American Society for Quality) or the ANSI organizations. To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO 9000 certification does not necessarily indicate quality products – it indicates only that documented processes are followed.

· IEEE = ‘Institute of Electrical and Electronics Engineers’ – among other things, creates standards such as ‘IEEE Standard for Software Test Documentation’ (IEEE/ANSI Standard 829), ‘IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), ‘IEEE Standard for Software Quality Assurance Plans’ (IEEE/ANSI Standard 730), and others.

· ANSI = ‘American National Standards Institute’, the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality).

What is Capability Maturity Model Integration? (CMMI)

noreply@blogger.com (The BASS) — Wed, 03 Dec 2014 10:46:00 +0000

What is Capability Maturity Model Integration? (CMMI)

Capability Maturity Model Integration (CMMI) is a process improvement approach that provides organizations with the essential elements of effective processes.

CMMI for Development Ver 1.2 consists of 22 process areas with Capability or Maturity levels. CMMI should be adapted to each individual company, therefore companies are not "certified." A company is appraised at a certain level of CMMI.

History

The CMMI is the successor of CMM. The goal of the CMMI project is to improve usability of maturity models for software engineering and other disciplines, by integrating many different models into one framework. It was created by members of industry, government and the SEI.

Appraisal

There are three different types of appraisals, type A, B and C. In Appraisal Requirements for CMMI (ARC) the requirements for CMMI appraisal methods are described. The Standard CMMI Assessment Method for Process Improvement (SCAMPI) is the only appraisal method which meets all of the ARC requirements for a Class A appraisal method. The results of the appraisal can be published on the SEI website.

Evaluation

The SEI states that 25 organizations measured increases of performances in the categories cost, schedule, productivity, quality and customer satisfaction. The median increase in performance varied between 14% (customer satisfaction) and 62% (productivity). However, the CMMI model mostly deals with what processes should be implemented, and not so much with how they can be implemented. SEI thus also mentions that these results do not guarantee that applying CMMI will increase performance in every organization. A small company with few resources may be less likely to benefit from CMMI.

Interestingly, Turner & Jain (2002) argue that although it is obvious there are large differences between CMMI and agile methods, both approaches have much in common. They believe neither way is the 'right' way to develop software, but that there are phases in a project where one of the two is better suited. They suggest one should combine the different fragments of the methods into a new hybrid method. The combination of the project management technique Earned value management (EVM) with CMMI has been described (Solomon, 2002). To conclude with a similar use of CMMI, Extreme Programming (XP), a software engineering method, has been evaluated with CMM/CMMI (Nawrocki et al., 2002). For example, the XP requirements management approach, (which relies on oral communication), was evaluated as not compliant with CMMI.

Structure

The CMMI comes with two different representations - staged and continuous. The staged model, which groups process areas into 5 maturity levels, was also used in the ancestor software development CMM, and is the representation used to achieve a "CMMI Level Rating" from a SCAMPI appraisal. The continuous representation, which was used in the ancestor systems engineering CMM, defines capability levels within each profile. The differences in the representations are solely organizational; the content is equivalent.

The CMMI uses a common structure to describe each of the 25 process areas (PAs). A process area has 1 to 4 goals, and each goal is comprised of practices. Within the 22 PAs these are called specific goals and practices, as they describe activities that are specific to a single PA. There is one additional set of goals and practices that apply in common across all of the PAs; these are called generic goals and practices. Table 1 describes CMMI terminology in more detail. The page numbers refer to the Staged Representation. Common Features are historical artifacts from the software CMM; they do not appear in the CMMI v1.2.

CMMI concept definition list


	· Capability Level o These levels belong to the continuous representation, apply to an organization’s process-improvement achievement for each process area. There are six capability levels, numbered 0 through 5. Each capability level corresponds to a generic goal and a set of generic and specific practices. · CMMI Model o Since the CMMI Framework can generate different models based on the needs of the organization using it, there are multiple CMMI models. Consequently, the phrase “CMMI MODEL” could be any one of many collections of information. The phrase “CMMI models” refers to one, some, or the entire collection of possible models that can be generated from the CMMI Framework. · Continuous Representation o Uses capability levels to measure process improvement. · Discipline Amplification o These are model components that provide guidance for interpreting model information for specific disciplines (e.g., systems engineering, or software engineering). Discipline amplifications are added to other model components where necessary. These are easy to locate because they appear on the right side of the page and have a title indicating the discipline that they address (for example, “For Software Engineering”). · Generic Goals o These are called “generic” because the same goal statement appears in multiple process areas. In the staged representation, each process area has only one generic goal. Achievement of a generic goal in a process area signifies improved control in planning and implementing the processes associated with that process area, thus indicating whether these processes are likely to be effective, repeatable, and lasting. Generic goals are required model components and are used in appraisals to determine whether a process area is satisfied. · Generic Practices o These provide institutionalization to ensure that the processes associated with the process area will be effective, repeatable, and lasting. Generic practices are categorized by generic goals and common features and are expected components in CMMI models. (Only the generic practice title, statement, and elaborations appear in the process areas.) · Generic Practice Elaborations o After the specific practices, the generic practice titles and statements appear that apply to the process area. After each generic practice statement, an elaboration may appear in plain text with the heading “Elaboration.” The GPE provides information about how the generic practice should be interpreted for the process area. If there is no elaboration present, the application of the generic practice is obvious without an elaboration. · Goal o A required CMMI component that can be either a generic goal or a specific goal. When you see the word “goal” in a CMMI model, it always refers to model components. · Maturity Level o This is a defined evolutionary plateau of process improvement. Each maturity level stabilizes an important part of the organization’s processes. Maturity levels, which belong to the staged representation, apply to an organization’s overall maturity. There are five maturity levels, numbered 1 through 5. Each maturity level comprises a predefined set of process areas. · Process Area o This is a cluster of related practices in an area that, when performed collectively, satisfy a set of goals considered important for making significant improvement in that area. All CMMI process areas are common to both continuous and staged representations. In the staged representation, process areas are organized by maturity levels. · References o These are informative model components that direct the user to additional or more detailed information in related process areas. Typical phrases expressing these pointers are “Refer to the Organizational Training process area for more information about identifying training needs and providing the necessary training” or “Refer to the Decision Analysis and Resolution process area for more information about evaluating and selecting among alternatives.” · Specific Goals o These apply to a process area and address the unique characteristics that describe what must be implemented to satisfy the process area. Specific goals are required model components and are used in appraisals to help determine whether a process area is satisfied. · Specific Practice o This is an activity that is considered important in achieving the associated specific goal. The specific practices describe the activities expected to result in achievement of the specific goals of a process area. Specific practices are expected model components. · Staged Representation o This uses maturity levels to measure process improvement. · Statement o This is designed to be used for process-improvement and appraisal purposes. · Subpractice o These are detailed descriptions that provide guidance for interpreting specific or generic practices. Subpractices may be worded as if prescriptive, but are actually an informative component in CMMI models meant only to provide ideas that may be useful for process improvement. · Work Product o The term is used throughout the CMMI Product Suite to mean any artifact produced by a process. These artifacts can include files, documents, parts of the product, services, processes, specifications, and invoices. Examples of processes to be considered as work products include a manufacturing process, a training process, and a disposal process for the product. A key distinction between a Work Product and a product component is that a work product need not be engineered or part of the end product.

What is the Capability Maturity Model? (CMM)

noreply@blogger.com (The BASS) — Wed, 03 Dec 2014 10:44:00 +0000

Capability Maturity Model (CMM) broadly refers to a process improvement approach that is based on a process model. CMM also refers specifically to the first such model, developed by the Software Engineering Institute (SEI) in the mid-1980s, as well as the family of process models that followed. A process model is a structured collection of practices that describe the characteristics of effective processes; the practices included are those proven by experience to be effective.

CMM can be used to assess an organization against a scale of five process maturity levels. Each level ranks the organization according to its standardization of processes in the subject area being assessed. The subject areas can be as diverse as software engineering, systems engineering, project management, risk management, system acquisition, information technology (IT) services and personnel management.

CMM was developed by the SEI at Carnegie Mellon University in Pittsburgh. It has been used extensively for avionics software and government projects, in North America, Europe, Asia, Australia, South America, and Africa.Currently, some government departments require software development contract organization to achieve and operate at a level 3 standard.

History

The Capability Maturity Model was initially funded by military research. The United States Air Force funded a study at the Carnegie-Mellon Software Engineering Institute to create a model (abstract) for the military to use as an objective evaluation of software subcontractors. The result was the Capability Maturity Model, published as Managing the Software Process in 1989. The CMM is no longer supported by the SEI and has been superseded by the more comprehensive Capability Maturity Model Integration (CMMI).

Maturity Model

The Capability Maturity Model (CMM) is a way to develop and refine an organization's processes. The first CMM was for the purpose of developing and refining software development processes. A maturity model is a structured collection of elements that describe characteristics of effective processes. A maturity model provides:

• a place to start

• the benefit of a community’s prior experiences

• a common language and a shared vision

• a framework for prioritizing actions

• a way to define what improvement means for your organization

A maturity model can be used as a benchmark for assessing different organizations for equivalent comparison. It describes the maturity of the company based upon the project the company is dealing with and the clients.

Context

In the 1970s, technological improvements made computers more widespread, flexible, and inexpensive. Organizations began to adopt more and more computerized information systems and the field of software development grew significantly. This led to an increased demand for developers—and managers—which was satisfied with less experienced professionals.

Unfortunately, the influx of growth caused growing pains; project failure became more commonplace not only because the field of computer science was still in its infancy, but also because projects became more ambitious in scale and complexity. In response, individuals such as Edward Yourdon, Larry Constantine, Gerald Weinberg, Tom DeMarco, and David Parnas published articles and books with research results in an attempt to professionalize the software development process.

Watts Humphrey's Capability Maturity Model (CMM) was described in the book Managing the Software Process (1989). The CMM as conceived by Watts Humphrey was based on the earlier work of Phil Crosby. Active development of the model by the SEI began in 1986.

The CMM was originally intended as a tool to evaluate the ability of government contractors to perform a contracted software project. Though it comes from the area of software development, it can be, has been, and continues to be widely applied as a general model of the maturity of processes in IS/IT (and other) organizations.

The model identifies five levels of process maturity for an organisation. Within each of these maturity levels are KPAs (Key Process Areas) which characterise that level, and for each KPA there are five definitions identified:

• 1. Goals

• 2. Commitment

• 3. Ability

• 4. Measurement

• 5. Verification

The KPAs are not necessarily unique to CMM, representing - as they do - the stages that organizations must go through on the way to becoming mature.

The assessment is supposed to be led by an authorised lead assessor. One way in which companies are supposed to use the model is first to assess their maturity level and then form a specific plan to get to the next level. Skipping levels is not allowed.

Timeline

• 1987 SEI-87-TR-24 (SW-CMM questionnaire), released.

• 1989 Managing the Software Process, published.

• 1991 SW-CMM v1.0, released.

• 1993 SW-CMM v1.1, released.

• 1997 SW-CMM revisions halted in support for CMMI.

• 2000 CMMI v1.02, released.

• 2002 CMMI v1.1, released.

• 2006 CMMI v1.2, released.

Current state

Although these models have proved useful to many organizations, the use of multiple models has been problematic. Further, applying multiple models that are not integrated within and across an organization is costly in terms of training, appraisals, and improvement activities. The CMM Integration project was formed to sort out the problem of using multiple CMMs. The CMMI Product Team's mission was to combine three source models:

1. The Capability Maturity Model for Software (SW-CMM) v2.0 draft C

2. The Systems Engineering Capability Model (SECM)

3. The Integrated Product Development Capability Maturity Model (IPD-CMM) v0.98

4. Supplier sourcing

CMMI is the designated successor of the three source models. The SEI has released a policy to sunset the Software CMM and previous versions of the CMMI. The same can be said for the SECM and the IPD-CMM; these models were superseded by CMMI.

Future direction

With the release of the CMMI Version 1.2 Product Suite, the existing CMMI has been renamed the CMMI for Development (CMMI-DEV), V1.2. Two other versions are being developed, one for Services, and the other for Acquisitions.

In some cases, CMM can be combined with other methodologies. It is commonly used in conjunction with the ISO 9001 standard, as well as with the computer programming methodologies of Extreme Programming (XP), and Six Sigma.

Levels of the CMM

There are five levels of the CMM:

• Level 1 - Initial

o Processes are usually ad hoc and the organization usually does not provide a stable environment. Success in these organizations depends on the competence and heroics of the people in the organization and not on the use of proven processes. In spite of this ad hoc, chaotic environment, maturity level 1 organizations often produce products and services that work; however, they frequently exceed the budget and schedule of their projects.

o Organizations are characterized by a tendency to over commit, abandon processes in the time of crisis, and not be able to repeat their past successes again.

o Software project success depends on having quality people.

• Level 2 - Repeatable

o Software development successes are repeatable. The processes may not repeat for all the projects in the organization. The organization may use some basic project management to track cost and schedule.

o Process discipline helps ensure that existing practices are retained during times of stress. When these practices are in place, projects are performed and managed according to their documented plans.

o Project status and the delivery of services are visible to management at defined points (for example, at major milestones and at the completion of major tasks).

o Basic project management processes are established to track cost, schedule, and functionality. The minimum process discipline is in place to repeat earlier successes on projects with similar applications and scope. There is still a significant risk of exceeding cost and time estimate.

• Level 3 - Defined

o The organization’s set of standard processes, which is the basis for level 3, is established and improved over time. These standard processes are used to establish consistency across the organization. Projects establish their defined processes by the organization’s set of standard processes according to tailoring guidelines.

o The organization’s management establishes process objectives based on the organization’s set of standard processes and ensures that these objectives are appropriately addressed.

o A critical distinction between level 2 and level 3 is the scope of standards, process descriptions, and procedures. At level 2, the standards, process descriptions, and procedures may be quite different in each specific instance of the process (for example, on a particular project). At level 3, the standards, process descriptions, and procedures for a project are tailored from the organization’s set of standard processes to suit a particular project or organizational unit.

• Level 4 - Managed

o Using precise measurements, management can effectively control the software development effort. In particular, management can identify ways to adjust and adapt the process to particular projects without measurable losses of quality or deviations from specifications. At this level organization set a quantitative quality goal for both software process and software maintenance.

o Subprocesses are selected that significantly contribute to overall process performance. These selected subprocesses are controlled using statistical and other quantitative techniques.

o A critical distinction between maturity level 3 and maturity level 4 is the predictability of process performance. At maturity level 4, the performance of processes is controlled using statistical and other quantitative techniques, and is quantitatively predictable. At maturity level 3, processes are only qualitatively predictable.

• Level 5 - Optimizing

o Focusing on continually improving process performance through both incremental and innovative technological improvements. Quantitative process-improvement objectives for the organization are established, continually revised to reflect changing business objectives, and used as criteria in managing process improvement. The effects of deployed process improvements are measured and evaluated against the quantitative process-improvement objectives. Both the defined processes and the organization’s set of standard processes are targets of measurable improvement activities.

o Process improvements to address common causes of process variation and measurably improve the organization’s processes are identified, evaluated, and deployed.

o Optimizing processes that are nimble, adaptable and innovative depends on the participation of an empowered workforce aligned with the business values and objectives of the organization. The organization’s ability to rapidly respond to changes and opportunities is enhanced by finding ways to accelerate and share learning.

o A critical distinction between maturity level 4 and maturity level 5 is the type of process variation addressed. At maturity level 4, processes are concerned with addressing special causes of process variation and providing statistical predictability of the results. Though processes may produce predictable results, the results may be insufficient to achieve the established objectives. At maturity level 5, processes are concerned with addressing common causes of process variation and changing the process (that is, shifting the mean of the process performance) to improve process performance (while maintaining statistical probability) to achieve the established quantitative process-improvement objectives.

The most beneficial elements of CMM Level 2 and 3:

• Creation of Software Specifications, stating what is going to be developed, combined with formal sign off, an executive sponsor and approval mechanism. This is NOT a living document, but additions are placed in a deferred or out of scope section for later incorporation into the next cycle of software development.

• A Technical Specification, stating how precisely the thing specified in the Software Specifications is to be developed will be used. This is a living document.

• Peer Review of Code (Code Review) with metrics that allow developers to walk through an implementation, and to suggest improvements or changes. Note - This is problematic because the code has already been developed and a bad design can not be fixed by "tweaking", the Code Review gives complete code a formal approval mechanism.

• Version Control - a very large number of organizations have no formal revision control mechanism or release mechanism in place.

• The idea that there is a "right way" to build software, that it is a scientific process involving engineering design and that groups of developers are not there to simply work on the problem du jour.

What is Capability Maturity Model (CMM)? What are CMM Levels?

Capability Maturity Model is a bench-mark for measuring the maturity of anorganization’s software process. It is a methodology used to develop and refine an organization’s software development process. CMM can be used to assess an organization against a scale of five process maturity levels based on certain Key Process Areas (KPA). It describes the maturity of the company based upon the project the company is dealing with and the clients. Each level ranks the organization according to its standardization of processes in the subject area being assessed.

A maturity model provides:

§ A place to start

§ The benefit of a community’s prior experiences

§ A common language and a shared vision

§ A framework for prioritizing actions

§ A way to define what improvement means for your organization

In CMMI models with a staged representation, there are five maturity levels designated by the numbers 1 through 5 as shown below:

1. Initial

2. Managed

3. Defined

4. Quantitatively Managed

5. Optimizing

Maturity Level 1 – Initial: Company has no standard process for software development. Nor does it have a project-tracking system that enables developers to predict costs or finish dates with any accuracy.

In detail we can describe it as given below:

§ At maturity level 1, processes are usually ad hoc and chaotic.

§ The organization usually does not provide a stable environment. Success in these organizations depends on the competence and heroics of the people in the organization and not on the use of proven processes.

§ Maturity level 1 organizations often produce products and services that work but company has no standard process for software development. Nor does it have a project-tracking system that enables developers to predict costs or finish dates with any accuracy.

§ Maturity level 1 organizations are characterized by a tendency to over commit, abandon processes in the time of crisis, and not be able to repeat their past successes.

Maturity Level 2 – Managed: Company has installed basic software management processes and controls. But there is no consistency or coordination among different groups.

In detail we can describe it as given below:

§ At maturity level 2, an organization has achieved all the specific and generic goalsof the maturity level 2 process areas. In other words, the projects of the organization have ensured that requirements are managed and that processes are planned, performed, measured, and controlled.

§ The process discipline reflected by maturity level 2 helps to ensure that existing practices are retained during times of stress. When these practices are in place, projects are performed and managed according to their documented plans.

§ At maturity level 2, requirements, processes, work products, and services are managed. The status of the work products and the delivery of services are visible to management at defined points.

§ Commitments are established among relevant stakeholders and are revised as needed. Work products are reviewed with stakeholders and are controlled.

§ The work products and services satisfy their specified requirements, standards, and objectives.

Maturity Level 3 – Defined: Company has pulled together a standard set of processes and controls for the entire organization so that developers can move between projects more easily and customers can begin to get consistency from different groups.

In detail we can describe it as given below:

§ At maturity level 3, an organization has achieved all the specific and generic goals.

§ At maturity level 3, processes are well characterized and understood, and are described in standards, procedures, tools, and methods.

§ A critical distinction between maturity level 2 and maturity level 3 is the scope of standards, process descriptions, and procedures. At maturity level 2, the standards, process descriptions, and procedures may be quite different in each specific instance of the process (for example, on a particular project). At maturity level 3, the standards, process descriptions, and procedures for a project are tailored from the organization’s set of standard processes to suit a particular project or organizational unit.

§ The organization’s set of standard processes includes the processes addressed at maturity level 2 and maturity level 3. As a result, the processes that are performed across the organization are consistent except for the differences allowed by the tailoring guidelines.

§ Another critical distinction is that at maturity level 3, processes are typically described in more detail and more rigorously than at maturity level 2.

§ At maturity level 3, processes are managed more proactively using an understanding of the interrelationships of the process activities and detailed measures of the process, its work products, and its services.

Maturity Level 4 – Quantitatively Managed: In addition to implementing standard processes, company has installed systems to measure the quality of those processes across all projects.

In detail we can describe it as given below:

§ At maturity level 4, an organization has achieved all the specific goals of the process areas assigned to maturity levels 2, 3, and 4 and the generic goalsassigned to maturity levels 2 and 3.

§ At maturity level 4 Sub-processes are selected that significantly contribute to overall process performance. These selected sub-processes are controlled using statistical and other quantitative techniques.

§ Quantitative objectives for quality and process performance are established and used as criteria in managing processes. Quantitative objectives are based on the needs of the customer, end users, organization, and process implementers. Quality and process performance are understood in statistical terms and are managed throughout the life of the processes.

§ For these processes, detailed measures of process performance are collected and statistically analyzed. Special causes of process variation are identified and, where appropriate, the sources of special causes are corrected to prevent future occurrences.

§ Quality and process performance measures are incorporated into the organizations measurement repository to support fact-based decision making in the future.

§ A critical distinction between maturity level 3 and maturity level 4 is the predictability of process performance. At maturity level 4, the performance of processes is controlled using statistical and other quantitative techniques, and is quantitatively predictable. At maturity level 3, processes are only qualitatively predictable.

Maturity Level 5 – Optimizing: Company has accomplished all of the above and can now begin to see patterns in performance over time, so it can tweak its processes in order to improve productivity and reduce defects in software development across the entire organization.

In detail we can describe it as given below:

§ At maturity level 5, an organization has achieved all the specific goals of the process areas assigned to maturity levels 2, 3, 4, and 5 and the generic goalsassigned to maturity levels 2 and 3.

§ Processes are continually improved based on a quantitative understanding of the common causes of variation inherent in processes.

§ Maturity level 5 focuses on continually improving process performance through both incremental and innovative technological improvements.

§ Quantitative process-improvement objectives for the organization are established, continually revised to reflect changing business objectives, and used as criteria in managing process improvement.

§ The effects of deployed process improvements are measured and evaluated against the quantitative process-improvement objectives. Both the defined processes and the organization’s set of standard processes are targets of measurable improvement activities.

§ Optimizing processes that are agile and innovative depends on the participation of an empowered workforce aligned with the business values and objectives of the organization.

§ The organization’s ability to rapidly respond to changes and opportunities is enhanced by finding ways to accelerate and share learning. Improvement of the processes is inherently part of everybody’s role, resulting in a cycle of continual improvement.

§ A critical distinction between maturity level 4 and maturity level 5 is the type of process variation addressed. At maturity level 4, processes are concerned with addressing special causes of process variation and providing statistical predictability of the results. Though processes may produce predictable results, the results may be insufficient to achieve the established objectives. At maturity level 5, processes are concerned with addressing common causes of process variation and changing the process (that is, shifting the mean of the process performance) to improve process performance (while maintaining statistical predictability) to achieve the established quantitative process-improvement objectives.

PMP Training Programs: Classroom vs. Online

noreply@blogger.com (The BASS) — Wed, 03 Dec 2014 10:33:00 +0000

PMP Training

To become eligible for the PMP Certification exam, you have to earn 35 contact hours of formal Project Management Training. This is one of the few conditions that PMI (Project Management Institute, USA) has mandated to apply for the exam. There are many ways to earn these contact hours; however, the recommended way is to get it from any R.E.P. (Registered Education Provider) by PMI.

Based on their type of delivery, a training program can be broadly divided into two categories: Classroom or Online training.

Classroom vs. Online: Which is Better?

In a classroom training program, you will first have to make sure that you can spare four to five days for their scheduled training dates, providing seats are available.

If things go well, you can register and join the training program. After completion of the training you will get the certificate of attending 35 contact hours program.

Online Training Program

The only requirement for an Online Training Program is a computer with Internet access. Once you make the payment, you get instant access to the complete training program. These providers supply 30 to 90 days to complete the training program. You login into the system, watch the video and submit a questionnaire, if any. You are free to complete this training around your schedule.

Some providers give you an option to download their video based lectures on your computer’s hard drive. Watch it, fill out their questionnaire and send to them. After reviewing your questionnaire, they will send you the certification of completion.

Which one is better?

So now the Big Question—which one is better?

In many PM groups and forums PMP aspirants regularly ask which option they should go for. What’s the answer? It depends…

Cost: Classroom training programs are generally very expensive. Their cost varies from $1,000 to $4,000, or sometimes even more.

The upside to this is some classroom training programs provide a 100% guarantee to pass the exam. If you failed in first attempt, they will pay your test fee and if you fail again, they provide you further assistance to pass the exam. However, they are still costly.

On the other hand, the cost of online training program is much less. Programs start from as low as $50.

Convenience: Classroom training programs are not always readily available. They are usually held in big cities where training providers can get a sufficient number of students. You may have to commute a long distance to attend it (I travelled 600 km for this purpose!). This alone may be a big enough inconvenience to not make it worth your while.

Online training programs allow you to earn contact hours from within your own home.

Schedule: Classroom training programs are conducted several times a year. You have to check if you’re free or you can spare four to five days to accommodate their fixed schedule dates.

Online training programs have no such schedule and can be joined anytime.

Limited Space: Classroom training programs have limited seats availability. You must be sure to register early to ensure a seat.

There is no such limitation with an online training program.

Value of Training: If the trainer is well experienced, knowledgeable and charismatic then you can learn a lot from his or her experience and knowledge.

Content for online training program is the same for all and usually does not change.

Active Participation: In classroom training discussions are very interactive. By participating in these you can learn a lot from your classmates and make connections with others.

Online training programs lack social interaction and learning from other students. For some, this may be a serious deterrent to learning.

My View:

If...

Cost does not matter to you
Training is available at your place, or you don’t mind travelling
You can spare four to five days
Trainer is exceptionally good and charismatic

then you can consider class-room training program.

CMMI vs CMM: Which is Better?

noreply@blogger.com (The BASS) — Wed, 03 Dec 2014 10:22:00 +0000

Carnegie Mellon developed CMM as a process maturity model. Implementation of CMM raised many challenges that led to development of CMMI as an improvement. CMMI however does not replace CMM and the effectiveness depends on the specific area of application.

Origins of CMM

The capability maturity model (CMM) is an assessment model developed by the Software Engineering Institute at Carnegie Mellon University in 1990, to ascertain the process maturity levels in the software.

The model describes five levels of best engineering and management practices based on data collected from various industries. Organizations looking for solutions to manage projects aim to attain CMM certification by compliance with the CMM level closest to their level of process maturity.

Challenged Faced During CMM Implementation

CMM became popular as it allowed software companies attain process consistency, predictability, and reliability. However, the implementation of CMM led to many hurdles.

1. Lack of Integration: CMM has separate models for each function. Such models often overlap, contradict, and display different levels of maturity. This lack of standardization leads to confusion and conflict during the implementation phase and increase training and appraisal costs.
2. Limitations of KPA: The “Key Performance Areas (KPA),” that define CMM levels focus on “policing” activities such as specifications, documentation, audits, and inspections, and do not reveal architecturally significant flaws.
3. Activity-based Approach: CMM is an activity-based approach that considers only the completion of a specific activity, and not whether the completed activity achieved the desired results.
4. Paperwork: CMM places great importance on paperwork and meetings that take management’s time and effort away from actual work processes. CMM traps the organization in recording and complying with processes, often at the cost of strategic goals.
CMMI: An Integrated Approach

The Software Engineering Institute at Carnegie Mellon University developed Capability Maturity Model Integration(CMMI) in 2006 to integrate and standardize the separate models of CMM, and to eradicate other drawbacks of CMM.

CMMI documents industry best practices categorized on separate areas of interests rather than separate functions. Organizations choose from any of the 22 available models depending on the business objectives, and each model covers all the functional areas.
CMMI vs CMM KPA

Both CMM and CMMI define five distinct levels of process maturity based on Key Performance Areas (KPA’s). The KPA's of CMMI levels overcome the inefficiency of CMM levels to unearth significant architectural flaws.
- Level 1 (Initial): The first level of both CMM and CMMI describes an immature organization without any defined processes, run in an ad hoc, uncontrolled, and reactive manner.
- Level 2 (Repeat): Organizations that repeat some processes attain Level 2 CMM. Level 2 of CMMI however requires management of organizational requirements through planned, performed, measured, and controlled processes.
- Level 3 (Defined): CMM Level 3 mandates a set of documented standard processes to establish consistency across the organization. CMMI Level 3 is an improvement of CMMI Level 2 and describes the organizational processes in standards, procedures, tools, and methods.
- Level 4 (Manage): CMM Level 4 requires organizations to attain control over processes by using quantitative statistical techniques. CMMI Level 4 demands likewise, but also identifies sub processes that significantly contribute to overall process efficiency.
- Level 5 (Optimized): CMM Level 5 mandates use of quantitative tools and objectives to manage process improvement. CMMI Level 5 on the other hand focuses on continuously improving process performance through incremental and innovative technological improvements.
While CMM is a certification tool, CMMI is not. An organization is appraised and awarded a CMMI Rating from 1 to 5 depending on the extent to which the organization adopts the selected CMMI model.
Differences in Approach

CMM measures the maturity level of an organization by determining if an organization completes the specific activities listed in the Key Performance Areas (KPA), oblivious to whether the completion of such activity leads to the desired result. CMMI is also an activity based approach but the major difference is that CMMI takes a more result-oriented approach when defining and measuring Key Performance Areas.

CMM KPA concentrates on the completion of specific tasks or processes and does not motivate the organization to focus on process architecture. CMMI, on the other hand has an iterative lifecycle that integrates the latest best practices from the industry and attacks risks in process architecture at an early stage.

CMMI supersedes CMM in software development processes, but CMM is still relevant and appropriate for sequential, activity-based management paradigm.
Paperwork

Both CMM and CMMI give importance to paperwork and meetings that distract management’s time and effort from actual work process. CMM is however concerned at recording processes whereas CMMI documentation and meetings focus on strategic goals of the organizations.

CMM has focused attention on processes, but the new CMMI goes a step further and focus attention on result-oriented processes.

Capability Maturity Model Integration (CMMI)

noreply@blogger.com (The BASS) — Wed, 03 Dec 2014 10:00:00 +0000

Capability Maturity Model Integration (CMMI) is a process improvement training and appraisal program and service administered and marketed by Carnegie Mellon Universityand required by many DOD and U.S. Government contracts, especially in software development. Carnegie Mellon University claims CMMI can be used to guide process improvement across a project, division, or an entire organization. Under the CMMI methodology, processes are rated according to their maturity levels, which are defined as: Initial, Managed, Defined, Quantitatively Managed, Optimizing. Currently supported is CMMI Version 1.3. CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.

[hide]

Overview

CMMI currently addresses three areas of interest:

Product and service development — CMMI for Development (CMMI-DEV),
Service establishment, management, — CMMI for Services (CMMI-SVC), and
Product and service acquisition — CMMI for Acquisition (CMMI-ACQ).

CMMI was developed by a group of experts from industry, government, and the Software Engineering Institute (SEI) at Carnegie Mellon University. CMMI models provide guidance for developing or improving processes that meet the business goals of an organization. A CMMI model may also be used as a framework for appraising the process maturity of the organization.^[1] By January of 2013, the entire CMMI product suite was transferred from the SEI to the CMMI Institute, a newly created organization at Carnegie Mellon.^[2]

CMMI originated in software engineering but has been highly generalized over the years to embrace other areas of interest, such as the development of hardware products, the delivery of all kinds of services, and the acquisition of products and services. The word "software" does not appear in definitions of CMMI. This generalization of improvement concepts makes CMMI extremely abstract. It is not as specific to software engineering as its predecessor, the Software CMM (CMM, see below).

History

CMMI was developed by the CMMI project, which aimed to improve the usability of maturity models by integrating many different models into one framework. The project consisted of members of industry, government and the Carnegie Mellon Software Engineering Institute (SEI). The main sponsors included the Office of the Secretary of Defense (OSD) and the National Defense Industrial Association.

CMMI is the successor of the capability maturity model (CMM) or Software CMM. The CMM was developed from 1987 until 1997. In 2002, CMMI Version 1.1 was released, Version 1.2 followed in August 2006, and CMMI Version 1.3 in November 2010. Some of the major changes in CMMI V1.3 ^[3] are the support of Agile Software Development,^[4]improvements to high maturity practices ^[5] and alignment of the representation (staged and continuous).^[6]

According to the Software Engineering Institute (SEI, 2008), CMMI helps "integrate traditionally separate organizational functions, set process improvement goals and priorities, provide guidance for quality processes, and provide a point of reference for appraising current processes."^[7]

CMMI topics

CMMI representation

CMMI exists in two representations: continuous and staged.^[1] The continuous representation is designed to allow the user to focus on the specific processes that are considered important for the organization's immediate business objectives, or those to which the organization assigns a high degree of risks. The staged representation is designed to provide a standard sequence of improvements, and can serve as a basis for comparing the maturity of different projects and organizations. The staged representation also provides for an easy migration from the SW-CMM to CMMI.^[1]

CMMI model framework

For more details on this topic, see Process area (CMMI).

Depending on the CMMI areas of interest (acquisition, services, development) used, the process areas it contains will vary.^[8] Process areas are the areas that will be covered by the organization's processes. The table below lists the collection of sixteen CMMI core process areas that are present for all CMMI areas of interest in CMMI Version 1.3.

Capability Maturity Model Integration (CMMI) Core Process Areas
Abbreviation	Name	Area	Maturity Level
CAR	Causal Analysis and Resolution	Support	5
CM	Configuration Management	Support	2
DAR	Decision Analysis and Resolution	Support	3
IPM	Integrated Project Management	Project Management	3
MA	Measurement and Analysis	Support	2
OPD	Organizational Process Definition	Process Management	3
OPF	Organizational Process Focus	Process Management	3
OPM	Organizational Performance Management	Process Management	5
OPP	Organizational Process Performance	Process Management	4
OT	Organizational Training	Process Management	3
PMC	Project Monitoring and Control	Project Management	2
PP	Project Planning	Project Management	2
PPQA	Process and Product Quality Assurance	Support	2
QPM	Quantitative Project Management	Project Management	4
REQM	Requirements Management	Project Management	2
RSKM	Risk Management	Project Management	3
SAM	Supplier Agreement Management	Support	2

Maturity levels in CMMI for development

There are five maturity levels. Maturity level ratings are awarded for levels 2 through 5. The process areas below and their maturity levels are listed for the CMMI for Development model:

Maturity Level 2 - Managed

CM - Configuration Management
MA - Measurement and Analysis
PMC - Project Monitoring and Control
PP - Project Planning
PPQA - Process and Product Quality Assurance
REQM - Requirements Management
SAM - Supplier Agreement Management

Maturity Level 3 - Defined

DAR - Decision Analysis and Resolution
IPM - Integrated Project Management
OPD - Organizational Process Definition
OPF - Organizational Process Focus
OT - Organizational Training
PI - Product Integration
RD - Requirements Development
RSKM - Risk Management
TS - Technical Solution
VAL - Validation
VER - Verification

Maturity Level 4 - Quantitatively Managed

OPP - Organizational Process Performance
QPM - Quantitative Project Management

Maturity Level 5 - Optimizing

CAR - Causal Analysis and Resolution
OPM - Organizational Performance Management

Maturity levels in CMMI for services

The process areas below and their maturity levels are listed for the CMMI for Services model:

Maturity Level 2 - Managed

CM - Configuration Management
MA - Measurement and Analysis
PPQA - Process and Product Quality Assurance
REQM - Requirements Management
SAM - Supplier Agreement Management
SD - Service Delivery
WMC - Work Monitoring and Control
WP - Work Planning

Maturity Level 3 - Defined

CAM - Capacity and Availability Management
DAR - Decision Analysis and Resolution
IRP - Incident Resolution and Prevention
IWM - Integrated Work Managements
OPD - Organizational Process Definition
OPF - Organizational Process Focus
OT - Organizational Training
RSKM - Risk Management
SCON - Service Continuity
SSD - Service System Development
SST - Service System Transition
STSM - Strategic Service Management

Maturity Level 4 - Quantitatively Managed

OPP - Organizational Process Performance
QWM - Quantitative Work Management

Maturity Level 5 - Optimizing

CAR - Causal Analysis and Resolution
OPM - Organizational Performance Management

Maturity levels in CMMI for acquisition

The process areas below and their maturity levels are listed for the CMMI for Acquisition model:

Maturity Level 2 - Managed

AM - Agreement Management
ARD - Acquisition Requirements Development
CM - Configuration Management
MA - Measurement and Analysis
PMC - Project Monitoring and Control
PP - Project Planning
PPQA - Process and Product Quality Assurance
REQM - Requirements Management
SSAD - Solicitation and Supplier Agreement Development

Maturity Level 3 - Defined

ATM - Acquisition Technical Management
AVAL - Acquisition Validation
AVER - Acquisition Verification
DAR - Decision Analysis and Resolution
IPM - Integrated Project Management
OPD - Organizational Process Definition
OPF - Organizational Process Focus
OT - Organizational Training
RSKM - Risk Management

Maturity Level 4 - Quantitatively Managed

OPP - Organizational Process Performance
QPM - Quantitative Project Management

Maturity Level 5 - Optimizing

CAR - Causal Analysis and Resolution
OPM - Organizational Performance Management

CMMI models[edit]

CMMI best practices are published in documents called models, each of which addresses a different area of interest. The current release, CMMI Version 1.3, provides models for three areas of interest: development, acquisition, and services.

CMMI for Development (CMMI-DEV), v1.3 was released in November 2010. It addresses product and service development processes.
CMMI for Acquisition (CMMI-ACQ), v1.3 was released in November 2010. It addresses supply chain management, acquisition, and outsourcing processes in government and industry.
CMMI for Services (CMMI-SVC), v1.3 was released in November 2010. It addresses guidance for delivering services within an organization and to external customers.

Appraisal

An organization cannot be certified in CMMI; instead, an organization is appraised. Depending on the type of appraisal, the organization can be awarded a maturity level rating (1-5) or a capability level achievement profile.

Many organizations find value in measuring their progress by conducting an appraisal. Appraisals are typically conducted for one or more of the following reasons:

To determine how well the organization’s processes compare to CMMI best practices, and to identify areas where improvement can be made
To inform external customers and suppliers of how well the organization’s processes compare to CMMI best practices
To meet the contractual requirements of one or more customers

Appraisals of organizations using a CMMI model^[9] must conform to the requirements defined in the Appraisal Requirements for CMMI (ARC) document. There are three classes of appraisals, A, B and C, which focus on identifying improvement opportunities and comparing the organization’s processes to CMMI best practices. Of these, class A appraisal is the most formal and is the only one that can result in a level rating. Appraisal teams use a CMMI model and ARC-conformant appraisal method to guide their evaluation of the organization and their reporting of conclusions. The appraisal results can then be used (e.g., by a process group) to plan improvements for the organization.

The Standard CMMI Appraisal Method for Process Improvement (SCAMPI) is an appraisal method that meets all of the ARC requirements.^[10] Results of an SCAMPI appraisal may be published (if the appraised organization approves) on the CMMI Web site of the SEI: Published SCAMPI Appraisal Results. SCAMPI also supports the conduct ofISO/IEC 15504, also known as SPICE (Software Process Improvement and Capability Determination), assessments etc.

This approach promotes that members of the EPG and PATs be trained in the CMMI, that an informal (SCAMPI C) appraisal be performed, and that process areas be prioritized for improvement. More modern approaches, that involve the deployment of commercially available, CMMI-compliant processes, can significantly reduce the time to achieve compliance. SEI has maintained statistics on the "time to move up" for organizations adopting the earlier Software CMM as well as CMMI.^[11] These statistics indicate that, since 1987, the median times to move from Level 1 to Level 2 is 23 months, and from Level 2 to Level 3 is an additional 20 months. Since the release of the CMMI, the median times to move from Level 1 to Level 2 is 5 months, with median movement to Level 3 another 21 months. These statistics are updated and published every six months in a maturity profile.^{[citation needed]}

The Software Engineering Institute’s (SEI) Team Software Process methodology and the use of CMMI models can be used to raise the maturity level. A new product called Accelerated Improvement Method^[12] (AIM) combines the use of CMMI and the TSP.^[13]

CMMI Security Guides[edit]

To address user security concerns, two unofficial security guides are available. Considering the Case for Security Content in CMMI for Services has one process area, Security Management.^[14] Security by Design with CMMI for Development, Version 1.3 has the following process areas:

OPSD - Organizational Preparedness for Secure Development
SMP - Secure Management in Projects
SRTS - Security Requirements and Technical Solution
SVV - Security Verification and Validation

While they do not affect maturity or capability levels, these process areas can be reported in appraisal results.^[15]

Applications

The SEI published that 60 organizations measured increases of performance in the categories of cost, schedule, productivity, quality and customer satisfaction.^[16] The median increase in performance varied between 14% (customer satisfaction) and 62% (productivity). However, the CMMI model mostly deals with what processes should be implemented, and not so much with how they can be implemented. These results do not guarantee that applying CMMI will increase performance in every organization. A small company with few resources may be less likely to benefit from CMMI; this view is supported by the process maturity profile (page 10). Of the small organizations (<25 employees), 70.5% are assessed at level 2: Managed, while 52.8% of the organizations with 1001–2000 employees are rated at the highest level (5: Optimizing).

Interestingly, Turner & Jain (2002) argue that although it is obvious there are large differences between CMMI and agile methods, both approaches have much in common. They believe neither way is the 'right' way to develop software, but that there are phases in a project where one of the two is better suited. They suggest one should combine the different fragments of the methods into a new hybrid method. Sutherland et al. (2007) assert that a combination of Scrum and CMMI brings more adaptability and predictability than either one alone. David J. Anderson (2005) gives hints on how to interpret CMMI in an agile manner.

CMMI Roadmaps,^[17] which are a goal-driven approach to selecting and deploying relevant process areas from the CMMI-DEV model, can provide guidance and focus for effective CMMI adoption. There are several CMMI roadmaps for the continuous representation, each with a specific set of improvement goals. Examples are the CMMI Project Roadmap,^[18] CMMI Product and Product Integration Roadmaps ^[19] and the CMMI Process and Measurements Roadmaps.^[20] These roadmaps combine the strengths of both the staged and the continuous representations.

The combination of the project management technique earned value management (EVM) with CMMI has been described (Solomon, 2002). To conclude with a similar use of CMMI, Extreme Programming (XP), a software engineering method, has been evaluated with CMM/CMMI (Nawrocki et al., 2002). For example, the XP requirements management approach, which relies on oral communication, was evaluated as not compliant with CMMI.

CMMI can be appraised using two different approaches: staged and continuous. The staged approach yields appraisal results as one of five maturity levels. The continuous approach yields one of four capability levels. The differences in these approaches are felt only in the appraisal; the best practices are equivalent and result in equivalent process improvement results.