The iTKO LISA Soapbox: Modern Application Testing, Validation & Virtualization

People are Still Testing User Interfaces

2009-11-06T13:51:28-06:00

So the heyday of UI testing is over as far as enterprise software is concerned, right? In modern apps, the meaningful business logic happens behind the webpage, in the service-to-service integrations that serve up the appropriate dynamic data. Testing the UI is out of fashion.

Well not exactly. While one type of manual UI acceptance testing faltered, we see another kind of UI testing taking its place. As much as it is true that there are an increasing number of connections and activities going on "behind the glass" that must be tested, it never will replace the need for thorough testing of the user experience. Why is this?

Enter RIAs, or Rich Internet Applications. For a while, serious business applications that leveraged the Internet largely treated the web page as a "request interface" - you enter all the information into a form, submit the page, and it makes a request to the web server, which gets the necessary information from the app server after plenty of back-end processing. Due to performance and latency issues, as well as some rather buggy initial "rich UI" approaches over the last 10 years, it was a lot safer to keep the browser-facing functionality rather simple.

Therefore, testing the UI became of diminishing value in web apps. We needed to validate the back-end systems - the messaging queues, business processes and systems of record. A symptom discovered in the UI meant you were very late to the game in testing.

Now the UI is shifting back into balance in the testing world with RIAs. As customers expect easier and ever-richer user experience on the web, development teams (and the community at large) have moved some of the business functionality back into the browser, to allow customers to more interactively make choices. Technologies such as AJAX, Flash/Flex, Java Swing, Silverlight and other tools have replaced some of the buggy and slow web UI tools of the past. REST-style approaches further advanced the architecture of richer functionality in the front-end.

With this change comes a renaissance in the practice of UI testing. As seen here, one RIA screen such as a trading or entertainment portal may look like one web page to the user, but actually represent hundreds or thousands of separate requests to different servers that populate that page with content. A mere user click or mouse-over on the screen can again, represent several calls from the browser for dynamic data to be displayed. And unlike the late 90's, these customers are not going to wait a few seconds to see results. Lagging performance, displayed glitches and inaccuracies will send the user elsewhere in a heartbeat.

It takes both human insight: "What is a good user experience and is that what I'm seeing as I test?" AND;
A whole lot of automation: "How can a tester possibly automate the tests I've already done, while remembering to cover all of the dynamic permutations of content that could be displayed in this RIA screen?"

We've been doing a lot of research in this area, and we've posted some new material on LISA Solutions for RIA and Web 2.0 User Interfaces on itko.com, if you are interested in this topic. Customers are bringing UI testing into harmony with service-based testing and validation approaches.

iTKO Chalk Talk - Test Data Management

2009-11-02T18:03:25-06:00

Here's a new feature of the iTKO Blog we hope you enjoy - a series of educational "Chalk Talk" short videos covering topics like Software Quality, SOA Governance, BPM Validation, Testing, Cloud Computing and more. This first one features our founder and Chief Geek, John Michelsen. He has a way of pushing a marker that really makes gritty enterprise software topics seem very approachable, so here's his first installment on the challenges of Test Data Management:

One topic that has been on the tip of most of our discussions and recent posts these days is that of virtualization as a solution for Test Data Management (or, "vDTM" as I like to call it...). This form of vTDM is important because it allows your test and development teams to virtually manage test data outside of the system of record.

We hope you find our new Chalk Talks engaging and informative, and look for more video from iTKO soon. If you have any topics you'd like us to cover in the future, send us a note at info@itko.com.

Virtual Data in the Clouds - Enabling Remote Testing Teams

2009-10-28T16:37:08-05:00

We had a chance to briefly touch on this topic in our founder John's latest webinar with Infosys' Aparna Sharma on "Accelerating Modern Application Quality" -- archive available now.

A question came in during the show that was particularly pertinent to the topic at hand. How can these virtualized services and data models work in the Cloud to allow collaboration with distributed teams, especially offshore test teams? Indeed, without a good approach for virtualizing data in a Cloud-based development environment, you simply cannot get there without wasting about 80% of your effort and cost in testing today's service-based applications.

You expected ubiquitous, flexible access to needed resources with Cloud Computing. So why can testing teams end up sitting on the bench instead of getting involved as early as possible? Much of the challenge lies within the test data itself.

In short there are "Wires hanging out of your Test Cloud" as we illustrate here - especially when it comes to Data. You can't just take a 4TB database or an external SaaS-type data provider and plop them into a hypervisor and provision them in the Cloud. Test Data is a frontier because it is not easy to replicate via known means - it is either too bulky or too dynamic to reliably move to the Cloud, but we absolutely must provision teams with good test data, in order to do enough testing to rely upon these systems for business.

Primarily there is the problem of valid data access needed for thorough testing. Most often, this is locked up in some mainframe or live service that is simply too critical to open up to the testing team. Since business continuity usually trumps new product testing, IT Ops guys will crack down on test access time windows to the live systems and the data within them. So you might have a 2 hour period of access - barely enough time to test an on-premise system, much less a Cloud-based system that is talking to other data providers.

Second, there is the problem of data sensitivity. Even if we have contracts and a good working relationship in place, we don't want to be putting data that needs to stay private in the hands of any third party. So we do need to mask or "desensitize" the data so it follows the expected structure, without BEING someone's social security number or bank account. This has always been a concern in healthcare for instance (see HIPAA guidelines), but the level of certification, compliance and controls around data privacy and security in every industry is only getting tighter. We need to be able to give these remote teams obfuscated data so they can continue testing efficiently.

Third, data setup and teardown continues to be difficult in Cloud environments. We touched on one of our favorite examples in the webinar, a Telco that literally spends 2 hours running tests, and 2 whole DAYS resetting data. Many of these problems wont go away when the Cloud-provisioned system is talking to external dependencies and systems of record that need to be accessed and later cleaned out of running systems. Quite often this is a main point of contention, as a test could corrupt the tests of other teams or even the live systems if not contained correctly.

These are just three concerns. There are many other open issues of how Test Data will be handled when we move to Cloud-based environments. I thought Judith Hurwitz nailed this as one of her observations in her "What are the Unanticipated Consequences of Cloud Computing" post:

"Data will increasingly be seen as a reusable resource that can be used in lots of different situations. There will continue to be strategic line of business applications but they will be more systems of record that keep track of the final result of actions that take place dynamically in the cloud. The value of data is not in its tight packaging as we have been used to for decades but it the flexibility to move, transform, and leverage data. The watch word for data in this new model will be Trusted Data in the Cloud."

Couldn't agree more with this observation. As we move to ever more distributed, flexible computing models, we are supporting these with very distributed development and testing teams. The data needed for the software lifecycle must be shared among business partners, on shore and off-shore teams, so success will all come back to our level of Trust that the Cloud will give us the data we need, and not give us the side effects of data we don't need.

Of course, here at iTKO, we've been stretching out our LISA Virtualize to better support Test Data Management (TDM) solution efforts with the capture, manipulation, dynamic "desensitization" and Virtualization of data used in Cloud-based development environments. Look for more discussion and research from us soon on how expert testing teams are taking advantage of Virtual Test Data in the cloud.

Field Stories - Managing Through the Logistics of Change

2009-10-19T10:51:55-05:00

Here's a new customer story we just completed about working with a leading Europe logistics company on integration and performance testing throughout a comprehensive change process. This large transportation vendor handles both sea and land operations for passengers and freight, and handles a very high volume of traffic through its management and reservation systems.

This situation represents a complete replacement of the custom systems the company cobbled together over the last 20+ years with a completely new solution set, using some of the best-of-breed logistics and TM software packages on the market, along with integration software from Software AG (webMethods) and some Oracle systems of record. In order to get there, this company built a new team to allow a "competency center" around integration, to match the investments it was making in a newer, more scalable platform.

Obviously, with so many critical operations on the line, the integration center immediately identified performance testing as a primary task. They already had individuals attempting to verify response times by testing UI elements such as passenger and ferry reservation websites, with little success, so they turned to the back-end elements, where the harder business process calculations need to be made.

By directly invoking the message broker, integration framework and the system of record behind the browser, they were able to deliver a system that has so far run without critical failures in its first seven months of operations. At the same time by automating the execution of regression and performance test runs, they avoided needing to hire several more testers - estimating a reduction of 33% or more of the testing cost.

You can download the complete story here: http://www.itko.com/resources/europelogics.jsp.

The News - Working with Harris on Complex Validation Problems

2009-10-13T20:29:51-05:00

Today iTKO made an announcement about our strategic partnership with Harris Corporation, a Fortune 200 systems integrator we have been working with on several significant projects in the US Department of Defense and other federal agencies. Read the press release here.

What is significant about this? Many organizations (commercial and government) are realizing that while developers can build and integrate software faster than ever, their testing efforts have proven inadequate at keeping up. These agencies need proactive, automated, and continuous validation to ensure trust and reliability in federated, mission-critical systems that are subject to constant change.

iTKO and Harris are responding to this challenge with a validation and virtualization solution that will appeal to organizations that trust and rely on these systems. Examples include federal emergency management centers, military command and control systems, financial institution trading systems, utility energy management grids, airline traffic control networks, and many others.

While we have been cooperating with leading government contractors like Harris for 4 years on integration projects like these (see this iTKO paper on Federated governance we first wrote 3 years ago...) we have seen a major shift occurring in the way Federal agencies specify and manage their complex applications. They are moving to a shared services / shared components model, where different functional units (or, services) are consumed and managed by different teams, with a central point of certification and testing for these services.

A federated, shared governance model means there is just enough governance enforced at a central level, and authority for managing and using the application components is doled out to the constituents. We compare it to a Federal government vs. state's rights, vs. city council authority model. The federal government might specify a budget for a highway, but it doesn't dictate where parking is located in the city.

This federated model works very well for defense agencies and the many firms that support them, as they are all working toward a common goal - delivering functionality to the warfighter with maximum efficiency. As we have frequently mentioned in this blog, businesses would do well to take cues from how government scales their systems and cooperates to meet goals - they were leaders in SOA and will likely continue to lead the way in many aspects of modern application governance.

Webinar 10/27 w/Infosys & ITKO - Accelerating Modern Application Quality

2009-10-08T14:48:46-05:00

You might say we are between a pillow and a hard place right now when it comes to modern applications. On one side, we have killer new development approaches such as service orientation, BPM, and SaaS that provide us with a much greater amount of flexibility and speed in assembling reusable components and services. Development time and costs have indeed gone down, while functionality gets ever more sophisticated.

However the unintended consequence of being able to build and change applications so quickly comes at a price. Testing and validating these systems has become harder than ever. So many interconnected, interdependent components means an exponential increase in potential points of failure, and an inability to have access to a stable and affordable test environment. So much so, that testing environment and error repair costs are outstripping development cost in many cases.

There's gotta be a way out of this situation. Fortunately, we have some experts ready to help shed some light on these new costs and track down the quality issues in modern applications. We are privileged to have our founder/"Chief Geek" John Michelsen, presenting with Infosys' head of Client Services, Aparna Sharma on this upcoming session:

Webinar with Infosys and iTKO: "Accelerating Modern Application Quality: Testing Strategies for Reducing Risk and Cost in Today's Distributed Application Environment"
Tuesday, October 27th, 2009 – 1:00 PM Eastern (US) Time

Between these two, you have a great deal of experience leading and advising test and development teams in hundreds of engagements with the most complex types of software environments. So I expect nothing less than finding out what works today (and what didn't work) for real enterprises. After all, time-to-market is measured by delivery success, not the rate at which developers can string together new (but untested) functionality. Hope to see you there, and bring your questions!

What's the next big obsession for Enterprise IT Geeks?

2009-10-07T11:23:04-05:00

As an occasional pundit and commentator on IT trends, I've had a lot of chances to watch the "next big obsession" come and go for enterprise IT shops. The one commonality we've seen, is that if the obsession successfully takes hold in business, then it will become so ubiquitous that it becomes kinda boring to the IT guys on the cutting edge.

So goes the case for SOA... We've tracked the "Is SOA Dead?" thread over the last couple years and it was fun, because it points out just how common SOA has become part of the enterprise IT landscape, delivering value so quietly that we almost take it for granted now. Just like "eCommerce" used to be a big deal, then it just became the nature of doing business over the Internet.

So one of these questions I posed to Peter Schooff of eBizQ was this very thing - and he posted it to their Forum - What are Enterprise IT Geeks Obsessed with Today? I was looking for stories, and not the next dot on an analyst wave or cycle diagram. He posted the question and some fun answers there.

Clearly you can't escape Cloud Computing right now as the grand champion of obsession (Make mine Private or Public?). Mobile platforms and smartphones - riding high again. RIAs, particularly REST-type web apps are garnering a lot of geek attention. But what never changes is that these innovative obsessions aren't just about money - they are inevitably tied to doing things better, more efficiently than before. The cost savings and revenue upside are the results of success, but not necessarily the motivator.

I particularly liked Sandeep Gupta's comment in the forum that "IT geeks do not change...they are always passionate about the technology space/environment that they work in.....obsession brings aspiration and passion...leading to effeciency and growth" and that to me describes the accelerating nature of innovation that is going on here.

Also great - Brenda Michelson's comment (via Twitter) about "Hopefully getting things done!" Well put. And Phil Ayers laments that IT folks he runs into "just don't have time to be geeky enough and keep up with, let alone apply, the vogue technologies and methodologies." Yeah, there is probably a little too much talk about the next big thing and not enough application going on. So until next time...

Finding Hidden Value in Test Data Virtualization

2009-10-01T15:18:12-05:00

As you've seen in recent posts here, most of our discussions around Service Virtualization center around the lack of availability of systems, environments, etc. -– i.e. the "Over-utilized" problem. A Virtual Service Environment (VSE) is a solution that helps eliminate these constraints and therefore can dramatically reduce the time to market, and/or infrastructure costs. Also, once these constraints are removed, testing (functional and performance) can be done much earlier, and therefore greatly improve the quality of the released software.

We also talk about the use of VSE to enable parallel development. Being able to simulate dependencies that don't exist, and allowing testing and development activities to happen in parallel. Again, this can greatly reduce time to market, costs and improve overall quality.

However, there is a value of VSE that we don't talk as much about. An issue that has applicability in almost every enterprise environment, is one related to Test Data challenges. Let me explain.

There are at least 4 Test Data related challenges that Service Virtualization helps solve or assuage:

Data Scenarios are difficult to create. In the modern application, data required to support scenarios will have to be synchronized across multiple systems. Not only is that highly time consuming, but due to access constraints, or skill constraints, or various other reasons, creating and synchronizing that data could become a near impossible task. This may either elongate the testing cycle or, even worse, encourage the testers to reduce the number of scenarios that the system is tested under.
Data is Volatile. This is partially related to (1). In a lot of cases, the systems being tested go directly against systems that are either in-production or are being shared by other teams. This causes the data to be volatile (changing randomly and often). This makes the testing process impossible, because any results of the testing against these volatile systems cannot be validated.
Data needs to be Desensitized. In a lot of cases, data challenges could be reduced by using production data. However, production data may have sensitive information like credit card numbers, social security numbers, etc. Not only is this a problem for the direct employees, the challenge is greatly magnified when offshore or outsourcing contractors are used.
Data should not impact production systems. Some test scenarios result in transactions being created that can mess up a production system. For instance, if a test scenario requires a new order getting placed, the order once generated will have to be traced and deleted from the production systems. This is a major pain for the testers, and for the operations personnel.

As you can see from the above examples, Test Data Management challenges represent a huge constraint on the productivity of almost any enterprise software development project of significant scope.

Here's how we have found customers can approach these challenges using Service Virtualization:

Reduce the number of points that need to be synchronized, by capturing relevant transactions, chaining them together into scenarios, and creating a virtual model that can be manipulated without programming.
Stabilize the data so that the test results are actually valid, for instance, maintaining the relevance of variables like changing dates and varying types of customer data entry.
Desensitize the data so it does not violate security and privacy policies.
Substitute a Virtual Service for the production system, so there is no need to impact the critical live system of record to retrieve data. This can be done alongside ALM tools and Service Registries for SOA - essentially routing a known test data request to the VSE instead of the real thing.

In this arena of virtualization, we have been amazed by how widespread these shortcomings are in test data management, with little recognition of any available solution for them. We believe our LISA Virtualize uniquely supports these practices, with automation for capturing, modeling, desensitizing and maintaining virtual test data.

Virtualizing Over-Utilized Systems: Part 6 - Customer Examples & Conclusion

2009-09-28T12:58:59-05:00

This is the sixth post in our series based on the new whitepaper "The Next Frontier for Virtualization: Over-Utilized Systems" by iTKO (Ahrens, English). In this issue we cover how companies are applying Service Virtualization to solve for over-utilization of core systems and services needed for testing and development.

Examples of how SV Answers the Challenges of Over-Utilized Systems:

Solving a Common Bottleneck – When each team can get their own replicated test environment and data provided as a Virtual Service at minimal cost, there is no longer a need to compete for testing time during a small availability window, much less secure a capital expenditure just to get access to a hosted service-based system.

Customer Example: A leading US airline replicated much of the functionality of a partner’s reservation system that charged high per-use fees when used for testing purposes, as well as restrictions on usage due to the fact that millions of transactions needed to be handled on the system daily. Switching to a virtual service model of the system allowed multiple teams to have their own stable environment and data for test scenarios, as well as saving millions of dollars annually (up to $12M) in non-revenue-generating access fees for the live service.

Solving Improper Capacity – Trying to pinpoint the reason for functional errors and performance problems in a modern distributed software architecture is becoming increasingly difficult. IT departments can seek to throw more hardware and software at the problem, but that cannot solve the costly and time consuming software setup and configuration effort of enabling test and pre-production environments for multiple distributed teams to use.

Customer Example: A leading bank which has grown by acquisition found that they could not effectively load test middleware which tied more than 70 acquired systems together using their existing tools. They had several developers attempting to hand-code “stubs” and “responders” to replicate the environment for 2 years, with little success. Using Service Virtualization techniques, two test engineers configured realistic models of the current bank middleware systems. They replaced the multi-year “stub” project in a few months and grew the practice to include another half-dozen teams across the bank. They have avoided up to $30M in costs for provisioning new environments over the course of a year.

Solving Inefficiency – Every business faces increased pressure to meet customer and regulatory demands with greater agility, and leveraging distributed software components and integration frameworks can add agility to our ability to deliver expected business outcomes. At the same time, the service-oriented approach to delivering software makes it harder to ensure service level performance and quality, due to an increased rate of change in the environment.

Customer Example: A US Federal agency needed to prove that their technology selections and architecture elements were performance ready, before they were put in place in the live environment. By virtualizing the rest of the environment and its expected response times and load variability, they were able to compare benchmarks and select the appropriate technology for their architecture, without needing to “bang” on critical live systems used for weather, air traffic and defense purposes.

Summary

Hardware virtualization is becoming ubiquitous in large enterprises, offering immediate cost savings and efficiency. While the concept of desktop and server virtualization has great value for optimizing less utilized systems, it is challenged in replicating heavily utilized or constrained systems such as mainframes, incomplete components, or third-party services.

Consider how much you invest in “under-utilized” hardware and desktop OS infrastructure, versus the high cost of integrating and maintaining traditional software test environments. There's a high probability that the development and integration cost of those traditional test environments exceeds your hardware costs by several orders of magnitude. The prohibitive expense of software test environments means applications are not tested early or often enough, which leads to even costlier failures in production. Virtualization of the behaviors of over-utilized systems can address this serious shortcoming.

Instead of copying the contents of a hard drive or replicating a piece of software running in an OS, Service Virtualization focuses on modeling the communication paths “between the boxes” – components such as web services, databases, RESTful services and asynchronous messaging. This lets teams break their dependencies on the bottlenecks which are preventing them from getting their jobs accomplished.

Development and testing teams need access to an ever-increasing number of services and systems of record that are not readily available. Service Virtualization is a strategy for letting teams take the principles of virtualization beyond the data center, simulating more distributed, complex environments, where significant value remains to be realized.

This concludes our series on SV and Over-Utilization. I'd like to thank Ken Ahrens for coming up with this unique way of looking at virtualization. We hope you find parallels of over-taxed and unavailable systems within your own environment that offer similar opportunities for efficiency improvement.

What are Modern Applications Anyway, and Why Do We Care?

2009-09-24T14:47:08-05:00

You may have noticed lately that a new terminology has surfaced in the way we talk about testing, validating and virtualizing software architectures of a certain type: Modern Applications. Most recently it was used in our release on the HP / iTKO agreement, and you can also see it now on our website. So why is this distinction important?

Understandably, the first thing that may spring to mind with "Modern Apps" is an architecture akin to a Mondrian compartmentalized painting, or maybe some kind of new wave synth music. However, the use of "Modern" makes a lot of sense in encompassing several commonalities inherent in today's enterprise software, without requiring adherence to a specific architectural approach such as SOA.

In short, "Modern Applications" are today's class of distributed, multi-tier, object-based, flexibly assembled software architectures, used to support complex business workflows. Modern applications may include all of the above technologies or architectures (SOA, REST, BPM, Virtualization, MDM, Cloud Computing, SaaS, what have you). While that sounds very general, that defines the kind of software we see getting put in place in most leading enterprises today. We are seeing analyst traction in this sense too, for the natural result of "legacy modernization" should, by rights, lead to "modern applications."

A couple days ago we had a live webinar with Perficient's chief technologist and SOA blogger Eric Roch - and we started with SOA but quickly migrated to other integration platforms, the use of Service Virtualization and Data Management which are very different from SOA, and there are many other aspects of Modern Apps that are not relevant to SOA. (You can watch a WebEx archive of the 9/26 webinar with iTKO and Perficient here.) Eric recently wrote about this in his blog: "Do We Need more than SOA for a Modern Application Architecture?" as well:

"While SOA is certainly a big part of the solution, a total modern architecture also includes leveraging business process management (BPM), data management and Web 2.0 technologies. SOA alone does not implicitly improve governance, IT strategy or business alignment, and top-down, strategic SOA has been largely disappointing."

We definitely agree. What is relevant is that this category of applications are pulled into an environment where they are built, leveraged and shared among multiple development teams and business partners - Modern Applications are not "stovepipe" architectures designed and built by a single team with a rigid set of protocols. Modern Apps are the kinds of flexibly assembled, agile software structures that modern businesses demand, in order to be competitive.

SOA, even if the term fades away, would be considered modern as a model for designing and building applications. However SOA alone leaves out a lot of other strategic and technology advances we are seeing in today's enterprise. We hope this explains our use of this admittedly broad term adequately. Your thoughts on this topic welcome!

Webinar 9/23 with Perficient's Eric Roch: Addressing SOA Quality Gaps

2009-09-17T11:54:00-05:00

Many of our initial customer discussions focus on assessing the "health" of your SOA and Integration efforts. How do you measure health in the distributed software applications like we see in every major enterprise? It's not just about measuring application performance and response time.

There are many details such as functional accuracy, time per release, and availability for testing that directly impact how healthy the IT environment will remain over time.

You may remember last year's blockbuster "ESB-Test-Fest" webinar with a leader from one of our key integration partner leaders and the author of "The SOA Blog," Perficient's Chief Technologist Eric Roch. Well now we have another one coming up that takes the next step into testing and validating the entire SOA application lifecycle with Eric and iTKO's co-Founder and CTO Ruston Vickers - two guys with as much practice experience solving the deepest integration quality challenges as you'll find.

Webinar: "Overcoming SOA Integration Testing Challenges and Other Critical Gaps to SOA Quality and Efficiency" on iTKO.com

September 23, 2009 - 1:00-2:00 PM Central
with Eric Roch, Chief Technologist, Perficient, and Ruston Vickers, CTO & Founder, iTKO.

Bring along your questions to this one-hour session and hope to see you there.

Virtualizing Over-Utilized Systems: Part 5 - Virtualize the Behavior

2009-09-14T07:49:14-05:00

This is the fifth post in our series based on the new whitepaper "The Next Frontier for Virtualization: Over-Utilized Systems" by iTKO (Ahrens, English). In this issue we cover how the practice of Service Virtualization provides solutions for over-utilization of core systems and services needed for testing and development.

Hardware Virtualization Can’t Solve Bottlenecks of Over-Utilized Systems

We’ve reviewed why systems are becoming over-utilized, as compared to under-utilized. The problem is that the hardware and desktop/OS virtualization tools we have at our disposal are great for optimizing under-utilized resources, but they are often incapable of virtualizing the behavior of over-utilized systems.

Even the largest virtual server deployment and centralized management of VMs may not allow for the virtualization of 3^rd party cloud services or mainframe assets. However, these same concepts of virtualization can be applied in new ways to eliminate bottlenecks in our overtaxed core services.

A Solution: Service Virtualization (or, "SV")

Service virtualization involves the simulation of software service behavior and the modeling of a virtual service to stand in for the actual system during development and testing. Think of it like a “stunt double” for your most constrained, critical applications. Service virtualization addresses each of the aforementioned hardware virtualization limitations such as:

Providing 24/7, on-demand access to ready test environments managed on your terms
Removing capacity constraints of over-utilized systems
Addressing test data volatility across distributed systems and conflicts among teams
Reducing or eliminating the cost of invoking 3^rd party systems for non-production use

While Hardware Virtualization focuses improvements on the systems an IT team can control, such as servers in the data center and desktop, the complementary practice of Service Virtualization is geared toward optimizing the distributed and shared types of applications that are resistant to being imaged as a hard drive in a hypervisor: the over-utilized systems such as mainframes, partner-managed services and components under development by other teams.

Let’s look at a sample enterprise with 3 downstream dependencies after applying SV techniques (from the bottom up):

Mainframe access problem eliminated: During an instance when the key transaction system is available, it is captured and reproduced as a sophisticated model of that behavior, which is then hosted as a virtual service and accessible 24/7 in a virtual service environment.
Database access and data volatility eliminated: Rather than having to negotiate set up and removal of test data sets from multiple sources and partner systems, the team can get a stable set of data to validate scenarios without impacting the live systems or other teams’ test data.
Mitigated an unavailable or incomplete system exposed through a Web Service: Construct a Virtual Service from a WSDL, SOAP documents, XML samples, and other artifacts, whether or not the service, or the back-end system is available for testing. If desired, the Virtual Service could be accessed as a “pass thru” to intercept test transactions, while allowing live transactions to continue through to the real service and legacy application.

Coming up: 6th and final issue in this Series: Customer Examples of SV in practice, and Conclusion.

Hitting the right QA Notes with SD Times' Feinman

2009-09-12T12:32:37-05:00

Recently I had the chance to catch up SD Times' Jeff Feinman for an interview about the latest trends we're seeing in software QA. In the resulting article, "Hitting the Right Notes in QA", Jeff surveyed practically every software testing vendor in the field, from unit-and-code coverage firms to the functional UI types of tool providers.

The article talks about the importance of metrics and measurement tools that support testing - comparing software testing in today's complex environments to attempting to play a symphony, saying it would be "like a symphony orchestra getting up on a stage without a single sheet of music and trying to play Bach or Beethoven."

The golden rule of software testing - the earlier you find bugs, the less costly they are to fix - was mentioned as the driving force for measuring the value of testing. Cost-per-bug found in development is indeed much cheaper.

Agreed - however there is also the aspect of automation closer to integration time which must happen in these complex environments that must happen - as there are simply too many possible combinations and points of failure in today's distributed and "loosely coupled" environments like SOA. How many aspects of enterprise software are really built from scratch, and how many are reused or re-purposed from existing components.

I wager that going forward in QA, the big challenge in testing is no longer at the code level, or even the user interface -- that isn't where the big chaos is happening, and there are already a number of processes and tools that do that successfully now.

All in all appreciated the perspectives in this industry-wide QA survey.

Webinar 9/17 w/ iTKO & Guest Accenture - Test and Develop Across Distributed Teams

2009-09-09T14:06:56-05:00

We have an upcoming event happening on the IDevNews network that we are very excited about participating in. iTKO and guest speaker Accenture are co-presenting a webinar Thursday, September 17, on achieving quality among distributed systems and teams. iTKO Chief Scientist, John Michelsen, and Accenture Partner, Jeff Wilkinson, will present and answer questions on the topic and related solutions.

We've had great success partnering with Jeff's group at several key customer engagements, demonstrating how application testing and integration expertise can be aligned with LISA validation and virtualization solutions to improve collaboration between client and partner software delivery teams. So expect this talk to be full of good customer examples encountered in the field.

Webinar: "Achieving Quality & Agility Across Distributed Test & Development Teams" with iTKO and Guest Speaker from Accenture, September 17

Here's the session information:(Now Available as an Archive)

Date: Thursday, September 17th
Time: 1:00 PM Eastern (10:00 AM Pacific)
Speakers: iTKO Founder & "Chief Geek" John Michelsen
Accenture Partner, Global Delivery Network, Jeff Wilkinson
Moderator: Vance McCarthy, Editor, Integration Developer News

Abstract: This webinar highlights how Accenture is applying a Center of Excellence approach to testing, including best practices to increase quality and cost savings in complex, modern applications. iTKO’s LISA Test and Virtualization solutions will be featured to show how complete, automated testing and the elimination of dependency constraints across the software lifecycle can drive down costs and enable faster time to market. Join “Achieving Quality and Agility Across Distributed Test and Development Teams” for proven advice and examples of how enterprise customers are improving their software quality and bottom-line results.

Virtualizing Over-Utilized Systems: Part 4 - Limited Capacity & Inefficiency

2009-09-01T00:22:00-05:00

This is the fourth post in our series based on the new whitepaper "The Next Frontier for Virtualization: Over-Utilized Systems" by iTKO (Ahrens, English). In this installment, we discuss 2 of the big 3 challenges of Over-Utilization: Limited Capacity and Inefficient Systems.
Over-Utilization Due to Limited Capacity:
With the low cost of hardware, you would assume that having environments that were just too small would be a thing of the past. But there are reasons why some key systems may continue to be under-sized:

Unexpected Growth – It's great when a new core service gains popularity and adoption across the enterprise. Perhaps it is an indication that well thought-out designs are being adopted. But the downside is that too much interest can drown a new service under a deluge of transactions. Or many parties may need to jockey for position to get their test data loaded into the new service.

IT Re-alignment – In some cases, companies (and their entire IT infrastructures) have been acquired, or hardware was only partially upgraded. For instance, the production environment is fully built, but pre-production is scaled down. This can be a problem when multiple teams are trying to integrate their systems to a common environment that is a moving target. Common strategies of dealing with this problem are to schedule the execution of tests across all teams, so they are not all hitting the constrained resource at the same time, which limits the new company's ability to take new functionality to market.

Expensive to Duplicate–Today's composite apps can involve a dozen components with the overall application no longer in one team's full control, so virtualizing one system doesn't make much of a difference, because of the interdependent nature of the application. Who cares about saving a few thousand dollars in hardware costs, when the software and maintenance effort of a complex system can cost millions?

Over-Utilization Due to Inefficient Applications
As much as we'd like enterprise applications to run smoothly, it's more common for systems to run slowly. This problem is exacerbated when your application might be performing poorly, and to improve this metric you depend on testing against other services/systems that are also running slowly.

COTS Products – Vendors have to write "one size fits all" types of applications. The result is that without investment in customization, these apps will run more slowly than if your exact use case was coded by hand. As the shift goes from developers writing nearly all the code in the past (e..g legacy COBOL) to focusing more on just writing the business logic (e.g. BPM), the amount of third-party code is skyrocketing. Unfortunately there isn't much that can be done to optimize third-party code in the short run, which may drastically affect testing and analysis of the code that your team *does* write.

Older Technology – Despite earlier predictions, the mainframe does not seem to be going away anytime soon. If your mainframe has a large partition for development and test then consider yourself lucky. Most IT shops have to deal with very small LPARs resulting in scheduling conflicts or adjustments that must be made to test strategy.

Customer Example: A large financial services company was unable to properly load test applications going to production. While they had dedicated environments and test cases, their small test mainframe was at CPU capacity while the non-mainframe apps were only at 30%. They never learned the breaking points of their system until they actually pushed code to production, a risky proposition. Eventually they found out the hard way when they had an outage that took out all transactions for their largest customer.

Popularity Reduces Flexibility – Being popular can be a curse. If you change something fundamental about your service, lots of other teams may need to be involved. Those other teams may have more clout, and could try to block your code changes from being made. This can result in possible efficiencies never making it into your core services, so they grow slower over time.

Compliance – Implementing features such as robust logging, audit trails, governance, monitoring, security, etc. can significantly slow your app. We typically think on the positive side that these features bring to visibility, but early in your development cycle you may not care too much about these benefits.

Next installment - Employing Service Virtualization practices in addition to your existing hardware virtualization approaches.

How Soon are BPM and SOA Merging?

2009-08-27T23:37:00-05:00

I just joined in another fast forum on eBizQ with a good question posed by uber-SOA blogger Joe McKendrick:
"There has always been a huge cultural divide between the business folks, who felt that they own BPM, versus the IT folks, who own the architecture or the technology architecture, which would be SOA. So do you think BPM and SOA are going to merge, and if so, how soon?"
Judging by the rapid-fire responses alone, I can tell this strikes a nerve with lots of the experts. I can certainly understand the concern about yet more redefinition and lumping together of terms that aren't really related. However, I don't see this confusion as a problem in practice as much as others might.
In our experience the most successful SOA implementations are being driven at a high level by a BPM process, even if we aren't talking about directly connecting a process engine with a SOA registry for instance. It isn't going to happen overnight, but the SOA implementations without business process orientation will die out and be replaced by business-process driven SOA, which the business can justify budgetary considerations for.
Anyway, yet another good conversation and I invite you to take a look.

Virtualizing Over-Utilized Systems: Part 3 - Common Bottlenecks

2009-08-27T03:41:00-05:00

This is the third post in our series based on the new whitepaper "The Next Frontier for Virtualization: Over-Utilized Systems" by iTKO (Ahrens, English). In this installment, we will break off one of the big 3 challenges of Over-Utilization: the "Common Bottleneck" that constrains multiple teams attempting to develop, test and integrate enterprise software. Conventional virtualization can break under the strain of attempting to eliminate these bottlenecks, as they are often too large or too complex to be replicated on a given VM.

[Nifty picture I took in Tierra del Fuego, Argentina - constant 50mph west wind]
Well-respected enterprise application design patterns are actually causing software development lifecycle bottlenecks where numerous teams rely on a key system or technology. These systems become new constraints to development and testing, and push the original goal of time and cost savings further out of reach. Here's the key points where we find Common Bottlenecks:

Design of Core Services – In some Enterprise IT organizations, a few key services are so fundamental that they are used in a majority of projects. From the standpoint of re-use this is exactly what we were looking for, but while designing, testing, and running our applications, these core services create dependencies which slow teams down, because unlimited access for software teams is often considered too risky.
Examples of core services include:

o        Customer/Account – An example of one of the most commonly created core services, so there is a single place to get customer records.

o        Billing – Customers don’t want to receive multiple bills from the same vendor, so this is also a common core service to create.

o        ERP System – Especially for internal purposes (HR, Financials, etc.) companies adopt a single platform and require everyone to integrate.

Customer Example:

A large North American Telco had issues in replicating their core SAP system across a myriad of environments. Only 3 of 7 non-production environments have an SAP instance they can use. Also the training system does not have a copy of SAP, so they cannot train their Customer Service Reps on applications that integrate with SAP until they are rolled into production. The result is that bugs are not found until it is too late to repair them and still deliver projects on time.
Data Warehouse – Numerous services depend upon the same set of data, especially for account information. In order to make data available for development purposes, this data usually comes from a production “refresh” as needed, and is put into pre-production.

Data setup & teardown – Managing relevant, stable and secure data in the environment can become a huge drain on team productivity, sometimes consuming 60% or more of testing time.

Data volatility - When data is treated as community resource for multiple development and test teams, other teams can pollute the test data (for instance deleting a key user required for your regression test suite).

Secure data - In addition, live data must be suitably scrubbed or “desensitized” to protect unauthorized tester access to account information, which is a time-consuming process.

Cloud Services – Saving money by using external service vendors on a pay-per-use basis is great, but not when they get in the way of being able to do testing and development. This means we have another team we need to coordinate with from a data standpoint (test accounts, etc.) and potentially we are charged a usage fee when calling on these services.
Incremental fees are fine if they are a part of a revenue-generating customer activity in the live application, but they are a problem when they become a development cost to the business. For instance, how can we avoid skyrocketing fees for load tests that include a cloud service, or analyze negative ramifications if the cloud service doesn’t meet their SLA?

Customer Example: Insurance in the Cloud

A regional insurance company needed to be cautious about how they run their performance tests for adding new auto insurance customers. Among other things, the new client application performs a credit check and runs an automobile history report. During one of their routine load tests against the new app, they sent actual requests for automobile history reports to a third party, resulted in a $15,000 access charge. After this unexpected charge they simply chose not to test any third-party service calls until they reach production.

Next installment: Limited Capacity and Other Inefficiencies

Virtualizing Over-Utilized Systems: Part 2 - Under vs. Over Defined

2009-08-24T11:02:28-05:00

Here's an introduction to the concept of "Over-Utilization" in terms of Virtualization of software and services. This and 4 more posts in this series will cover concepts detailed in a recent whitepaper by Ken Ahrens and Jason English, "The Next Frontier for Virtualization: Over-Utilized Systems", downloadable on iTKO's site.
Are we Virtualizing the Path of Least Resistance?
Hardware virtualization technology is now par for the course in enterprise IT infrastructures. Enterprises can realize rapid cost savings through the use of hardware virtualization to reduce server proliferation, consolidate desktop environments, and run numerous apps with a smaller combined hardware footprint. The key value espoused by leading virtualization vendors (such as VMware, Microsoft, Citrix, etc.) is making better use of “under-utilized” servers that do not require dedicated system resources.

But what are we supposed to do about the kinds of systems that are not good candidates for hardware virtualization? What about the critical over-utilized applications that create bottlenecks among teams that need access throughout the software development and testing lifecycle? Hardware virtualization has a much less compelling value proposition in this scenario.

A strategy to apply in this case is the use of Service Virtualization (SV) to reduce multiple team dependencies on over-utilized systems, allowing them to work in parallel. Let's take a look at the utilization patterns encountered in enterprise IT environments, and which type of Virtualization (Hardware virtualization, Service virtualization, or a combination of both) should be applied to receive the greatest value.

Defining Utilization Patterns

For simplicity, we’ll just categorize enterprise systems into one of 3 categories of utilization and figure out what this means from the standpoint of virtualization:

Under-Utilized systems have low CPU/memory/disk usage, or otherwise there is significant capacity on the system to run additional applications.

Over-Utilized systems may have high CPU/memory/disk usage (for instance a transaction mainframe or multi-terabyte system of record), or perhaps they are services outside of the control of the Enterprise, with usage limitations or per-use costs (such as third-party services, cloud computing, SAAS, etc.).

Properly Utilized systems, which are working optimally, and can be left alone.

Causes of Under-Utilized Systems:

Infrequently Used – Low usage does not necessarily equate with low business value (as a rarely used service could process very expensive orders), but it does usually mean that the system could get by with less processing power during those hours when it is not used.

Excess Capacity – For teams that didn’t have the time or resources to do significant capacity planning tests, many architects hedge on the safe side of providing too much capacity rather than too little. At some enterprises this creates a capacity glut of servers and bandwidth.

More Efficient Software – Let's give developers and testers some credit, they can build very robust software, or perhaps optimize the software performance and use of system resources over time, to get their services to run very efficiently.

In many of the above cases of under-utilization, conventional Hardware Virtualization approaches have proven successful in eliminating excess capacity by consolidating servers and applications. Performance testing and benchmarking can help identify good targets for hardware virtualization, but do these under-utilized systems represent the greatest cost and risk to IT?

Causes of Over-Utilized Systems:

Why would systems end up over-utilized and unable to handle ongoing transactions, particularly in supporting software test and delivery teams?

Common Bottlenecks – Literally this means that many teams depend upon access to the same constrained system or service. Usually this is due to a design decision early on, but in shared environments, the usage pattern is hard to predict when a service is called upon by many customers, and other teams’ services in unplanned ways.

Limited Capacity – More likely for services that have been in production for some time, it’s not hard to imagine that new technology teams get hardware upgrades, while older technology gets left in the dust. Or the production environment may be re-built, but pre-production servers needed by development and testing may not get the same upgrade due to budget constraints.

Inefficient or Unscalable – Unfortunately, technologies and platforms may not provide the scalability initially advertised.

While we hope that responsible architecture, integration and deployment strategies will result in proper system utilization, it typically doesn’t work out that way in today’s heterogeneous, distributed application environments. There are simply too many variations in how the systems are connected and used, and too many components in the environment that aren’t under one team’s control.
Next post in this series: Outlining the Challenges created by Over-Utilization of key systems.

Virtualizing Over-Utilized Systems: Part 1 - A new concept

2009-08-19T22:32:43-05:00

One of our really savvy Architects Ken Ahrens recently had an Aha! moment while visiting a financial institution. The customer was asking how our definition of virtualization was different than the more commonly known types of hardware and desktop virtualization already happening in most IT shops.
That's when it hit him: You are already familiar with virtualizing under-utilized systems -- servers and specific desktop OS configurations that don't need as much dedicated hardware and capacity as expected. That's certainly useful for trimming some IT costs.
But what about the over-utilized systems -- those critical and costly applications, services and mainframes that are just too big, or too remote to be replicated on a VM? The over-utilized systems represent a lot of cost, and big constraints for our ability to deliver and test software. Surely there must be a way to virtualize these too?
Well after a lot more analysis on this topic with Ken, we have a new iTKO whitepaper, "The Next Frontier for Virtualization: Over-Utilized Systems," that addresses how companies can identify and start to virtualize these over-utilized systems.
We'll be posting some additional notes and excerpts from the paper explaining the concept here in the future as well.

HP to Resell LISA Virtualize - The News

2009-08-13T12:34:37-05:00

iTKO and HP recently signed an agreement which created a significant piece of news: "iTKO's LISA Virtualize Software to Be Resold by HP to Reduce Testing Costs and Accelerate Time to Market for Modern Applications"
There is a lot of value to capture here. In every major enterprise IT shop we go into, we find that there are a growing number of constraints to testing created by what the industry is calling "modern apps" -- highly distributed, componentized, loosely coupled applications that no longer reside within a company's four walls. Call it SOA, SaaS, Cloud, pull in BPM, legacy migration, etc. and you have a recipe for quality chaos.
Many of the systems we need for software development are supporting live transactions, contain critical customer data, are inaccessible to the team, and perhaps even charge significant fees for each access. IT shops can literally spend 40-60% of their development time trying to configure test beds, code "test responders" or stubs of needed services, and massaging data into and out of the system before and after each functional and performance test. Faced with these constraints, companies simply do not test as early or often as they should, and quality and performance suffers.
We are familiar with virtualizing servers and desktop apps, but what about the above highly constrained pieces that make up a modern app? These systems are resistant to hardware virtualization, but they can definitely be virtualized behaviorally with LISA, so now HP test and development team members can get access to them on their terms.
This functionality is complementary, but very different from the well-known practice of hardware-based virtualization, in that it is not imaging a hard drive or desktop OS as a VM. Instead it is an automated software-based solution to virtualize the behavior of downstream dependent systems, services, data and components that may not be under a team's control. There are huge benefits to be gained in terms of faster delivery cycles, fewer production errors, and much lower cost and effort of managing test environments, coding stubs and managing test data.
It's a fact that there are more performance and functional testers with HP's testing software installed than any enterprise solution on the planet. And every one of them could stand to benefit from having robust test environments that are readily available to work with. We think this is great news for the software lifecycle when the world's largest technology companies are working to solve some of the largest problems out there.
Stay tuned for more news about this partnership here.

Recent coverage of the announcement (in progress):

Dana Gardner in ZDnet: HP Partners with iTKO on LISA Services Testing Suite for SOA, BPM

Virtualization Journal: We particularly liked an article from Maureen O'Gara: "HP's Got a New Girlfriend Called LISA"

Vance McCarthy in IDevNews: "HP To Sell iTKO LISA for Virtualized SOA, Apps Testing"

eBizQ "iTKO's Virtualize Software to be Resold by HP"

StickyMinds.com News

Official iTKO release on BusinessWire.

How the Recession Affects CIO Budgets

2009-08-10T00:13:52-05:00

Interesting article in a recent Forbes magazine. In "IT After the Recession," Ed Sperling interviews Gartner's Mark McDonald about research conducted with a sample of corporate CIOs about the economic impact of the global downturn on their budgets. Not surprisingly, 90% of the answers did say they had reduced budgets this year.
The report points out a couple interesting trends that call for virtualization on a greater scale in my opinion. The first is in the management of server environments, an area that is only becoming more constrained. The survey notes that budgetary cuts are focused on reducing third-party and contractor support, or renegotiating support contracts.
That is a good sign compared to knee-jerk layoffs to meet budget goals, but it still will call for a lot more automation and reduction of costs wherever they are available. Hardware virtualization will remain a top priority for CIOs, given the need to "do more with less" with every technology asset, as well as the ballooning amount of storage and processor power required to handle complex workflows.
Second, in the article McDonald points out a discontinuous trend that calls for more innovation: "There traditionally was a relationship between transaction volume and revenues. As revenues grew, so did transaction volume. Starting about four-and-a-half months ago, CIOs began reporting that relationship is broken. Transactions continue to grow even though revenues are flat to down. In banking, for example, balance inquiries are one of the fastest-growing transaction types. It doesn't generate any revenue, but it does generate a lot of work."
That is a stunning report, and one that we have seen corroborated in banking/insurance, travel, defense and other industries - more transactions no longer means more revenue. Today, both customers and business partners expect 24/7 any where, real time data from the enterprise -- in fact a companies' ability to provide good real-time data such as order status, tracking, pricing, balance, etc. are key differentiators in customers' minds.
The same also applies to development and testing transaction volumes -- non-revenue generating transaction fees, bandwidth, and data setup while working to build functionality, can increase much faster than the revenue the business functionality supports. In these cases, Service Virtualization is a valid approach, since you have systems that are either unavailable or not under the teams' control for Hardware Virtualization.
We hope as the CIOs start to employ these leaner practices during the downturn, they will come out stronger in the years to come.

Definitions of the Cloud Abound

2009-08-02T10:20:11-05:00

We have been writing quite a bit here about how the public and business sector are defining the cloud. Here is another useful look at the cloud and how it can be positioned in the range of IT options. Ruben S. Montero provides a quadrant for sorting things out.
The two dimensions he espouses are local vs remote (where are resources located?) and physical vs. virtual (where are resources provisioned?). It is nice to see the options fit in the classic 2 x 2 table. It is also useful to clarify that the cloud is not simply remote provisioning, despite what some providers are claiming. Ruben uses his table to show the movement connected with the three major resource provisioning options (other than your own chip-and-mortar data center) employed today.
As Gartner's Lydia Leong recently wrote, "If you worry about hardware, it's not cloud, even if it is remote." As they added, "...if it's not on-demand, seamless, and nigh-instant, it's not cloud."
Classic IT outsourcing involves a movement from on-site to remote systems, but stays within the physical world of data centers. Cloud outsourcing goes all the way across the grid to move from local and physical to remote and virtual. The hybrid cloud moves from local virtual to remote virtual. Each represents different challenges and opportunities.
One fulcrum for tipping off any given Cloud approach is that of authority and governance. By looking at the way the business or market wants to pay for the solution, and defines and manage service levels, you get a pretty good picture of whether or not a Cloud model is needed (or just virtual hardware/grid is needed) and which type of cloud would make a better target depending on security, authority and control factors (Private vs. Public). Of course, these distinctions will continue to abound.

Modernizing Legacy Apps: Still Sexy?

2009-07-27T13:21:37-05:00

Yes, legacy modernization might be about as un-sexy as you can get as far as IT practices go. But we know that these venerable IT assets are a reality of any serious operating enterprise, and making those mainframes and core systems work in today's architecture is still a very hot topic for discussion.
Our John Michelsen recently joined in a good discussion on the eBizQ Forum. Question posed by one of our favorite bloggers, Joe McKendrick: Are Web services protocols such as SOAP and REST and AJAX effective for building SOA off of mainframe, large systems or legacy environments? Or is message-oriented middleware more advisable?
Well, yes, and yes as John mentions... they better be effective at bringing those systems into a modern architecture, if you are using them. Many developers and architects can get quite dogmatic about SOA vs. REST, or how MOM solutions like ESBs are good or evil. It sounds like a blanket statement, but the longer we are in the business of aligning IT around business, the more ambivalent we must become about these distinctions.
Anyway I think the Q&As going on there are worth reading, and we are seeing a lot of the "usual suspects" in SOA punditry and integration expertise weighing in.

Keeping Development Close in the Cloud with ZDNet's Gardner

2009-07-20T13:17:48-05:00

Continuing our discussion on "Personal Cloud" technology from last week, I had a chance to catch up with ZDNet/BriefingsDirect analyst Dana Gardner, whom we also ran into at the most recent HP Software Universe event in Las Vegas.
In his latest blog: "The Cloud Gets Up Close and Personal," Dana was particularly interested in virtualization and Cloud Computing in terms of freeing up developers from dependencies and costs by replicating the target environment on their local workstation, a capability which is entirely possible today:
"The confluence of SaaS and cloud with application development and the test phase is changing rapidly..." he observed. "Compressing the test phase into the development and production becomes more feasible. And as virtualization becomes more common, building an application or service in its own runtime stack bubble from inception to sunset starts to make sense..."
Indeed, we feel that ANYTHING you can do to eliminate waiting on incomplete environments, unavailable systems, unexpected costs and fees, all of those things are what we need to solve if we are going to really see the value we expect from today's distributed, flexible kinds of apps -- which will likely lean heavily on the Cloud as a deployment strategy.
Having a personal-size development environment is almost as popular with developers as say, a personal-size pizza. And in terms of the costs IT departments currently must spend to enable robust test environments, the cost to the business can now seem as small as adding one topping to that personal pizza. He quoted me from the briefing:
"First, the developer working on a cloud application is free to work anywhere, anytime regardless of whether the real cloud services are available or accessible. If a cloud service for a shopping cart is down for some reason, developers are not impacted since their version of the service is on their laptop. They can also code when they are on a plane, or in another environment with no access to the cloud.

Second, although this is probably first in the minds of budget conscious IT managers, the developer is not running up charges for accessing the cloud services..." [Chris Kraus, iTKO]

Now, this sounds great. But what are the unaddressed frontiers of such an approach? Well, for one thing, Dana is looking for the whole package coming out of it:
"...while we’re combining all the elements of an application and platform from cradle to grave, why not tune the whole package before, during and after development too … then load the entire package as a portable cloud-supported production unit?

Now, that’s a “personal” cloud (I prefer cloud service nodule), but with high service performance output, and far less time in cost in the total lifecycle. Higher overall quality too. What do you think?"
We like it - however with critical deployed systems, even if they are very well tested before they are packaged and deployed by the developer, you still MUST test continuously in deployment, as all of the components and usage patterns the developer needed to have stable on the desktop, may not be so stable or used in known ways in Cloud deployment. We can get most of the way there, but probably not 100% there, as much as I'd like to be proven wrong someday.
In addition we will still see these efforts put into Public and Private Cloud environments, due to the ease of administration and economy of scale benefits of provisioning shared "anytime/anywhere" environments for the software lifecycle.
I do believe we will continue to see innovation in this realm - the personal cloud or "local virtualized" environment enabling multiple individuals and teams to work in parallel - married to next-generation application packaging and deployment options. Indeed there are already some nice technologies out there, which we will discuss with partners in future posts.

Could Open Cloud Computing Continue to Coalesce?

2009-07-16T16:39:27-05:00

I wrote recently about the public sector (US Federal Government) in playing a leadership role in cloud computing. The private sector is also making moves, as James Urquhar wrote in IBM, Microsoft, others align on open clouds.
James noted that representatives of Cloud Computing Interoperability Forum (CCIF), CloudCamp, Cisco, IBM, Intel, Microsoft, and the IEEE-ISTO met. They agreed on a shared goal to promote use and awareness of open and interoperable cloud computing. They also discussed the ability to enable participants to be able to contribute and use results of broad community collaboration. There was also discussion of the possibility of a trade association or marketing association for cloud computing, but no actions were agreed upon at that time.
These kinds of activities typical for a technology that is still developing. I agree with the hope James expressed for open cloud standards operating in a cooperative, rather than competitive, fashion. I hope vendors are able to move forward with some vision around shared interests for Public clouds, while at the same time we see companies employing more specialized and secure "Private Clouds" and other hybrids such as the "Personal Cloud" we recently talked about, for specific software lifecycle purposes.
The release of an Open Cloud Manifesto, originally authored by IBM, was not without some disagreement, as Microsoft objected, and there were plenty of differing opinions thrown around the blogs I read after this announcement. The group's site shows that many players have signed up but, as James notes in another post, the "big four" of cloud computing, Amazon.com, Microsoft, Google and Salesforce.com, are not signatories so far. You can find the Open Cloud Manifesto at opencloudmanifesto.org.
It notes five challenges to adoption: security, data and application interoperability, data and application portability, governance and maintenance, and metering and monitoring. Several of the challenges this collective is concerned with, are also factors that in our experience, customers are very concerned with; especially the quality and validation aspects of governance, and the portability of Cloud services through virtualization of software components hosted in the cloud to eliminate some dependencies and per-use costs.
The manifesto concludes that it is meant to "begin the conversation, not deﬁne it." Obviously these kinds of industry groups are promoted merely as representatives of common interests, and not legal entities. However, we look forward to the rest of the story as the cloud coalesces.

On-Demand App Dev Strategies with Compuworld's Mark Hall

2009-07-12T19:43:38-05:00

I recently had a chance to catch up with Mark W. Hall, an analyst and writer for Compuworld. Mark was especially interested in the concept of bringing development and testing environments into readiness in his blog "On Demand App/Dev? Not yet, but closer."
Mark talked to several vendors that are in application quality related fields, as this can often be an underdeveloped aspect of development. In other words, we are now take an extremely "agile" approach to developing new functionality, and the quality or ability to test all of the components and interconnectedness in an application's environment just hasn't kept up.
Here's an excerpt of Mark's article:
"At iTKO Inc. in Dallas the company's LISA product lets you tests services (in the cloud or in your data center) virtually. Chief Geek John Michelsen says LISA watches how your app interacts with a given service over time, then it creates a virtual instance of the service with all of its behavioral quirks and features that you can test as often as you like. Michelsen claims it is unrealistic for a developer to test against a live service."
[Just as a clarification - I believe we definitely must enable developers to test live services for checking implementation - however I do believe developers and testers cannot get access to the production environment to do so 95% of the time they really need it, especially if it is already handling live customer transactions...]
Mark also asked about hosting these virtual environments in the Cloud, and a resounding YES, we already see Cloud providers and partners offering up Virtual Services capabilities, playing nicely alongside other virtual servers and machines that make up a well-rounded test environment. Look for more from us on this topic soon.

eBizQ discussion: Are we Meaningfully Applying SOA Governance?

2009-07-10T07:51:40-05:00

Editor of eBizQ Peter Schooff recently asked a small group of us pundits to post questions to their new expert Forum site, and this week he ran a question from me that we've been asking our customers: Are we meaningfully Applying SOA Governance?
"While we've seen a large number of firms realizing significant value from SOA approaches, are companies truly leveraging SOA Governance principles across their entire design, development and delivery lifecycle? Are solutions such as Service Management, UDDI Registry/Repositories, Testing and Validation being employed as part of this effort? How can SOA Governance accomplish greater adoption levels in today's enterprise environments?"
Our friend Miko Matsumura from Software AG said a resounding yes, and chalks success up to behavioral change and not tooling: "Today, the 'install software and it just works' concept is all but moot. Despite the desire of software that 'just works', the value obtained by software is pretty much only attainable when the organization changes its behavior, for example in a business process."
Author Michael Poulin chimed in and extended the business responsibility aspect, while cautioning that any organization leveraging the tools before the principles has already lost. Fair point, and one reinforced by Joe McKendrick later in the column.
JP Morgenthal also noted from a technology perspective: "I believe a lot of implementations give 'lip service"' to governance and will implement the registry as part of the ESB support requirements, but do not have a plan for how to move beyond this phase of implementation." This is very much in line with what we've seen. Perhaps this will be the year people start utilizing these tools hand in hand.
This particular topic has been ongoing for some time here (see our blog post from a couple years ago on "Wringing Value out of UDDI Enforcement and Policy" or click on our SOA Governance category). We often find that without disciplined practices, and without a proper means of validation and enforcement of the Policies defined in a Registry/Repository or Service Management solution, the SOA Governance initiative faces some challenges.

In any case I am really enjoying the caliber of these ongoing discussions, and look for us to comment more when interesting topics come up.

Who needs a "Personal Cloud?" - Gualtieri

2009-07-06T15:47:43-05:00

With all of the hulabaloo around Cloud Computing, you'd think the LAST thing we need is yet another kind of cloud, in addition to Public Clouds and Private Clouds (secured within a company's domain). Forrester's Mike Gualtieri, an application development analyst we have enjoyed talking to in the past, agrees in his blog "Cloudmania: Developers Need A Personal Cloud" that we don't need a new cloud for everything in principle, while proposing another cloud anyway.
"What developers could use is a Personal Cloud that would allow them to configure their local environment in multiple way and take it with them wherever they go. I know this sounds like virtualization and it is to some extent, but extend PC virtualization with cloud concepts and you get the Personal Cloud."
What is a "personal cloud" anyway? Is it the "Most Private Cloud?" In essence it is a way for developers to completely isolate themselves from dependencies, by having all of the services and resources they need to work and test in their own "personal cloud" environment right on their own computer. No connection to the Internet or a hosted service needed.
At this point you may ask yourself if this can even be called a Cloud, and if it is even relevant to the real world, much less plausible. Indeed, many of the comments on Mike's blog are challenging the feasibility of a "Personal Cloud" altogether. If you are talking about working with enterprise applications, you can't possibly replicate monolithic mainframes, systems of record, and simulate complex transaction and service environments right on the developer's laptop. Right?
In reality this kind of thing is happening today with the practice of Service Virtualization (you can read a paper on the concept here), when coupled with a localized Virtual Environment (or virtual "appliance") that allows a very genuine simulation of the developers' dependencies to run locally. Even if the moniker "Personal Cloud" doesn't stick, we think he is onto something.
Mike mentions Moore's law: exponentially increased processing power and capacity on the developer's desktop at lower prices. That makes what seemed to be an impossibility 5-10 years ago, a very real proposition today. As developers have more local computing power at the ready, they will be increasingly able to model sophisticated test and development environments within a VM, provided they are detailed, robust, and can perform close enough to reality to be relevant.
Anyway a good original blog and this idea is worth watching as it develops. While there are compelling reasons why Public and Private Cloud deployment are here to stay, don't be surprised when developers ask for the ability to truly work "off the grid" sometimes to move their lifecycles into parallel and eliminate the costs and constraints of access to the environment.

Whitepaper: Demystifying Cloud-Based Testing and the Related Benefits

2009-06-29T13:46:00-05:00

You might have noticed a lot of commentary about Cloud Computing recently on this blog, and it is not a coincidence. In our conversations with analysts, peers and of course customers in the field, we are frequently asked about how this approach can add value to the testing and virtualization practices we support.
We have just put out an iTKO whitepaper on this topic, titled "Demystifying Cloud-Based Testing and the Related Benefits." The PDF is free to download.
There are many hard lessons and new techniques learned from industry-wide attempts to decouple and flexibly assemble enterprise software around key business processes. In order to reduce risk as we move into Cloud environments, we need systematic testing that gets around the roadblocks of inaccessibility and unexpected costs. We need to make use of automation and virtualization of the extended environment as much as possible. Cloud-based services need to be governed and validated as part of a continuous business process.
At this stage of the Cloud's development, there are many vendors and service providers that can contribute value to a comprehensive deployment and development environment solution. This short paper does not seek to recommend a comprehensive solution set, rather it focuses on gaining value within the software testing lifecycle. It outlines the key challenges and the top 4 strategies in play - first steps for virtualizing and testing distributed assets in the Cloud. We hope you find the information useful, and invite your feedback.