The iTKO LISA Soapbox: Modern Application Virtualization, Validation & Quality

What is a Cloud App?

2010-03-19T15:36:35-05:00

We at iTKO are not claiming to coin the term “Cloud App” -- in fact a number of vendors out there have already been referring to cloud apps – for instance, vApps from VMware and others. But I wanted to frame the discussion around what cloud applications really are, and the challenges and benefits that will shape what cloud apps could be.

I do like that we can participate in this dialogue. We’ve been a part of many of the intermediate steps of making customers ready to have a cloud application over the last decade. The more distributed, service-based and loosely coupled applications have become, and the more they are connected through commoditized middleware and messaging, the more viable they therefore become as cloud applications.

Cloud apps must have these attributes, because without them, you would have a hard time delivering the value of Cloud to the business -- being able to interconnect various components from different vendors in different ways, and being able to do so in an on-demand basis.

For instance if I had 4 or 5 different service vendors underneath my application, I could elect to use Vendor A for a time, and then remotely switch to Vendor B for a similar service, based on price, availability, contractual reasons, what have you. This is an important concept, and not a new one. When we didn’t have Cloud applications and the ability to make that switch easily and remotely, the cost of switching was too high and thus, rarely happened.

The IT community tried to address this with SOA, to offer a “plug and play” capability for services, but the infrastructure didn’t allow for it. We did get service-oriented benefits around more modular programming, better reuse, and most importantly the distribution of business logic between teams, and a better way to get teams to iterate on development of components more quickly, without having to create a new application.

However, the ability to put the entire infrastructure in a service mindset really couldn’t have happened without a cloud infrastructure. So, Cloud Apps are just the next step in a journey you are already on. If you are building apps that are web-based, using web services, N-tier, with connected legacy systems, and service-based access to third-party systems or SaaS providers, you are already on your way to having a Cloud App.

The only difference is, you’ll now think of deployment as a service-based activity as well, and you will have a greater ability to switch from one provider to another, when it comes to acquiring and accessing those services.

I hope that explains what we mean by Cloud Applications, and you can expect to see more discussions on this topic here.

Employing Virtual Environments for Manual Testers: It Still Doesn't Suck

2010-03-17T11:46:09-05:00

Here's a brand new article in Virtualization Journal: Employing Virtual Environments for Manual Testers from one of our expert architects, Andy Nguyen.

Here at iTKO, we evangelize virtualizing and automating the entire software development lifecycle. While manual testing is a topic we don't cover as often - Virtualization of Manual Testing labs is a really cool thing that can add a lot of value in any large IT organization. For every development and test automation expert in an enterprise, there can be a whole team of on-site, partner, outsourced and seasonal testers sitting idle - waiting for access to a valid, configured test environment to run their scenarios against.

This article contained a cartoon - you might remember our distant history when we advertised the claim that "Your Testing Sucks" - and nobody ever disagreed with us. So here's the comic by Sal Monella:

Service Virtualization is the best way to solve the above constraint for Larry Lamprey, not buying more hardware or conducting laborious, expensive custom setups of a multi-tier environment behind the UI for testers.

Manual testing is never going to disappear, as someone will always be expected to test application UIs for user experience, etc. However, to eliminate downtime from testing, and conflicts over limited available test environments, ALL testers can use a Virtual Service Environment (VSE) to stay productive. Virtualization of those setups and data scenarios also delivers a huge productivity boost.

Why Stubs Don’t Work: Part 2 – Performance & Virtual Services

2010-03-15T09:43:00-05:00

[This is the 2nd Part of John's article on "Why Stubs Don't Work." For "Part 1: Producers & Consumers" read here.]

The last (but certainly not least important) audience that must weigh Stubs versus Virtual Services, is the Performance profiling team – the folks charged with measuring and tuning an application to meet customer expectations or SLAs in production.

Of course, a developer creating a stub “responder” would write that simple object to return a given response as quickly as possible. Yet, many of the most complex scenarios that would want to consume that stub are in the performance lab – so the team can do capacity planning, and get a feel for how the targeted application will perform or behave, before it’s all fully assembled and delivered to production.

You need a pretty sophisticated instrument to vary the responsiveness and handle all the variety of performance profiles that are needed to support tuning.

For example, I may have “What-if” scenarios -- where I want to performance test my system, with all the back-end services stubbed out. Now, I want to see what happens if those back-end systems respond 20% faster, or, 20% slower than they are supposed to be. With a stub, I can only get an immediate response about as fast as the network can deliver it, which is unfortunately not how the live downstream system is going to perform.

And I certainly don’t have the ability to vary that performance in an “adjust the slider” kind of way – not without, again, building a pretty sophisticated framework for that simulation. And this creates – you guessed it – a whole group of people charged with coding stubs to respond at certain times or with certain payloads. There must be a better way.

Thus the need for LISA Virtualize, and Virtual Services.

Virtual Services are the technology we built on top of the LISA Virtualize framework to bring dynamic behavior to this functionality. LISA Virtualize allows Virtual Services to be automatically modeled or captured from live traffic, flat files or definitions, and then quickly adjusted for data or performance characteristics without coding. Here's a basic picture of how that happens:

Virtual Services have proven, time and time again, that development teams can get out of the stubbing business, and that consumers of these services can take control of their own destiny, and produce their own Virtual Services instead of waiting for a stub.

That’s important, because the producer or Developer of a given service should not be responsible for infinitely providing stubs to Consumers -- every Consumer is going to have their own requirements, and need to validate different scenarios for how they use the service. If there is only one stub available, then consumers will be missing the responsiveness they need.

From the other side of the equation, if Consumers are responsible for building their own Virtual Service models from downstream services, they will understand the kinds of performance levels, data scenarios and business processes they need to validate. And, they would know better how that service would need to integrate for their required scenarios with other downstream systems that are available.

Conclusion: The Tipping Point for Virtual Services is here…

Just like the “roll your own” database I used to make years ago, the requirements for this functionality have reached a tipping point. Once the requirements for stubbing or mocking reach the point where they become an item in your project plan, with an associated timeline and resource cost, you need to think about productizing and looking for solutions rather than building one.

One of the largest telecoms in the world, and a company I respect greatly, was talking about this in their Architecture Review meeting a few weeks ago. I was there, and one of their resources said “I hear where you are coming from - we thought of stubbing and mocking as a simple activity, but we have now evolved an entire career path out of people becoming simulation developers!”

There were literally scores of people whose work function was architecting, developing and maintaining simulation artifacts, or “stubs.” Think about it – what company (other than a couple software vendors) would split off an entire category of developers to go build their own database? The person who runs technology for this entire telecom would not make that kind of investment, without serious scrutiny for finding a better solution off the shelf.

So this is the awkward tipping point we find ourselves in today, moving from stubs and mocks to Virtual Services. People who have been building stubs for a long time out of necessity, are finding it harder and harder to keep this up. Our LISA Virtualize happens to be the first (and, without exaggeration, the only) solution on the market to handle this particular challenge.

Now customers are doing the math -- and rationalizing how we used to do it versus this new paradigm with Virtualize. By no means will Virtual Services create a career problem for so-called “simulation developers” either – I have yet to encounter a profitable enterprise that is seeking to do LESS development, or deliver LESS functionality to the business. What ends up happening is better aligned development to the business needs, both within the company, and with partners. That is of course the goal of any healthy IT organization.

Long story short: I hope this provided some useful perspectives you can apply when thinking about using Virtual Services versus stubbing and mocking in your software development going forward.

Why Stubs Don’t Work: Part 1 – Developers & Consumers

2010-03-11T10:55:40-06:00

I’m frequently asked what the difference is between what we call a Virtual Service, and what developers have done for years, which is “stubbing” and/or “mocking out” a particular service. There are a number of ways I can address this, depending on who I’m talking to.

First, for the Developer, I’m going down the path that Virtualization is now taking on the task of industrializing or productizing the practice of stubbing itself. Some 20 years ago, when I first started development, I wrote my own database, wrote my own development tools, my own middleware to get components to talk to each other – in fact, I used to code all kinds of things that today, I would never even dream of building anymore.

The requirements for proper databases and middleware became advanced, so that it was no longer feasible for developers to roll their own. Instead, those requirements became a category of software, and vendors started delivering solutions for that space, thus we saw the rise of a database market, and a middleware market, and so on. We see that the requirements for simulating components have become too steep to roll your own. So in that light, we see Service Virtualization as essentially the “space” that is coming out of the requirements of stubbing and mocking.

The second answer I would give, more often to the Consumer of a stub or mock is just how brittle these stubs or mocks have to be in order to be efficient. The whole reason why a team would construct their own stub is summarized as: “I don’t need a product for that, because it is something I can make quickly and easily by myself.”

Well if the goal is to do something quick and easy, the best you will be able to do with this artifact, is do a “round trip” to the stub, to prove connectivity of a particular application to that stub. In reality, there is so much more detail needed by the consumer of a service, that calling it “something you can mock up quickly” means you don’t understand the true nature of the problem.

Let me explain: If every customer response for a stub has the exact same profile, and the exact same address, account balance, etc. all with hard coded values and dates, then that may help the consumer of that stub with ONE data scenario. But, what about customers with high account balances? What about old invoices? What about transactions that occurred yesterday – and will they work the same tomorrow when that date becomes 2 days ago?

The problem with a stub that has static data, is that it won’t support the real variety of scenarios that are needed for a real world application. And you can’t just randomize the data – that would be even worse! Consumers need predictability and control over the data they’re seeing. And they need to integrate that data not only from the simulated application, but from all the other services that might be associated with that particular application they are building.

A simple example: Say you have customer lookup in one service, while you have account management in another service. So if you have Account IDs and Product Codes and Balances that are not aligned properly between the two systems, you will have an application that fails, either at the consumer level, or somewhere in the downstream systems.

When you think about data required to support development in this scenario, stubbing a system simply doesn’t hold up. Considering all the things you would have to do to make that stub more real, you would need a pretty sophisticated framework to build those stubs. So you might start out building stubs, thinking “it’ll only take a few hours,” but you end up taxing your whole project plan with the hidden cost of building the first generation of a stub, and then modifying it over and over again. And by the way, they don’t even work very effectively.

This is part one of John's discussion on "Why Stubs Don't Work." Read part 2 of the blog, where he will discuss Performance Lab uses of stubs, and the "Tipping Point" for moving from stubs to Virtual Services.

Seen in SA: The LISA Bank?

2010-03-10T10:06:19-06:00

If you've ever been visited by one of iTKO's fine Solution Architects, no doubt you have seen our "LISA Bank" demonstration scenario of our virtualization and testing capabilities. Well little did we know there's an actual business just outside of Johannesburg, South Africa with the same name.

Thanks Randy for snapping this one. I guess if you stopped by for a demo, you might get a good price on a dealer car. We'll keep it in mind next time we are around the Cape and looking for a discount ride to a LISA demo.

Question: Are Pre-Prod Environments Balooning while Production Infrastructure Shrinks?

2010-03-08T15:33:24-06:00

I recently posed a question to eBizQ and got some good answers, both reasonable and unexpected perspectives. See my eBizQ SOA forum question here: Are Servers Bloating in Pre-Production, but Shrinking in Production due to SOA?

K. Scott Morrison and Joe McKendrick discussed how Virtualization is a tester's best friend in reducing this phenomenon - and yes we have seen virtualization of hardware reducing some infrastructure needs. What about the ones you can't virtualize? I vote the best answer (received by 5:00PM CST) goes to Avi Rosenthal, whose comments highlight why this phenomenon is happening:

"In distributed Service Based, Virtualized and Cloud based environments the number of other environment servers is skyrocketing vs. the number of Production servers. This is true as well for virtualized server instances. The reason for that phenomenon is that Production environments are more controlled because no one can allow uncontrolled Production. The risks of non controlled Development or Test environments are less obvious. As the number of instances grows the number of servers in the non-controlled environments grows more rapidly than the number of servers included in the controlled environment. However, logically the number could be reduced at least to a ratio of 1:3 or 1:4 and not 1:20 if the non-Production environments will be managed and virtualized similar to the Production environment."

Really good point Avi.

However HP's Kelly Emo then came up with the gist of an issue I wanted to address further: the huge rift between measuring costs and effort for Production vs. Pre-production environments. It seems these activities are usually owned by two completely different departments and cost centers within the enterprise as she notes:

"In a majority of customers I speak with, their organization, culture and funding models are no where near having this happen... different silos are managing hardware and software resources and different SLAs are associated with them. In fact, in some cases, SOX compliance keeps organizations from being able to mix and mingle their networks and data between pre-production and operations. That said, I see one place where this can happen.. in the cloud..."

That's a next-generation view. Indeed the cloud is a big piece of the puzzle. However, many of the companies who get excited about the value of deployment savings using VMs and Cloud-based infrastructure to replace servers, simply do not have a reason to care about the number of groups maintaining and consuming dev, test and integration environments.

The same goes vice-versa for pre-production - when you break up development into smaller, agile dev & test teams, those teams each provision and set up their own hardware and software environments in a largely unregulated fashion - and often find out that there are many dependencies they can't copy or replicate via conventional means. This leaves a lot of cost and effort unaccounted for in the wash of the software lifecycle, or as a cost of per-employee infrastructure or support.

You will see a lot more on this topic from us - while we aren't in the business of virtualizing servers and hardware, we are particularly curious about why companies aren't recognizing the value they get by optimizing Production vs. optimizing their infrastructure at every step along pre-production. Is a ballooning number of servers a side-effect of distributed, agile approaches, or is the symptom we are reporting simply a matter of perception? We'll keep you posted as we do more research in this area.

Webinar: No "free beer" in Service-Oriented and Cloud-based Applications?

2010-02-23T08:34:00-06:00

We have harnessed the flexibility of readily available services, which can be assembled into business processes, and provisioned on Cloud-based, pay-as-you-need infrastructure. But despite our best expectations, there still is no "free beer" available in service-oriented and cloud-based applications for the enterprise.

What's raining on our parade? Someone has to pay to make sure that as we increase the number of distributed software components and processes, that the combined slurry of services will perform and function as expected by the business.

Face it, we are caught in a conundrum. While we can develop and assemble new business processes faster than ever from components, we also must deal with the fact that the rate of change and complexity of business software is only increasing. Every new service we connect to has the potential for failure. With that, the quality assurance effort and infrastructure cost of assuring reliable delivery of enterprise applications is also ballooning exponentially.

Let's turn the lights on, and discuss why the advent of SOA, BPM and Cloud Computing approaches, while a good thing on so many levels, also increases risk and cost. And maybe along the way, we will find some practices like robust automation and Service Virtualization that may get us closer to AFB (Almost Free Beer) -- readily available Virtual Test Environments, and increased reuse, so we can improve performance and quality with a much smaller incremental cost.

We are excited to present an upcoming "dual continent" webinar March 3rd for Europe and North America, featuring our partner Accenture's John McEvoy, head of EMEA testing, and our founder and Chief Geek, John Michelsen.

Webinar: "No Free Beer" in Modern and Cloud-Based Application Performance and Quality - http://www.itko.com/resources/webinar_accenture_mar2010.jsp.

Hope to see you on the webinar, and bring your questions for John and John.

Chalk Talks: Validation 101 & 102

2010-02-17T10:32:18-06:00

For this week's Chalk Talk installment, founder John Michelsen revisits what Validation means to today's modern application lifecycle. It is a crucial part of any well-governed software architecture, one that is still too often neglected, once the project test and release cycle is complete.

This first installment covers the basics of Validation ("Validation 101"):

Next, John talks about how we apply this practice using LISA Validate:

We are always looking for more topics, customer stories and problems we can address, so if you have a question for us to answer in our upcoming Chalk Talks, please comment here or contact us.

Chalk Talk: Service Virtualization 101

2010-02-10T16:12:06-06:00

Time for the next installment of our iTKO Chalk Talk Series. What is Service Virtualization and how is it unique from existing forms of hardware virtualization? iTKO's John Michelsen provides an updated primer on the practice of virtualizing the behavior of constrained systems.

iTKO Chalk Talk - Service Virtualization 101 - John Michelsen (3:30)

We have found a greater understanding in the IT community over the last year of how Service Virtualization is a next logical step after the value companies already achieve with Hardware and OS Virtualization. Service Virtualization solves for the kinds of applications you can't easily replicate and drop on a VM for use while developing your own apps - they could be third party Services that you don't have control or access to, "big iron" systems like mainframes and systems of record, and even components that are under development.

2010 iTKO-TV "Chalk Talk" Series - Episode 1: Why LISA?

2010-02-03T09:50:00-06:00

Yes, we're rolling with a new season of educational "Chalk Talks" on how companies can reduce software time-to-market and costs through the practices of Service Virtualization, Continuous Validation and more. For the first one, let's set the stage with an introduction to the high-level problem we are trying to address. Here, John Michelsen discusses "Why we built LISA" and the business challenge that is driving our practices.

Look for future videos from iTKO this season, including even more practices learned from the field. As always, we welcome your feedback and ideas on topics you'd like to see covered - so contact or comment to me here.

iTKO Releases LISA 5

2010-01-26T08:18:17-06:00

Big news this month for iTKO, not the least of which -- perhaps the biggest growth spurt our LISA Suite has had so far. LISA 5.0 contains more new features than any major release we've put out, and that makes sense, since many of the elements contained in today's LISA were more than 2 years in the making, both through development and extensibility in the field to meet new challenges.

Read the press release here:

"LISA 5.0 Product Release Extends iTKO's Position as Market Leader and Innovator with Powerful Virtualization, Test Data Automation, and Defect Collaboration"

Look for a LISA perspective and retrospective on this blog soon from iTKO's John Michelsen, as well as more discussion on how customers are applying new enhancements and features such as Virtual Service Management, Personal VSE, Test Data Automation and Pathfinder Pro.

News Today: iTKO's LISA is Ready for Rational

2010-01-19T12:09:02-06:00

[Official news commentary] -- This morning we put out a press release regarding how we are extending LISA’s value and capabilities to IBM customers. You can read the news here:

iTKO Press Release January 19, 2010: iTKO Achieves Ready for IBM Rational Software Validation

LISA achieved a “Ready for IBM Rational” validation with a Best Practices designation, meaning that LISA integrates seamlessly with IBM Rational Quality Manager. In the press release we talk about how LISA complements the IBM Rational platform, and that our IBM partner status got upgraded to Advanced due to increased customer engagements and wins.

The progress is significant - over the last 2 -3 years we have experienced a lot of interest from the Quality Management space for using LISA Virtualize and Validate to eliminate constraints from testing, by virtualizing the behavior of services, middle tiers, web portals, databases and other components in modern architectures.

Today's applications have many components that are too large, or not accessible for copying onto new test lab hardware, or conventionally replicating via a virtual machine. For joint customers, this makes Quality Management solutions like Rational even more productive, while increasing efficiency in the hardware-based test lab environments that support them. The benefits of Virtual Services are even more evident in Cloud-based development and test environments.

Read further information about iTKO's other LISA Solutions for IBM's enterprise integration and development platforms, such as WebSphere, WMB, MQ-series messaging, DB2 and more.

Insurance Case Study: Managing a Virtual Performance Budget

2010-01-04T13:34:40-06:00

We just published a new case study from a leading Insurance (Property & Casualty) provider, which we think neatly outlines the advantages of using Service Virtualization to help manage performance of a complex, multi-tier application.

In this example, the company basically breaks apart an application-wide Performance Test into all of the steps that make it up, and assigns a target time for each step that makes up that application workflow. The decomposed target time for each step makes up a "Performance Budget" that contributes to the overall performance time.

Without decomposing the performance budget, you get no visibility into which element of the application is causing the hit on response time - you only know the End-to-End response time is not up to snuff (4.0 seconds here vs. expected 2.1 second response time for customers).

Given that, the first impulse of the performance tester is often throwing more hardware at the test lab, trying to get that speed up. This may help a little but it usually misses the mark in pinpointing the real time hog underneath the surface.

This is where we apply Service Virtualization - to virtualize the Behaviors of each of the component services in the end-to-end workflow. Behaviors in this sense are not functional, but they are the realistic response times of each component when subjected to load. By virtualizing the components around your component under test, you can figure out which one needs tuning, and which ones aren't creating performance problems in the overall application. We think this customer employed the practice, with the assistance of our LISA Virtualize product, with a high level of success.

This topic was covered for starters in a paper I worked on with Ken Ahrens called "Virtualizing Over-Utilized Systems" and a series of blog posts on the topic last year. Look for a next installment on virtualizing performance testing from us this year - I hope we can cover some new ground for you "application speed freaks" out there.

The discipline of Performance Testing is sometimes also called Non-Functional Testing (or NFT), but the lines for NFT are blurry when you are also performance testing with the realistic use cases and variable data required to get a true snapshot of real-world performance levels.

We're thankful for IT change and complexity in the 00's.

2009-12-22T12:39:19-06:00

The challenges of change and complexity are what we are thankful for this season here at iTKO - because without them, we frankly wouldn't have much to talk about. Likely, if you are reading this blog, I wager that these two factors might be keeping you up at night. But the source of this change and complexity might not be what you expect.

Simplicity and agility in development, is quite literally becoming the cause of change and complexity in IT, especially in ensuring that a reliably functioning software environment is delivered on time. We have been covering this theme for years - but for some reason it has really turned on the light bulb for IT listeners this year, when pressures to deliver more functionality with less resources have been at their greatest.

Enterprises tried to reduce release cycles and costs by focusing on one part of the lifecycle: development. They've brought in new development tools, techniques, methodologies, to make software development more agile, done in smaller, reusable units, more distributed, more iterative, with ever-shortening release cycles.

Companies have spent billions, in fact, on buying platforms to enable SOA, BPM, ESB, and legacy modernization approaches. It created a new problem however... development is getting faster, but the time and cost savings is more than eaten up by QA and maintenance -- and the overall release timeline is getting longer. This imbalance has continued to grow, and organizations can not afford to keep going down this path.

To resolve it, you have to focus on test AND delivery, to wit: How do I optimize the larger lifecycle to keep up with Complexity and Change in my software? We can't afford the support nightmare of software failures in production, but if we throw resources and infrastructure at testing to keep up with all the changes, our costs and timelines, will balloon.

This is why companies are turning to Service Virtualization as a way to alleviate the huge cost of having a valid system environment available for testing, validation and performance purposes. We are thankful to be here, working to solve this problem, at a time of so much change and complexity.

Hear what some other musing friends feel thankful for on the eBizQ forum here. I personally have also been thankful to see less top 10 predictions, and more things to be thankful for, as we finally head out of what has been a tough decade for many businesses. Here's to a very successful 10's!

iTKO Webinar Dec. 15th with TCS: Modernizing Application Quality

2009-12-10T13:10:00-06:00

Here's a little reminder to stop by and register for next week's webinar with iTKO and TCS, titled "Leveraging Modern Software Quality to Prevent Cost and Schedule Overruns" - running December 15, 2009 at 1:00PM EST. This session will feature Siva Ganesan, the Global head of Assurance Services of TCS, and iTKO founder/Chief Geek John Michelsen.

A sneak peek into this production: a lot of new thinking appears in the content. TCS has one of the most mature software testing practices around, but this webinar focuses firmly on the future. Automation techniques, virtualization of constraints and a rapid, iterative approach are needed to tackle the kinds of advanced, distributed, heterogeneous architectures we need to test.

Sign up for the webinar here: http://www.itko.com/resources/webinar_tcs_dec2009.jsp. If you miss the time, we'll send you an archive link later.

Thoughts from Gartner AADI 2009 Summit - SOA's still on.

2009-12-09T15:50:16-06:00

I recently returned from the Gartner AADI conference in Las Vegas, and had the chance to enjoy dozens of good conversations with IT execs who were there primarily to explore strategies for the coming year. Some thoughts from the conference and sessions I was able to catch:

Budget optimism... Overall it seemed that there was a guarded optimism for re-invigorated architecture and integration projects and budgets in 2010, many people said they were part of a new group or new reporting structure specifically tasked with defining a strategy.
SOA Won't Die - in fact any lead architect or CIO I caught up with says they are quietly going about service-orientation and getting real value from it, even if it's not sexy anymore. After the age of boundless service proliferation earlier in the decade, it seems that much of the talk is now about authority - who owns and manages these services, and how are they compensated for doing so. Recently a lot of the "SOA elite" posted some good discussion on this topic (check out this eBizQ SOA forum here).
Heterogeneity is king. The optimism for integration is tempered by a big dose of skepticism about "one stop shop" vendor claims - even more so than you normally hear from the analysts. Companies are dealing with the realities of multiple acquired systems from different vendors, and looking for ways to modernize their applications.
People understand the distinction of service virtualization. Another pleasant surprise was the understanding shown for hardware/network virtualization vs. the practice of Service or behavioral virtualization of constrained assets that simply can't be captured or dropped on a VM. I believe the analyst community has talked to application layer customers about how these assets can be brought to bear earlier into the lifecycle, even if this practice doesn't exactly fit under defined research fields like software quality, app dev, virtualization and IT ops.
Cloud Computing still uncommon in large-scale enterprises - There was a great deal of content about Cloud-based approaches at the summit, and it is certainly important to get some forward insight. In practical terms however I did not see these $500M+ company IT execs mention Cloud in their basket of must-do items for 2010, other than a notable exception of specifically wanting trading partners to use the Cloud more to ease cost and access issues. (This reminds me of public transportation, "it would be great if everyone else would do it to free up the road...") Certainly a lot of curiosity and predictions around this. I am sure at a less enterprise software heavy conference you would see more cloud practitioners and this will continue to grow.

In any case it was a productive time at Caesar's, and we will discuss some topics we touched on this week at a later time.

iTKO Chalk Talk: Event-Driven SOA

2009-11-24T10:13:54-06:00

How do you ensure successful delivery today's distributed SOA types of applications, when you don't even know exactly how they are structured? Understanding the challenges of an "Event Driven SOA" model means discovering when and how decoupled applications kick off business processes, and how to validate that the business process is complete as messages cross several technologies.

iTKO Chalk Talk: "Event Driven SOA" (2009, YouTube video 5:57)

iTKO LISA founder and Chief Geek John Michelsen describes how Event-Driven SOA came to be a common architectural pattern in today's modern, distributed business applications, and how testing and validation needs to answer the quality challenges inherent in governing these kinds of apps. Example from the Insurance industry discussed.

Look for more educational and interesting video from iTKO soon. We are getting ready to shoot our next season, so if you have deep questions or topics you'd like use to cover, tell us about it at news[at]itko.com.

The Future of Cloud is the Past of SOA: DBTA article

2009-11-14T07:42:52-06:00

I realize I might take some heat from a few Cloud pundits for comparing admittedly disparate items in this new article in Database Trends & Analysis: "The Future of Cloud Computing is the Recent Past of SOA for the Software Lifecycle".

As the hype around SOA faded into the overall idea of enterprise architecture, and Virtualization became a standard procedure for saving costs, Cloud Computing became the Next Big Thing for IT. (Somewhere, a self-described Cloud Expert is groaning while reading this sentence and getting ready to blast it on Twitter...)

The ability to provision a complete solution, and scale it on a pay-as-you-need basis on Cloud-based systems is very appealing. However, Cloud Computing faces an often unfounded lack of Trust - one that has nothing to do with the viability of well-designed and managed software, whether or not it resides within a company's datacenter.

So now, let's compare it to the recent past of SOA, which is well beyond it's "Wild West" days of a bunch of loosely assembled services that weren't appropriately governed or tested. Even 2 or 3 years ago, we found people asking very similar questions about trusting SOA. Now? It's quietly become a common strategy for integration in the enterprise.

I'm just saying, there will be a day, not so far in the future, when IT collectively wakes up, rubs its eyes and realizes almost everything has become some sort of Cloud Computing, some combination of Public and Private clouds. The concerns we hear today about trusting the Cloud will again fade into trusting applications in general: Are they well-designed and implemented with appropriate security?

And we can move on to worry about the Next Big Thing.

iTKO Chalk Talk: What is Validation vs. Testing?

2009-11-13T09:14:56-06:00

Welcome back to another issue of iTKO Chalk Talk. This episode focuses on the discipline of Validation of complex, distributed integration environments, with an example specifically from one of our Defense partners.

We've mentioned before that the US Government is leading the way in many aspects of IT governance (or SOA Governance) - of which Validation is a key component. With so many mission critical systems to support, Validation means more than just business continuity, in this sense it can mean saving lives both at home, and for the warfighter abroad.

iTKO Chalk Talk: What Does Validation Mean? (5:15, YouTube video)

Software Validation is unique from testing - it deals with the need for business and service level continuity of systems, as opposed to a testing checkpoint that occurs along a software release cycle. Validation is what we need to do to ensure that our systems, once tested and integrated, stay healthy - even as underlying services and customer usage patterns change.

Our LISA Validate solution works with many of the leading tools for IT Governance and operations monitoring, such as SOA Registry/Repositories, ALM platforms, performance dashboards, what have you. Pick a set of tools that works for your environment. In the end, validation is not really about the software, it is about making the commitment to ensure that critical applications have the continuous attention and quality coverage they deserve.

Safer Computing Environments at iTKO

2009-11-12T09:45:11-06:00

File under "Day in the Life" - Our good buddy Reid-O just got an office makeover courtesy of some overzealous techies - he should enjoy his new ergonomic, safe computing environment. This is what you get when you are out creating valuable solutions with global partners!

People are Still Testing User Interfaces

2009-11-06T13:51:28-06:00

So the heyday of UI testing is over as far as enterprise software is concerned, right? In modern apps, the meaningful business logic happens behind the webpage, in the service-to-service integrations that serve up the appropriate dynamic data. Testing the UI is out of fashion.

Well not exactly. While one type of manual UI acceptance testing faltered, we see another kind of UI testing taking its place. As much as it is true that there are an increasing number of connections and activities going on "behind the glass" that must be tested, it never will replace the need for thorough testing of the user experience. Why is this?

Enter RIAs, or Rich Internet Applications. For a while, serious business applications that leveraged the Internet largely treated the web page as a "request interface" - you enter all the information into a form, submit the page, and it makes a request to the web server, which gets the necessary information from the app server after plenty of back-end processing. Due to performance and latency issues, as well as some rather buggy initial "rich UI" approaches over the last 10 years, it was a lot safer to keep the browser-facing functionality rather simple.

Therefore, testing the UI became of diminishing value in web apps. We needed to validate the back-end systems - the messaging queues, business processes and systems of record. A symptom discovered in the UI meant you were very late to the game in testing.

Now the UI is shifting back into balance in the testing world with RIAs. As customers expect easier and ever-richer user experience on the web, development teams (and the community at large) have moved some of the business functionality back into the browser, to allow customers to more interactively make choices. Technologies such as AJAX, Flash/Flex, Java Swing, Silverlight and other tools have replaced some of the buggy and slow web UI tools of the past. REST-style approaches further advanced the architecture of richer functionality in the front-end.

With this change comes a renaissance in the practice of UI testing. As seen here, one RIA screen such as a trading or entertainment portal may look like one web page to the user, but actually represent hundreds or thousands of separate requests to different servers that populate that page with content. A mere user click or mouse-over on the screen can again, represent several calls from the browser for dynamic data to be displayed. And unlike the late 90's, these customers are not going to wait a few seconds to see results. Lagging performance, displayed glitches and inaccuracies will send the user elsewhere in a heartbeat.

It takes both human insight: "What is a good user experience and is that what I'm seeing as I test?" AND;
A whole lot of automation: "How can a tester possibly automate the tests I've already done, while remembering to cover all of the dynamic permutations of content that could be displayed in this RIA screen?"

We've been doing a lot of research in this area, and we've posted some new material on LISA Solutions for RIA and Web 2.0 User Interfaces on itko.com, if you are interested in this topic. Customers are bringing UI testing into harmony with service-based testing and validation approaches.

iTKO Chalk Talk - Test Data Management

2009-11-02T18:03:25-06:00

Here's a new feature of the iTKO Blog we hope you enjoy - a series of educational "Chalk Talk" short videos covering topics like Software Quality, SOA Governance, BPM Validation, Testing, Cloud Computing and more. This first one features our founder and Chief Geek, John Michelsen. He has a way of pushing a marker that really makes gritty enterprise software topics seem very approachable, so here's his first installment on the challenges of Test Data Management:

One topic that has been on the tip of most of our discussions and recent posts these days is that of virtualization as a solution for Test Data Management (or, "vDTM" as I like to call it...). This form of vTDM is important because it allows your test and development teams to virtually manage test data outside of the system of record.

We hope you find our new Chalk Talks engaging and informative, and look for more video from iTKO soon. If you have any topics you'd like us to cover in the future, send us a note at info@itko.com.

Virtual Data in the Clouds - Enabling Remote Testing Teams

2009-10-28T16:37:08-05:00

We had a chance to briefly touch on this topic in our founder John's latest webinar with Infosys' Aparna Sharma on "Accelerating Modern Application Quality" -- archive available now.

A question came in during the show that was particularly pertinent to the topic at hand. How can these virtualized services and data models work in the Cloud to allow collaboration with distributed teams, especially offshore test teams? Indeed, without a good approach for virtualizing data in a Cloud-based development environment, you simply cannot get there without wasting about 80% of your effort and cost in testing today's service-based applications.

You expected ubiquitous, flexible access to needed resources with Cloud Computing. So why can testing teams end up sitting on the bench instead of getting involved as early as possible? Much of the challenge lies within the test data itself.

In short there are "Wires hanging out of your Test Cloud" as we illustrate here - especially when it comes to Data. You can't just take a 4TB database or an external SaaS-type data provider and plop them into a hypervisor and provision them in the Cloud. Test Data is a frontier because it is not easy to replicate via known means - it is either too bulky or too dynamic to reliably move to the Cloud, but we absolutely must provision teams with good test data, in order to do enough testing to rely upon these systems for business.

Primarily there is the problem of valid data access needed for thorough testing. Most often, this is locked up in some mainframe or live service that is simply too critical to open up to the testing team. Since business continuity usually trumps new product testing, IT Ops guys will crack down on test access time windows to the live systems and the data within them. So you might have a 2 hour period of access - barely enough time to test an on-premise system, much less a Cloud-based system that is talking to other data providers.

Second, there is the problem of data sensitivity. Even if we have contracts and a good working relationship in place, we don't want to be putting data that needs to stay private in the hands of any third party. So we do need to mask or "desensitize" the data so it follows the expected structure, without BEING someone's social security number or bank account. This has always been a concern in healthcare for instance (see HIPAA guidelines), but the level of certification, compliance and controls around data privacy and security in every industry is only getting tighter. We need to be able to give these remote teams obfuscated data so they can continue testing efficiently.

Third, data setup and teardown continues to be difficult in Cloud environments. We touched on one of our favorite examples in the webinar, a Telco that literally spends 2 hours running tests, and 2 whole DAYS resetting data. Many of these problems wont go away when the Cloud-provisioned system is talking to external dependencies and systems of record that need to be accessed and later cleaned out of running systems. Quite often this is a main point of contention, as a test could corrupt the tests of other teams or even the live systems if not contained correctly.

These are just three concerns. There are many other open issues of how Test Data will be handled when we move to Cloud-based environments. I thought Judith Hurwitz nailed this as one of her observations in her "What are the Unanticipated Consequences of Cloud Computing" post:

"Data will increasingly be seen as a reusable resource that can be used in lots of different situations. There will continue to be strategic line of business applications but they will be more systems of record that keep track of the final result of actions that take place dynamically in the cloud. The value of data is not in its tight packaging as we have been used to for decades but it the flexibility to move, transform, and leverage data. The watch word for data in this new model will be Trusted Data in the Cloud."

Couldn't agree more with this observation. As we move to ever more distributed, flexible computing models, we are supporting these with very distributed development and testing teams. The data needed for the software lifecycle must be shared among business partners, on shore and off-shore teams, so success will all come back to our level of Trust that the Cloud will give us the data we need, and not give us the side effects of data we don't need.

Of course, here at iTKO, we've been stretching out our LISA Virtualize to better support Test Data Management (TDM) solution efforts with the capture, manipulation, dynamic "desensitization" and Virtualization of data used in Cloud-based development environments. Look for more discussion and research from us soon on how expert testing teams are taking advantage of Virtual Test Data in the cloud.

Field Stories - Managing Through the Logistics of Change

2009-10-19T10:51:55-05:00

Here's a new customer story we just completed about working with a leading Europe logistics company on integration and performance testing throughout a comprehensive change process. This large transportation vendor handles both sea and land operations for passengers and freight, and handles a very high volume of traffic through its management and reservation systems.

This situation represents a complete replacement of the custom systems the company cobbled together over the last 20+ years with a completely new solution set, using some of the best-of-breed logistics and TM software packages on the market, along with integration software from Software AG (webMethods) and some Oracle systems of record. In order to get there, this company built a new team to allow a "competency center" around integration, to match the investments it was making in a newer, more scalable platform.

Obviously, with so many critical operations on the line, the integration center immediately identified performance testing as a primary task. They already had individuals attempting to verify response times by testing UI elements such as passenger and ferry reservation websites, with little success, so they turned to the back-end elements, where the harder business process calculations need to be made.

By directly invoking the message broker, integration framework and the system of record behind the browser, they were able to deliver a system that has so far run without critical failures in its first seven months of operations. At the same time by automating the execution of regression and performance test runs, they avoided needing to hire several more testers - estimating a reduction of 33% or more of the testing cost.

You can download the complete story here: http://www.itko.com/resources/europelogics.jsp.

The News - Working with Harris on Complex Validation Problems

2009-10-13T20:29:51-05:00

Today iTKO made an announcement about our strategic partnership with Harris Corporation, a Fortune 200 systems integrator we have been working with on several significant projects in the US Department of Defense and other federal agencies. Read the press release here.

What is significant about this? Many organizations (commercial and government) are realizing that while developers can build and integrate software faster than ever, their testing efforts have proven inadequate at keeping up. These agencies need proactive, automated, and continuous validation to ensure trust and reliability in federated, mission-critical systems that are subject to constant change.

iTKO and Harris are responding to this challenge with a validation and virtualization solution that will appeal to organizations that trust and rely on these systems. Examples include federal emergency management centers, military command and control systems, financial institution trading systems, utility energy management grids, airline traffic control networks, and many others.

While we have been cooperating with leading government contractors like Harris for 4 years on integration projects like these (see this iTKO paper on Federated governance we first wrote 3 years ago...) we have seen a major shift occurring in the way Federal agencies specify and manage their complex applications. They are moving to a shared services / shared components model, where different functional units (or, services) are consumed and managed by different teams, with a central point of certification and testing for these services.

A federated, shared governance model means there is just enough governance enforced at a central level, and authority for managing and using the application components is doled out to the constituents. We compare it to a Federal government vs. state's rights, vs. city council authority model. The federal government might specify a budget for a highway, but it doesn't dictate where parking is located in the city.

This federated model works very well for defense agencies and the many firms that support them, as they are all working toward a common goal - delivering functionality to the warfighter with maximum efficiency. As we have frequently mentioned in this blog, businesses would do well to take cues from how government scales their systems and cooperates to meet goals - they were leaders in SOA and will likely continue to lead the way in many aspects of modern application governance.

Webinar 10/27 w/Infosys & ITKO - Accelerating Modern Application Quality

2009-10-08T14:48:46-05:00

You might say we are between a pillow and a hard place right now when it comes to modern applications. On one side, we have killer new development approaches such as service orientation, BPM, and SaaS that provide us with a much greater amount of flexibility and speed in assembling reusable components and services. Development time and costs have indeed gone down, while functionality gets ever more sophisticated.

However the unintended consequence of being able to build and change applications so quickly comes at a price. Testing and validating these systems has become harder than ever. So many interconnected, interdependent components means an exponential increase in potential points of failure, and an inability to have access to a stable and affordable test environment. So much so, that testing environment and error repair costs are outstripping development cost in many cases.

There's gotta be a way out of this situation. Fortunately, we have some experts ready to help shed some light on these new costs and track down the quality issues in modern applications. We are privileged to have our founder/"Chief Geek" John Michelsen, presenting with Infosys' head of Client Services, Aparna Sharma on this upcoming session:

Webinar with Infosys and iTKO: "Accelerating Modern Application Quality: Testing Strategies for Reducing Risk and Cost in Today's Distributed Application Environment"
Tuesday, October 27th, 2009 – 1:00 PM Eastern (US) Time

Between these two, you have a great deal of experience leading and advising test and development teams in hundreds of engagements with the most complex types of software environments. So I expect nothing less than finding out what works today (and what didn't work) for real enterprises. After all, time-to-market is measured by delivery success, not the rate at which developers can string together new (but untested) functionality. Hope to see you there, and bring your questions!

What's the next big obsession for Enterprise IT Geeks?

2009-10-07T11:23:04-05:00

As an occasional pundit and commentator on IT trends, I've had a lot of chances to watch the "next big obsession" come and go for enterprise IT shops. The one commonality we've seen, is that if the obsession successfully takes hold in business, then it will become so ubiquitous that it becomes kinda boring to the IT guys on the cutting edge.

So goes the case for SOA... We've tracked the "Is SOA Dead?" thread over the last couple years and it was fun, because it points out just how common SOA has become part of the enterprise IT landscape, delivering value so quietly that we almost take it for granted now. Just like "eCommerce" used to be a big deal, then it just became the nature of doing business over the Internet.

So one of these questions I posed to Peter Schooff of eBizQ was this very thing - and he posted it to their Forum - What are Enterprise IT Geeks Obsessed with Today? I was looking for stories, and not the next dot on an analyst wave or cycle diagram. He posted the question and some fun answers there.

Clearly you can't escape Cloud Computing right now as the grand champion of obsession (Make mine Private or Public?). Mobile platforms and smartphones - riding high again. RIAs, particularly REST-type web apps are garnering a lot of geek attention. But what never changes is that these innovative obsessions aren't just about money - they are inevitably tied to doing things better, more efficiently than before. The cost savings and revenue upside are the results of success, but not necessarily the motivator.

I particularly liked Sandeep Gupta's comment in the forum that "IT geeks do not change...they are always passionate about the technology space/environment that they work in.....obsession brings aspiration and passion...leading to effeciency and growth" and that to me describes the accelerating nature of innovation that is going on here.

Also great - Brenda Michelson's comment (via Twitter) about "Hopefully getting things done!" Well put. And Phil Ayers laments that IT folks he runs into "just don't have time to be geeky enough and keep up with, let alone apply, the vogue technologies and methodologies." Yeah, there is probably a little too much talk about the next big thing and not enough application going on. So until next time...

Finding Hidden Value in Test Data Virtualization

2009-10-01T15:18:12-05:00

As you've seen in recent posts here, most of our discussions around Service Virtualization center around the lack of availability of systems, environments, etc. -– i.e. the "Over-utilized" problem. A Virtual Service Environment (VSE) is a solution that helps eliminate these constraints and therefore can dramatically reduce the time to market, and/or infrastructure costs. Also, once these constraints are removed, testing (functional and performance) can be done much earlier, and therefore greatly improve the quality of the released software.

We also talk about the use of VSE to enable parallel development. Being able to simulate dependencies that don't exist, and allowing testing and development activities to happen in parallel. Again, this can greatly reduce time to market, costs and improve overall quality.

However, there is a value of VSE that we don't talk as much about. An issue that has applicability in almost every enterprise environment, is one related to Test Data challenges. Let me explain.

There are at least 4 Test Data related challenges that Service Virtualization helps solve or assuage:

Data Scenarios are difficult to create. In the modern application, data required to support scenarios will have to be synchronized across multiple systems. Not only is that highly time consuming, but due to access constraints, or skill constraints, or various other reasons, creating and synchronizing that data could become a near impossible task. This may either elongate the testing cycle or, even worse, encourage the testers to reduce the number of scenarios that the system is tested under.
Data is Volatile. This is partially related to (1). In a lot of cases, the systems being tested go directly against systems that are either in-production or are being shared by other teams. This causes the data to be volatile (changing randomly and often). This makes the testing process impossible, because any results of the testing against these volatile systems cannot be validated.
Data needs to be Desensitized. In a lot of cases, data challenges could be reduced by using production data. However, production data may have sensitive information like credit card numbers, social security numbers, etc. Not only is this a problem for the direct employees, the challenge is greatly magnified when offshore or outsourcing contractors are used.
Data should not impact production systems. Some test scenarios result in transactions being created that can mess up a production system. For instance, if a test scenario requires a new order getting placed, the order once generated will have to be traced and deleted from the production systems. This is a major pain for the testers, and for the operations personnel.

As you can see from the above examples, Test Data Management challenges represent a huge constraint on the productivity of almost any enterprise software development project of significant scope.

Here's how we have found customers can approach these challenges using Service Virtualization:

Reduce the number of points that need to be synchronized, by capturing relevant transactions, chaining them together into scenarios, and creating a virtual model that can be manipulated without programming.
Stabilize the data so that the test results are actually valid, for instance, maintaining the relevance of variables like changing dates and varying types of customer data entry.
Desensitize the data so it does not violate security and privacy policies.
Substitute a Virtual Service for the production system, so there is no need to impact the critical live system of record to retrieve data. This can be done alongside ALM tools and Service Registries for SOA - essentially routing a known test data request to the VSE instead of the real thing.

In this arena of virtualization, we have been amazed by how widespread these shortcomings are in test data management, with little recognition of any available solution for them. We believe our LISA Virtualize uniquely supports these practices, with automation for capturing, modeling, desensitizing and maintaining virtual test data.

Virtualizing Over-Utilized Systems: Part 6 - Customer Examples & Conclusion

2009-09-28T12:58:59-05:00

This is the sixth post in our series based on the new whitepaper "The Next Frontier for Virtualization: Over-Utilized Systems" by iTKO (Ahrens, English). In this issue we cover how companies are applying Service Virtualization to solve for over-utilization of core systems and services needed for testing and development.

Examples of how SV Answers the Challenges of Over-Utilized Systems:

Solving a Common Bottleneck – When each team can get their own replicated test environment and data provided as a Virtual Service at minimal cost, there is no longer a need to compete for testing time during a small availability window, much less secure a capital expenditure just to get access to a hosted service-based system.

Customer Example: A leading US airline replicated much of the functionality of a partner’s reservation system that charged high per-use fees when used for testing purposes, as well as restrictions on usage due to the fact that millions of transactions needed to be handled on the system daily. Switching to a virtual service model of the system allowed multiple teams to have their own stable environment and data for test scenarios, as well as saving millions of dollars annually (up to $12M) in non-revenue-generating access fees for the live service.

Solving Improper Capacity – Trying to pinpoint the reason for functional errors and performance problems in a modern distributed software architecture is becoming increasingly difficult. IT departments can seek to throw more hardware and software at the problem, but that cannot solve the costly and time consuming software setup and configuration effort of enabling test and pre-production environments for multiple distributed teams to use.

Customer Example: A leading bank which has grown by acquisition found that they could not effectively load test middleware which tied more than 70 acquired systems together using their existing tools. They had several developers attempting to hand-code “stubs” and “responders” to replicate the environment for 2 years, with little success. Using Service Virtualization techniques, two test engineers configured realistic models of the current bank middleware systems. They replaced the multi-year “stub” project in a few months and grew the practice to include another half-dozen teams across the bank. They have avoided up to $30M in costs for provisioning new environments over the course of a year.

Solving Inefficiency – Every business faces increased pressure to meet customer and regulatory demands with greater agility, and leveraging distributed software components and integration frameworks can add agility to our ability to deliver expected business outcomes. At the same time, the service-oriented approach to delivering software makes it harder to ensure service level performance and quality, due to an increased rate of change in the environment.

Customer Example: A US Federal agency needed to prove that their technology selections and architecture elements were performance ready, before they were put in place in the live environment. By virtualizing the rest of the environment and its expected response times and load variability, they were able to compare benchmarks and select the appropriate technology for their architecture, without needing to “bang” on critical live systems used for weather, air traffic and defense purposes.

Summary

Hardware virtualization is becoming ubiquitous in large enterprises, offering immediate cost savings and efficiency. While the concept of desktop and server virtualization has great value for optimizing less utilized systems, it is challenged in replicating heavily utilized or constrained systems such as mainframes, incomplete components, or third-party services.

Consider how much you invest in “under-utilized” hardware and desktop OS infrastructure, versus the high cost of integrating and maintaining traditional software test environments. There's a high probability that the development and integration cost of those traditional test environments exceeds your hardware costs by several orders of magnitude. The prohibitive expense of software test environments means applications are not tested early or often enough, which leads to even costlier failures in production. Virtualization of the behaviors of over-utilized systems can address this serious shortcoming.

Instead of copying the contents of a hard drive or replicating a piece of software running in an OS, Service Virtualization focuses on modeling the communication paths “between the boxes” – components such as web services, databases, RESTful services and asynchronous messaging. This lets teams break their dependencies on the bottlenecks which are preventing them from getting their jobs accomplished.

Development and testing teams need access to an ever-increasing number of services and systems of record that are not readily available. Service Virtualization is a strategy for letting teams take the principles of virtualization beyond the data center, simulating more distributed, complex environments, where significant value remains to be realized.

This concludes our series on SV and Over-Utilization. I'd like to thank Ken Ahrens for coming up with this unique way of looking at virtualization. We hope you find parallels of over-taxed and unavailable systems within your own environment that offer similar opportunities for efficiency improvement.