Performance Matters

Index of Migrated Posts

2007-03-18T09:59:00.000+00:00

All posts from this site have now been migrated to their new locations at Web Performance Matters. This index is a cross-reference to the new site for anyone who arrives here from an old link.

My previous post (see below) also lists new locations for a few recent items that are not included in this list.

Performance Matters has moved

2007-02-22T22:15:00.000+00:00

I have joined UpRight Marketing as a partner, and after a lot of site design and development work, Performance Matters is now being transformed into a new journal with its own domain, at WebPerformanceMatters.com.

I built both sites myself using the Squarespace platform, and I'm now in the process of migrating all the old posts. Other than details of the progress of the migration, I won't be posting anything new here. But check out this post on the new site for all the details of the changes I'm implementing on the new platform.

See you there!
--Chris

Understanding Web Usability

2006-12-18T08:07:00.000+00:00

One of the great things about the Web has always been its democratic nature. Anyone can participate. But once you do, your contributions are wide open to public scrutiny. Good or bad, someone will evaluate your Web content.

People Recognize Poor Design
In the Web's early days, when people were adjusting to this new medium, most online critiques applied to a site's design. If you created a really ugly site, before long your handiwork would end being featured on a site dedicated to bad design, or even included on someone's all-time list of bad sites. Today, the collaborative features of the Web 2.0 environment such as blogs, forums, and widely used folksonomies practically guarantee that truly awful design will receive the recognition it deserves. Such criticism can be constructive; people can learn from good examples of what NOT to do.

Today, blogs and wikis extend their democratic and educational influences beyond site design to site content. This can be tough on the writer who wants to explore new ideas. But as Wikipedia, one of the best example of online collaboration, advises all contributors:

If you don't want your writing to be edited mercilessly or redistributed by others, do not submit it.

This is excellent advice, because the collaborative search for truth online is not subject to parliamentary or academic niceties. If someone advances a weak argument, people will be quick to point out its flaws. And an unpopular opinion can produce flaming responses.

Is Web Usability a Sham?
For example, last week Ryan Stewart, who has his own blog, and also writes a ZDNet blog about Rich Internet Applications (RIAs), wrote that Usability on the web is a sham, arguing that ...

While accessibility and standards are great for the Web, the concept of usability has been overblown. Usability as we define it is basically the rules for how the Web should behave while in the confines of the Web browser. But Web applications don't have to exist inside the browser and relegating them to these antiquated notions of usability is bad for progress.

To support this argument, he used the Back button as an example of a browser usability problem that RIA developers could eliminate altogether.

I think the central idea behind his argument was that the RIA technologies -- because they offer the developer more options -- can be applied to deliver usability improvements in Web-based applications. But I'm afraid he expressed it very poorly, first by over-generalizing in his criticisms of the concept of Web usability, and second by trying to use the Back button -- one of the most intuitive and widely-understood browser conventions -- as an example of poor usability.

Naturally, his post has been generating a small firestorm of responses ranging in tone from expletive-laden outrage to carefully-argued disagreement. In their different ways, both those examples (along with other responses) argue that Stewart's post is full of opinions and assertions that are not supported by any evidence, and that (to put it bluntly) Stewart doesn't really know what he's talking about.

Marshaling the Right Skills
Unfortunately, this is a common problem in the field of Web design. An effective online application must be available, responsive, easy to navigate, and deliver value to its user. This demands a wide array of skills rarely found in a single individual. As a result many sites are designed and built by people who are aware of only a small subset of the issues they should be considering. And all too often, someone in the site development process -- a business manager, someone in marketing, a Web designer, or a site developer -- makes key design decisions without understanding the consequences. And the challenges are even greater when developing a Rich Internet Application.

Simply getting more people involved isn't the solution. Some really bad Web site designs have been the result of design by committee. Even if you follow a good checklist, like this one by Vitaly Friedman, you will overlook some important aspect -- often site performance. The problem is the sheer breadth of knowledge required to do a good job. For evidence, read the Wikipedia entry on Web Design. It's an unbalanced, poorly organized, collection of information that offers little help with the process of creating an effective site.

The only answer is to get better, more knowledgeable, people involved. Start with a good overview of the process, like Usability for the Web: Designing Web Sites that Work by Tom Brinck, Darren Gergle, and Scott D. Wood. As they say in the book's introduction:

To ensure high usability on our own Web projects, we defined a development process that incorporates proven techniques from software and usability engineering, graphic design, project management, and other disciplines. The process had to be practical and lean. It had to allow us to work on multiple projects of varying sizes with fixed budgets. It had to help us keep track of the details that can kill usability and destroy profitability. This book is about that process.

Then find people who understand what the book is talking about to do the work -- and don't interfere!

Tags: Usability, Web design, Worst website, Ryan Stewart, Rich Internet Application, RIA, performance.

Web Design and Mouse Rage Syndrome

2006-12-16T02:17:00.001+00:00

Have you ever been frustrated at a Web site that downloads with the speed of an Alaskan glacier? Or become angry when a favorite site, or your Internet connection, is down. Have you experienced any of these symptoms:

Faster heart rate?
Increased sweating?
Furious clicking of the mouse?
Simultaneous clicking and cursing the screen?
Bashing the mouse?

Come on now -- admit it! Maybe some of them, just once or twice? Because if any of this sounds familiar, you're not alone.

Mouse Rage Syndrome
The consequences of poor Web site performance don't usually make news, unless there's a big outage on Black Friday or Cyber Monday. Then the story invariably focuses on the business lost by companies whose sites were overloaded or down. But what about the effects of poor performance on customers? A recent study by the Social Issues Research Centre (SIRC), an independent, not-for-profit organisation based in Oxford, UK., provides this perspective.

Researchers found that badly designed and hosted websites cause stress and anger, and coined the term "Mouse Rage Syndrome" (or MRS). They concluded that five key IT flaws in the way websites are designed and hosted may lead to harmful health effects.

Those five problems will come as no surprise to any regular user of the Web: slowly loading pages, confusing layouts, excessive pop-ups, unnecessary advertising, and site unavailability. But as I was saying in my previous post, it is not fair to blame IT for these problems, when most of them are the result of poor design choices by Web site designers and developers.

Damaging Customers' Health
The study combined data from a YouGov poll of 2,500 people with physiological tests on a separate sample of Internet users, who were asked to find information from a number of different websites. Tests measured physical and physiological reactions to website experiences, looking at brainwaves, heart-rate fluctuations, muscle tension and skin conductivity. According to the report:

When the test participants came to the 'problem' sites that we had deliberately chosen as comparisons for the 'Perfect Website' evaluation exercise [a prior study], responses changed quite dramatically in most, but not all, cases. While a few managed to stay calm and simply 'rise above' the problems presented by crazy graphics and slow-loading pages, others showed very distinct signs of stress and anxiety.

Some changes in muscle tension were quite dramatic...While this was happening, the participant's faces also tensed visibly, with the teeth clenched together and the muscles around the mouth becoming taught. These are physically uncomfortable situations that reduce concentration and increase feelings of anger.

These reactions, if not managed, can eventually lead to other, more serious, health problems.

Poor Designs Can Kill
According to the SIRC report, "users want Google-style speed, function and accuracy from all of the websites they visit, and they want it now. Unfortunately, many websites and their servers cannot deliver this". The result is consumers seeking alternative websites in a bid to avoid undue stress and Mouse Rage.

Jacques Greyling, managing director of Rackspace Managed Hosting, who commissioned the study, commented:

We believe that businesses that are selling online have a duty to their customers to ensure that the experience is as stress free as possible. The public has shown that it wants to buy online, ... (so) businesses need to provide simple and easy to navigate layouts, whilst focusing on speed and uptime.

If more studies confirm these findings, which will probably seem obvious to anyone who spends a lot of time online, maybe more Web designers will get the message that their poor designs are not just killing the business. They could be killing people -- really.

Performance Matters!

Tags: Social Issues Research Centre, SIRC, YouGov, mouse rage, Web design, performance, Web performance management, health.

Yahoo! on Web Page Performance

2006-12-12T08:00:00.000+00:00

A recent post by Tenni Theurer, who works in a performance team at Yahoo!, appeared in the Yahoo! User Interface Blog. The post begins with the claim that ...

... most of web page performance is affected by front-end engineering, that is, the user interface design and development.

Theurer introduces the Pareto Principle, commonly known as the 80/20 rule, which states that 80% of the consequences come from 20% of the causes. In the case of Web page download time, she argues that the backend systems which generate an HTML document -- apache, C++, databases, etc. -- should be regarded as the 80% of causes that account for only 20% of download time. The other 80% of download time is spent fetching the elements that make up the page, such as images, scripts, and stylesheets.

She presents two graphics to illustrate this point. A table summarizing home page measurements of 8 popular Web sites -- Yahoo!, Google, MySpace, MSN, eBay, Amazon, YouTube, and CNN -- shows that retrieving HTML accounts for between 5% and 38% of their download times, and just 12% on average. The other 88% is taken up downloading and rendering other page elements.

Waterfall Charts
To illustrate the reason why HTML accounts for such a small percentage of all page download time, Theurer uses the diagram shown here [click to enlarge]. I call this kind of data visualization a 'waterfall chart', because of its shape, and because it is similar to the way the waterfall model of software development has always been always depicted graphically.

Theurer says (in blog comment #22) she used the IBM Page Detailer tool to get the measurement data shown in this diagram. I have used this tool, and it produces a Chart Panel similar to the one above, which is a graphical representation of all of the component downloads that make up a single page.

I have not seen much documentation online, but if you take the time to register with IBM and download the software, its Help file is quite informative. It explains that the ...

... IBM Page Detailer attaches to the workstation TCP/IP socket stack and monitors all of the HTTP/HTTPS protocol based communications between your Web browser and the network. IBM Page Detailer then displays web activities in a series of color-coded bars.

It also has quite a long section on Results Analysis and other Considerations regarding Web download times.

This kind of analysis is not new. Since 1998, Keynote has offered a similar Web Content Diagnostics feature in its MyKeynote portal -- here's a datasheet describing the newest version of this service, recently released. For example, the figure on the right [click to enlarge] shows a Keynote diagnostic measurement of the Yahoo Search Marketing page.

Unlike the IBM Page Detailer, which always measures from the desktop where it is installed, Keynote's diagnostic measurements can be taken from anywhere on the company's worldwide network of 2500 measurement computers (or "agents"). This example comes from an agent located on the Verizon network in Houston.

The Lesson for Web Designers
Even without looking at the data in any detail, the message of these waterfall charts is evident from their general shape:

Web page download times rise with the number of bytes to be downloaded and with the number of separate content elements comprising a page.

This fact is not news to anyone whose job involves monitoring or managing site performance. Even for a Web page consisting only of a single HTML file, you cannot compute download time simply as page size divided by bandwidth. That's because of the way the TCP slow-start protocol works, sending one or two packets (typically 1.5Kb - 3KKb), then doubling, and doubling again, up to the receive window size of the client.

But most Web pages these days contain many imbedded graphic objects, CSS files, and script files, all of which extend download times. Most of these files are small, requiring just a few TCP packets to transmit. But every transmission requires a separate "Turn", adding another round-trip delay time (RTT) from the client (i.e. browser) to the server(s) and back. This behavior has been widely documented and is well-understood by performance specialists:

In a previous post here, I described the Web Site Response Time Model and linked to the CMG2000 paper (E-Commerce Response Time: A Reference Model) in which I introduced it.
In 2001, Peter Sevcik and John Bartlett of NetForecast published their paper on Understanding Web Performance, in which they first outlined a formula for computing Web page download times based on total bytes, the number of components, and on round-trip times between browser and server(s).
That formula was simplified and explained in a highly readable paper by Alberto Savoia -- Web Page Response Time 101 -- published in July 2001 in STQE, the magazine of software testing and quality engineering (now Better Software Magazine).

Since then, many articles and papers have reiterated this message. An entire industry segment -- Application Delivery Systems -- has emerged to help organizations offset delays in delivering their Web pages and satisfy customers' expectations of responsiveness. All the same, as is clear from the comments posted to the Yahoo! User Interface Blog, some Web designers and developers still need to learn this lesson.

Tags: Yahoo, Pareto principle, 80/20 Rule, performance, Web performance management, IBM Page Detailer,Keynote, page size, round trip time, RTT, TCP, slow start.

ITIL Crash Course

2006-10-27T05:24:00.000+00:00

Anyone involved with IT these days knows that ITIL is a hot topic, and one that seems to get hotter every month.

ITIL, the Information Technology Infrastructure Library, has evolved from work sponsored by the UK Government in the late 1980's. According to the official ITIL site, it is "the most widely accepted approach to IT service management in the world". For a lot more detail, see the wikipedia entry for ITIL, which describes it as "a framework of best practice approaches intended to facilitate the delivery of high quality information technology (IT) services", and goes on to explain that:

ITIL outlines an extensive set of management procedures that are intended to support businesses in achieving both quality and value for money in IT operations. These procedures are supplier independent and have been developed to provide guidance across the breadth of IT infrastructure, development, and operations.

If you subscribe to any technical magazine, newsletter or blog written for an IT audience, then you have probably seen more than one story on the benefits of adopting ITIL-based approaches to IT management. The latest publication to jump on the ITIL bandwagon is this week's InfoWorld (October 23, 2006, issue 43) -- also available online -- which contains an excellent 5-page article introducing ITIL.

The magazine cover touts the importance of ITIL today: To keep pace with business strategy, IT need a blueprint for delivering services. That's where ITIL, the modern-day bible of governance, comes in. The article itself is arranged like an interview in which the authors, Randy Steinberg and Michael Goodwin, answer ten questions. I won't list each question, because often you have to read the answers for the next question to make sense. But here's a summary.

Ten questions
Question 1 presents three examples to show how ...

ITIL raises customer satisfaction, reduces waste in the IT organization, and lowers operating costs.

Questions 2 and 3 introduce the processes that together make up the ITIL discipline of Service Support, and the related idea of a Configuration Management Database (CMDB) that provides the foundation for storing and retrieving all information about the IT infrastructure, and making timely management decisions.

Question 4 moves to the equally important discipline Service Delivery, and includes an informative diagram showing the central role of Service Level Management as the function that regulates the interface between Business and IT. [Incidentally, this highlights a perennial issue we face in high tech, where the ever-shifting meanings of common terms makes it hard for people to communicate clearly. In this instance, ITIL's definition of Service Level Management covers a narrower range of activities than my previous use of the same term here.]

Question 5 introduces the other ITIL books: Introduction to ITIL, Planning to Implement Service Management, ICT Infrastructure Management, Applications Management, The Business Perspective, Security Management, and Software Asset Management.

Question 6 asks, ITIL has been around for more than 15 years. Why is it now taking hold in the United States?, and answers:

The United States has always lagged behind Europe in the discipline of IT infrastructure management. Go to a bookstore, and you'll find precious few books on the subject. Go to the universities, and you'll visit a lot before finding one that teaches it. One reason that this is changing is the increasing impact of European- and Asian-owned companies operating in the United States, all saying, Hey, get on board! You need to be ITIL-compliant. That's how we run our shops over here.

It also refers to something I have written about here. Whereas IT customers of the past were "internal staffers, managers, and auditors" ...

these days increasing numbers of IT customers ... are external -- actual customers of the business itself, interacting via public Web sites. If systems fail, potential buyers are likely to take their business elsewhere; potential damage to corporate reputation is high.

Question 7 deals with the needs of large and small IT shops.

Questions 8 and 9 address getting started and getting outside help with your ITIL implementation.

Question 10 talks about ITIL's relationship to ISO 20000. I will discuss the relationship between ITIL, COBIT, and some ISO standards in a future post.

Summing up
The article concludes that ...

Companies that have initiated ITIL efforts are already seeing higher customer-satisfaction levels and reduced costs and labor. Although not a panacea for all IT challenges, ITIL is a fundamental conceptual change for how IT will be doing business in the 21st century. Its time has come.

From all the evidence I see, this is an accurate summary. If ITIL's time has not yet come for your organization, maybe you should expect it to arrive soon. In future posts, I'll be writing a lot more about how ITIL applies to Web Performance Management. But if you just finding out about ITIL, I recommend this InfoWorld article as a good starting point.

Tags: ITIL, InfoWorld, ISO 20000, performance, Web performance management, service level management, SLM.

Web Performance Engineering [4]

2006-08-28T04:01:00.000+00:00

Continuing my series of posts on Web performance guidelines, today I'm reviewing one chapter of a new book, Deliver First Class Web Sites: 101 Essential Checklists, by Shirley Kaiser of SKDesigns, published by Sitepoint in July 2006.

Sitepoint is known among Web developers for its practical publications, and Kaiser deserves credit for including a full chapter on site performance alongside all the other useful advice in this book.

As its title states, this book does not cover topics in great depth. Each checklist item is stated, then discussed briefly. But Kaiser does include enough detail to justify each recommendation, and often includes examples of how to do it. Chapter 11 (Web Site Optimization) is 19 pages long, presenting 41 recommendations arranged into six checklists:

Creating Clean, Lean Markup
Minimizing URLs
Optimizing CSS
Optimizing JavaScript
Supporting Speedy Server Responses
Optimizing Images, Multimedia, and Alternative Formats

However, once I started reviewing Kaiser's checklists, I soon noticed a remarkable similarity between her choice of topics and those covered in Andrew King's book, Speed Up Your Site -- the subtitle of which also happens to be Web Site Optimization.

A bit more scrutiny, summarized in this spreadsheet, revealed similarities in the naming, wording, and organization of the checklist items too. Although Kaiser cites Speed Up your Site in her first paragraph, she does not actually recommend it explicitly, or mention any collaboration with Andrew King. But she seems to have relied heavily on that one source as the main inspiration for her Web Site Optimization chapter.

To give Kaiser her due, rather than simply parroting all of King's recommendations, she has ignored the more extreme ones. All the same, her material still shares most of King's limitations and omissions, many of which I listed earlier when I reviewed his book, and which I plan to cover in future posts in this series.

In conclusion, if you'd like a handy summary of the material in King (except for his excellent discussion of the Psychology of Performance), Kaiser's book has it, plus about 300 more pages of useful checklists on other topics. I recommend it, because I don't think you can go far wrong with any book from Sitepoint. If you want to read more about the same topics, you can find Speed Up your Site on sale on Amazon these days under half price. But don't assume that either book will give you a well-rounded picture of Web Site Optimization issues and techniques.

For more ideas about that, continue reading Performance Matters, and I'll do my best to fill in the holes.

Tags: Shirley Kaiser, SKDesigns, Sitepoint, performance, Web site optimization, Checklist, page design, site design, service level, SLM, optimization, Web optimization, tuning, Web tuning.

Web Performance Engineering [3]

2006-08-22T07:01:00.000+00:00

Continuing my series on Web performance guidelines, today I am reviewing another book -- Speed Up Your Site, by Andrew B. King, published by New Riders in 2003.

A while back, when I was reviewing Web Usability Books, I promised to cover Speed Up Your Site, but never got around to doing so -- for reasons I will explain. A full table of contents listing all 19 chapters is available online; in summary, the book has six parts:

Part I - The Psychology of Performance (38 pages)
Part II - Optimizing Markup: HTML and XHTML (99 pages)
Part III - DHTML Optimization: CSS and JavaScript (111 pages)
Part IV - Graphics and Multimedia Optimization (85 pages)
Part V - Search Engine Optimization (39 pages)
Part VI - Advanced Optimization Techniques (79 pages)

Part of my difficulty in reviewing this book was that I have mixed feelings about it. Naturally, I am always pleased to see an entire book devoted to performance issues. Also, Part I is particularly good. Containing 77 references, it is a very well-researched survey of an important subject rarely covered in books about Usability. On the other hand, I am not nearly as impressed with the rest of its advice and guidelines, for two reasons: their correctness, and their completeness.

First correctness. Regarding the actual content, my issue is not that Speed Up Your Site contains factual errors. As far as I can tell, its content is accurate. But some of its recommendations -- although they may indeed improve performance -- are so unnatural that they are not really the correct way to tackle the problem. This concern is summarized by Alexander Bunkenburg in this review on Amazon.com:

This book concentrates almost exclusively on sending fewer bytes from the server to the browser. It gives a large collection of tricks how to write shorter html, xhtml, css, and JavaScript. Some of these tricks are useful. Others however go against standards, and some seriously go against maintainability. I'd be reluctant to give this book to my team. One may be tempted into shaving off bytes, spending a big effort and yet producing unmaintainable code.

The problems that can result from deliberately violating standards are highlighted by David Rose in another Amazon review:

In today's world, where "standards based" coding is becoming more prevalent and adherence to the W3C standards for HTML coding is being recommended, this book just grated on me. While there is a great deal of great information, there are also a large number of "gotchas" to watch out for as well.

The book proposes to use HTML tags without their corresponding closing tags, not to use required elements whenever possible, avoid using quotes in HTML tags, and many other ways of creating "non-valid" code. This will "optimize" your code a bit more by reducing the characters in it, but it will also create problems for you in the future.

In summary, while the book does give a lot of good information, it often steers you away from standard code. If you are unsure what is considered "standard" and required for creating valid XHTML/CSS, you are best served skipping this book as it will teach you to create invalid code.

Bunkenburg's review also touches on my second area of concern -- completeness. Any book about speeding up Web site performance should (in my view) at least mention all the important topics, to let the reader know what options exist. Some important topics that receive little or no coverage in Speed Up Your Site are:

Page Types: All pages are not created equal, and users' tolerance for delays changes depending on where they are in their interaction with a site. So a single page design approach cannot be applied to all pages.
Maximize content reuse: A common mistake as sites grow is to use many different names (URLs) for the same thing, reducing the efficiency of browser caching.
Ratio of HTML base to content: There is a significant difference between the way browsers handle the base (or index) portion of the page, and referenced content elements, which can affect page download time.
Image resizing: For efficient page rendering, it helps to specify HTML HEIGHT and WIDTH tags for all embedded images. And for download speed, avoid resizing images in the browser.
Performance of SSL: All online business sites use encryption, and pages that use SSL encryption incur significant overheads. This is an issue that demands a separate section in any book about site performance.
Content Delivery Networks (CDNs): Akamai went public in 1999, and was widely used by 2002. Mirror Image, Cable and Wireless (formerly Digital Island), and Speedera also offered CDN services in 2002. How can any book about speeding up your site not even mention this technology?

In fairness to King, any discussion of the book's coverage should point out the following disclaimer, which appears in its Introduction:

Although the primary emphasis is on optimizing client-side technologies, this book also covers server-side techniques and compression to squeeze the maximum performance out of your site. These are all techniques that most designers and authors can control. Instead of focusing on esoteric server-side tuning limited to system administrators, this book focuses on optimizing the content that you deliver. For a server-oriented look at performance, see Web Performance Tuning, by Patrick Killelea.

But most of the omissions I listed are not related to server-side tuning, and of the two that are (SSL and CDNs), only SSL is discussed by Patrick Killelea, and his only recommendation is to use an SSL accelerator card. And in 2002, CDN technology could hardly be considered "esoteric", especially in a book which, to quote its author, is "not for beginners".

There is a lot more to be said on most the above topics, and in future posts I will expand upon them.

Tags: Andrew King, Speed up your site, performance, Web site optimization, page design, site design, service level, SLM, optimization, Web optimization, tuning, Web tuning.

Web Performance Engineering [2]

2006-08-18T07:01:00.000+00:00

Today I'm going to look at another list of Top Ten Web Performance Tuning Tips, following up on my promise to review Web site and application performance advice.

Today's list of tuning tips was created by Patrick Killelea, the author of Web Performance Tuning, first published by O'Reilly in 1998, then revised in 2002. When the second edition came out, Patrick also updated his 1998 top ten list, presumably to reflect changes in the rapidly maturing Internet and Web environment. But O'Reilly still publishes the 1998 list alongside the 2002 list without any further explanation, even though just four recommendations appear on both lists!

I see this as evidence that publishers are a lot more interested in selling a book than they are in the usefulness of its content. So let's blame O'Reilly and give Patrick the benefit of the doubt here, and focus on his latest list only. Abbreviating his recommendations, they are:

1. Check for compliance with standards
2. Minimize use of JavaScript and style sheets
3. Turn off the Web server's reverse DNS lookups
4. Try out a free analysis tool (to find bottlenecks)
5. Use simple servlets or CGI
6. Get more memory
7. Index your database tables well
8. Make fewer database queries
9. Look for packet loss and retransmission
10. Monitor your Web site's performance

When someone publishes a top ten list, I expect it to include the ten most important and useful recommendations -- especially when its author has written the most comprehensive book available on the subject. In this case, even allowing for the maturing of the Web since 2002, I have no idea how Patrick could have come up with this list. I have three problems with it -- what's in it, what's not in it, and its order. Today I will tackle mainly the first area; here are some brief thoughts about each of his recommendations:

Check for standards compliance by using Weblint or other HTML checking tools.
Content that conforms to the HTML 4.0 standard will load faster and work in every browser because the browser then knows what to expect. Note that Microsoft-based tools create content that does not even use the standard ASCII character set, but instead uses many proprietary Microsoft characters that will display in Netscape as question marks and can slow down rendering.

performance

Watch out for Composition Tools with a Bias

Minimize the use of JavaScript and style sheets.
JavaScript is a major source of incompatibility, browser hangs, and pop-up advertising. Style sheets require separate downloads before the page can be displayed. There are some nice features to both JavaScript and style sheets, but at a big cost. Life is better without them.

Turn off reverse DNS lookups in the Web server.
If left on, reverse DNS will log a client's machine name rather than IP address, but at a large performance cost. It is better left off. You can always run log analysis tools which look up the names later.

in 1997

Prior to Apache 1.3, HostnameLookups defaulted to On. This adds latency to every request because it requires a DNS lookup to finish before the request is completed. In Apache 1.3, this setting defaults to Off.

Try out a free analysis tool.
I've provided a free analysis tool at my Web site that can tell you whether or not your bottleneck is in DNS, or because of connection time or content size, or is on the server side. Work on improving the slowest part first.

a page about his book

Use simple servlets or CGI.
Use simple servlets, CGI, or your Web server's API rather than any distributed object schemes like CORBA or EJB. Distributed object schemes are intended to improve a programmer's code-writing productivity, but they do so at an unacceptable cost in performance for end-users.

Get more memory.
Your Web server, middleware, and database all will probably do better with more memory, if they still use their hard disks frequently. Hard disks are literally about a million times slower than memory, so you should buy more memory until the disks are phased out.

Index your database tables well.
Spectacular improvements are possible if you are inadvertently doing full-table scans on every hit of a particular URL. Indexes allow you to go directly to the data you need.

spectacular improvements

Make fewer database queries.
If you can cache content in your middleware or servlets, do it. Making connections to a database and using those database connections is typically a bottleneck for performance.

doing less work

any

Look for packet loss and retransmission.
There are many network snooping and monitoring tools to help you do this. Intermittent slowness is often due to packets being lost or corrupted. This is because a time-out period needs to pass before the packet is retransmitted.

Set up monitoring and automated graphing of your Web site's performance.
This information is free online in Chapter 4 of the second edition of Web Performance Tuning.

you can't manage what you don't measure

researching it

So, to sum up my audit of Patrick's list of ten guidelines, I vote to reject two altogether (#1 and #2), downgrade one (#3) to a priority well outside my top ten, accept four (#4, #5, #6, and #7), restate two (#8 and #10) as general principles that don't belong on this list, and reserve judgment on one (#9).

That opens up 5 or 6 slots for the things that Patrick missed -- but what should they be? I will tackle that subject in a follow-up post.

Tags: Killelea, Patrick Killelea, O'Reilly, performance, Web performance, page design, site design, service level, SLM, optimization, Web optimization, tuning, Web tuning.

Baseball and the Price of Gas

2006-08-17T05:35:00.000+00:00

This is only tangentially related to the usual subjects I cover in this blog, but it certainly relates to the way I approach research and blogging. I am always doing research online, and during summer evenings and weekends that activity is often accompanied by the day's radio broadcast of the Oakland A's baseball game -- the best baseball team here in the San Francisco Bay Area, by any objective standard.

Tonight was no different, and A's broadcaster Robert Buan caught my attention when he opened the post-game show. He pointed out that in winning tonight, the A's have secured their biggest lead in their division since September 30, 1992. As a fan of the team, this is interesting; to everyone else it's probably instantly forgettable. But more intriguing was what he actually said -- in the very first sentence of the program. He opened his show with this statement:

According to a reliable authority, wikipedia.com, the last time the A's had a lead of six and a half games in the American League West, gas was selling for $1.38.

I couldn't help reflecting on how much the Web is changing the way everyone approaches information and research!

Reporting Web Application Responsiveness

2006-08-15T17:45:00.000+00:00

In a previous post, I discussed some complications of measuring Rich Internet Applications (RIAs). In particular, I concluded that …

… to report useful measurements of the user experience of response times, instead of relying on the definition of physical Web pages to drive the subdivision of application response times, we must break the application into what we might call logical pages. To do this, a measurement tool must recognize meaningful application milestones or markers that signal logical boundaries of interest for reporting, and thus subdivide the application so that we can identify and report response times by logical page.

Today I am going to look inside the logical page, and consider what happens when the application responds to a user action. I have previously written about the stages of this process here, and in this paper.

Two Standard Service Level Metrics
Because Web pages are constructed using many separately downloaded components, low level monitoring tools can collect a ton of data about the communications activity triggered by a user interface action. And for diagnostic and tuning purposes, it is always useful to measure each separate component of the download process. But for most service level monitoring, this low level data can usually be summarized in just two important metrics: initial response time and page download time.

From the perspective of a typical user, these metrics represent two distinct events in the page download experience. After 'clicking' on a page, the first measures the apparent pause until the site begins to respond, while the second measures the time for that response to download.

The elapsed time between these two events can be large enough to affect the way pages are designed. Because pages containing many images take a while to download over slower connections, such pages are normally laid out so that the most popular links are displayed early in the page download process, allowing a user to navigate to another part of the site without having to wait for the all the page content to complete.

But while initial response time and page download time provide a good indication of the user’s experience of a traditional Web application, they may be less useful for some RIAs. In these applications, the time to complete a page download might include additional asynchronous activity that does not affect the user’s experience of the current page, such as fetching additional content (like executable script files, application data, images, or even streams) for future use within the application.

An Additional Metric for RIAs
This suggests that, while the two existing metrics are still useful, a standard scheme for measuring and reporting RIA responsiveness probably needs to also include a metric that represents the time from user interface action until an intermediate event, one that we might call page display complete. This metric would reflect the time it takes to download everything needed to complete the display of the current page.

Even that definition is ambiguous, because it may not be obvious when some pages are 'completely displayed'. I see two issues here. First, what about parts of the page outside the display window, that won’t be visible until the user scrolls? Let’s deal with that by assuming the user has a screen of unlimited size, so that the entire page could be visible without scrolling.

Second, what if a page includes a streamed video, or a series of images that are displayed in sequence, each replacing the previous one? On reflection, I conclude that the 'page display complete' event should occur when the complete display is first visible -- in my examples, when the video begins playing, or the first image is displayed. That's because measuring to the last view of the display would make this metric a lot less useful as a measure of a user's experience of responsiveness. And nowhere between those two extremes offers any possibility of generality across applications.

Summarizing What’s Needed
So now I have defined three distinct metrics that seem to have fairly universal applicability when measuring the responsiveness of RIAs:

Initial response time
The elapsed time from a user interface action (a mouse click, or any other user action that triggers a download process) until the browser places the page in a state that permits a user to perform another action (technically, the user is unblocked). We could also say that this is the time until the new page first becomes usable.

Page display time
The elapsed time from a user interface action until the complete resulting page is first displayed, or could be first displayed if the screen were large enough.

Page download time
The elapsed time from a user interface action until the complete resulting page and all associated content elements have been downloaded. This time includes all secondary or background downloads triggered by the user's action, whether or not they are required for the current page display.

These definitions provide only a rough statement of requirements for RIA measurement -- none is technically precise. In practice, any attempt to implement any one of these requirements could present problems for some applications, browsers, or measurement tools, and I will write about some of those complications in a future post. But I think it is useful to identify these generally applicable goals for measuring and reporting on the responsiveness of Rich Internet Applications.

Tags: RIA, Rich Internet Application, measurement, service level, SLM, page download, Web application, Webapp.

Web Performance Engineering [1]

2006-08-15T00:12:00.000+00:00

This post is the first in a new series on how to build Web pages, sites and applications that perform well -- by design. I will be combining my own observations with online research and recommendations on best practices contributed by my colleague Ben Rushlo. Ben makes his living measuring Web site performance and giving companies advice on how to improve the performance of their sites and applications.

Despite the crucial contribution of performance to online application effectiveness, good advice on the subject is surprisingly scarce on the Web, and even scarcer in books about Web design. Although you do sometimes find small sections (rarely chapters) of such books devoted to performance, their content is usually weak and almost never a systematic treatment of the issues. I won't bore you with proof, but I do mention one example below.

On the other hand, the exceptions are certainly worth noting. Recently I found an excellent discussion of 10 Realistic Steps to a Faster Web Site, posted in February 2006 by Alexander Kirk, a Web application programmer in Vienna, Austria. I mention his profession because in my experience, a lot of advice about performance written by programmers completely ignores the central rule of all performance optimization -- to speed anything up, remove the biggest bottleneck.

Applying just this one rule repeatedly (think about it) will always produce the best results. And in the world of distributed systems, the biggest bottlenecks will almost always be related to the way an application uses a network, not to the way a computer (client or server) processes some program code. In other words, the application's logic is usually far more important than the way a programmer coded it.

Kirk obviously understands this. After complaining about an example of misguided performance advice (read the comments), he presents his view of a systematic approach to Web application performance analysis. I will not attempt to summarize it beyond simply listing the outline, which is:

1. Determine the bottleneck
1.1. File Size
1.2. Latency
2. Reducing the file size
3. Check what’s causing a high latency
3.1. Is it the network latency?
3.2. Does it take too long to generate the page?
3.3. Is it the rendering performance?
4. Determine the lagging component(s)
5. Enable a Compiler Cache
6. Look at the DB Queries
7. Send the correct Modification Data
8. Consider Component Caching (advanced)
9. Reducing the Server Load
9.1. Use a Reverse Proxy (needs access to the server)
9.2. Take a lightweight HTTP Server (needs access to the server)
10. Server Scaling (extreme technique)

Kirk's discussion does omit a few topics (subjects for future posts in this series) that I think are important, but this is an excellent starting point. While his wording implies a reactive approach to his subject matter (writing about tuning or improving the performance an existing site), most of his guidelines relate to best practices in site design and engineering. So there is no reason why they should not be implemented proactively, without waiting for problems to surface.

Tags: Web performance, page design, site design, service level, SLM, optimization, Web optimization.

Rich Internet Applications: an update

2006-07-23T06:23:00.000+00:00

Blogging has taken a backseat to other activities for a couple of months, so I have some Performance Matters to report.

Following publication of the seventh in my earlier series of posts about managing Rich Internet Applications (RIAs), I reworked most of that material into a single paper. After some delays to reformat the content and redraw the diagrams, it is now available an an Industry Brief, which you can download from the Keynote resource library. The title is Rich Internet Applications: Design, Measurement, and Management Challenges.

Although I referenced Wikipedia when introducing the subject of RIAs, and retained those references in the paper, I did not feel that the current contents of Wikipedia did a very good job of explaining RIAs. In my second post I described how I had met Aleksandar Šušnjar online after reading his contributions to the discussion of RIAs in Wikipedia. I wrote that Aleks had:

... shared insights gained from his experience building a product first released in 2002 ... In delivering desktop capabilities via the Web, it clearly predated much of today's thinking about the purpose of RIAs and how to build them.

Naturally, Aleks also discovered some potential pitfalls an RIA designer might run into, which I will be mentioning in later posts. Regrettably, many of his insightful contributions to Wikipedia (which can still be found in its history pages) were subsequently removed by other contributors who lacked either his intimate knowledge of the technology or his historical perspective. Rather than waging Wikipedia edit wars, which for a hot topic can continue interminably, he has simply posted his own page about RIA and AJAX.

Despite Aleks' experiences, having made the effort to understand and summarize the SLM challenges posed by RIAs as coherently as I could, I felt it was time to try my hand at editing Wikipedia. So I have tackled the Wikipedia RIA page; I suspect that this entry may not be quite as "hot" as the Ajax entry, where Aleks was posting. So today (at least) you can find my handiwork throughout the section that compares RIAs with traditional Web applications, including (not surprisingly) the entire section on management complications.

Tomorrow may be another story. After all, the Wikipedia editor carries the clear warning: If you don't want your writing to be edited mercilessly or redistributed by others, do not submit it. So please feel free to improve on my efforts -- I know what to expect!

Tags: RIA, Rich Internet Application, management, wikipedia, service level, SLM, Web application, Webapp.

Insights from Interop 2006

2006-05-02T19:32:00.000+00:00

In my last post I promised that if I get any new insights worth sharing this week while I'm at Interop 2006, I'll write about them. Well, here's the first installment, which consistes of just three items. I'm supposed to be at the Keynote booth on the show floor soon, so this will be short, because it's quite a hike to get there.

I'm staying at the Luxor, a hotel cleverly designed to resemble an Egyptian pyramid. Egyptian motifs abound. To reach the Conference Center in the Mandalay Bay hotel, which is next door, you can take a cab, a shuttle bus, or ride on a tram. But I usually walk. I hike about half a mile through a network of corridors cleverly disguised as places to spend money -- two casinos, a shopping mall, theaters and entertainment centers, restaurants, etc. I know I need to exercise more, so this a token effort.

Back to the insights. Yesterday afternoon I attended the Application Performance Day. Peter Sevcik of NetForecast presented a very thorough analysis of Performance Improvement Solutions, covering QoS, Compression, CDN's and other Acceleration technologies. He also provided a sheet of references, some of which I will publish when I have more time. Two insightful comments I wrote down were:

Fast delivery of sites results in slow delivery of pages. We all know that when the development process does not include time to think about performance, the results are not optimal. I thought Peter's simple rule captured this issue in a memorable way.
Most developers build applications in a LAN-based environment, and don't insert a WAN simulator to test their performance. Another well-known problem nicely summarized. If you don't actually test your application in the production environment, or something that resembles it, you will surely run into performance problems in the real world.

Which brings me to my third item. Last night I dreamt I was one of a small group of people lugging a grand piano from the Luxor to the conference center. But instead of taking the inside route, we were outside on the street. And every time we came to a gap in the sidewalk, several of us had to lift the piano over the kerbs. This was becoming hard work, and so at one point Steve Jobs, who owned the piano, wanted to leave it where it was. But John Chambers (CEO of Cisco Systems), who seemed to be supervising the move, said "No Steve -- we can't just leave it here, we have to deliver it."

Now I'm not sure what lessons (if any) I should draw from this dream. It certainly seems to be a metaphor for many of the discussions going on here about how to deliver large payloads to remote destinations via less than ideal routes! Maybe John Chambers actually offered some more concrete suggestions during his Interop keynote speech this morning; I was still in my hotel room moving the piano at that time.

Now I'm going to mingle with the crowds and find out if Interop can make me smarter today.

Alistair Croll on Ajax

2006-05-02T09:24:00.000+00:00

Because of my particular interest in software performance optimization and SLM, I often search the Web to see what people are writing about performance. And even though there are always new things to find, it seems to be getting harder to locate them.

One reason is that until recently, anyone writing about the performance of computer systems or applications was discussing SLM topics like speed, availability, optimization, capacity planning, or load testing. But in the last few years, as the Web has made information systems and business systems synonymous, the term performance has gradually taken on a much broader meaning. Now to locate the interesting links, I have to include technical keywords that are designed to filter out unwanted information about business success.

So when I do find something worth reading, I assume it's a good idea to make a note of it, to save someone else the trouble. I have already included the most useful references in my previous posts about the performance of Rich Internet Applications (RIAs), but yesterday I found this excellent blog post by Alistair Croll of Coradiant on the impact of AJAX on web operations. From there, I also located his related presentation on Ajax and Networks to Interop in December 2005.

Finding this material was a complete coincidence, in that I happen to be attending Interop 2006 in Las Vegas this week, yet I found that link independently. But it was a good very thing to know about, because on Wednesday I am speaking at the WebOps Summit (scroll down to Wednesday for details), which is chaired by none other than ... Alistair Croll. And I have actually devoted the last third of my talk on Best Practices to the emerging RIA measurement issues I discussed here. Even though I have been researching and writing about the performance implications of Ajax and RIA's in general, and scheduling this Interop presentation with Alistair, I had still missed this connection.

This illustrates just how hard it is these days to stay on top of any topic being discussed online. So I hope this post will help you in that regard. And if I make any new connections worth sharing this week, I'll be writing about them when I get a chance. The cover of the conference guides say Interop Makes You Smart, so I'm hoping that works on me. Those marketing slogans are always true, aren't they?

Software Engineering Matters

2006-04-16T22:13:00.000+00:00

With the obvious exception of a few bloggers like Markos Moulitsas Zúniga, whose blog Daily Kos receives over a million visits per day, I think most bloggers are happy just to know that someone cares enough to read what they write. In my own case, having spent my career working on software performance, I learned long ago that only a small fraction of the population is actually interested in Performance Matters. So I don't expect a lot of feedback.

All the same it's nice to hear from a reader occasionally, and yesterday was one of those days -- I received a comment from Damith C. Rajapakse.

Curious about why a post on Waterfall Methods would be the one to generate a response, I did some digging. For readers who would like to hear about interesting stuff for the serious software engineer, Damith's blog -- Another Day in Mythical Man Month -- is worth a look. In particular, his recent post on maintaining future web applications (subtitled 'it's time to brace ourselves') is very relevant to my current focus on Rich Internet Applications. Damith notes that while ...

rapid development of Web apps is receiving the lion’s share of attention, maintainability of Web apps (is) hardly receiving a thought.

He warns about three trends that are likely to make Web applications hard to maintain:

The rush towards visual programming
Overuse of frameworks
Overdoing that 'AJAX ' thing

While it's good just to know that someone is reading what I write, it's even better when they have something useful to contribute to the discussion. And in this case I agree completely with Damith's perspective on the issues. Ten years ago, on the very first page of the introduction to my book about client/server performance, I wrote:

In today's world of visual programming, rapid development, and out-of-the-box solutions ... performance is one area of software design that is frequently misunderstood, forgotten, ignored, or postponed, often with disastrous results.

The same can be said of maintainability -- which, like performance, is always less interesting to software developers than creating new application function. That is why your software development process must ensure that characteristics like performance and maintainability are designed and engineered into applications from the beginning. In other words, it is important to realize that good Software Engineering involves a lot more than just programming and testing your code. According to Wikipedia, it is ...

... the practice of creating and maintaining software applications by applying technologies and practices from engineering, computer science, project management, application domains and other fields... Like traditional engineering disciplines, it deals with issues of cost and reliability ...

As I have noted previously, the complexity of Rich Internet Applications makes it essential to actually practice rigorous software engineering, and not just crank out code.

Waterfall Methods: Past and Ever-Present

2006-04-16T05:17:00.000+00:00

In my previous post, I commented that adopting a waterfall development methodology was not going to work well when developing Rich Internet Applications, because more agile methods were needed. While writing the post, I did a bit of research into the literature about Waterfall methods, a concept I clearly recall encountering for the first time in 1978 or '79 soon after I joined IBM's DB2 development team in the Santa Teresa Lab.

My own introduction to the Waterfall method was in a talk about its limitations. So it was interesting to read Tarmo’s psycho and techno blog explaining that the very first description of the model by Winston W. Royce in 1970 was also as a counterexample. This point of view is amplified by Conrad Weisert, who claims that in fact there's no such thing as a Waterfall Model!

But despite the fact that (maybe) it should not exist, the Waterfall model has taken root. This is evident from the fact that it merits a place in methodology comparisons and analyses. What's more, despite the many rumors of its demise, especially in the face of today's need for more agile methods, researchers have discovered that Waterfall methods refuse to die:

... in a survey of almost 200 practitioners, accounting for several thousands of projects over the past five years, the dominant process model reported was the Waterfall, with more than a third claiming its use.

Indeed, perhaps the ultimate tribute to the sheer persistence of the Waterfall approach in everyday development practice is the fact that the agile development community recently promoted the Waterfall 2006 Conference. Even if you couldn't attend, I recommend reviewing the agenda and the session descriptions carefully, just to see what you missed. I don't know if or when more documentation will be published, but I'm sure it will be worth waiting for.

Managing Rich Internet Applications [7]

2006-04-10T07:11:00.000+00:00

This is the seventh post in a series devoted to the challenges of Service Level Management (SLM) for Rich Internet Applications (RIAs). In these applications, some processing is transferred to the Web client while some remains on the application server. Previous posts introduced the subject and the SLM topics I plan to address, reviewed RIA technologies, introduced The RIA Behavior Model, and introduced and discussed RIA measurement challenges.

RIA Usability and the Site Development Process

Having discussed the measurement challenges posed by RIAs, I intend next to consider the demands that RIA development may make of SLM processes that have been designed and fine-tuned to manage traditional Web applications. I covered some of this ground in a Webinar on How to Overcome the Challenges of Rich Internet Applications. So in this post I will lay out the general framework of that material, using the illustrative figures I created for the Webinar.

In previous posts I have introduced and discussed a simple four-part framework for thinking about Web Site Usability. I divided usability into four dimensions, each of which is essential to the ultimate success of any site -- Availability, Responsiveness, Clarity, and Utility.

The graphic presents the four essential qualities in the order of their significance, at least for all first-time visitors to a site.

Interestingly, companies developing Web sites or eBusiness applications tend to approach these four aspects in exactly the reverse order, as shown in the next figure.

A business goal leads to a design process, which in turn results in an application being built and then rolled out into production. And at each point along the way, people do their best to make the outcome a success. But is that enough?

Well, if you take this approach you may get lucky. But to be safe you really have to take a more systematic approach to Service Level Management, like ITSO.

[On this subject, it is hard for me to be brief, because 10 years ago I actually wrote a 750-page book about it. But here goes!] I have always believed that you cannot separate issues of software performance from the software development life cycle (SDLC). The best way to ensure acceptable performance is to follow the discipline of Software Perfrmance Engineering -- to build in the desired level of performance, just as a mechanical engineer would. The next figure attempts to sum up this idea.

I think everyone can agree that there is an application development life cycle – applications have to be designed, developed, tested, and deployed. People may argue about the details, but for the purposes of this discussion we can set aside those religious debates about methodologies. All I care about here is that each stage of your development process -– whatever it is -- must include a deliberate focus on performance (or SLM), alongside other aspects of site usability.

If you want an application to achieve certain levels of service, don’t expect that to just happen on its own. You have to agree on your objectives, then design, develop, and test your application with those objectives firmly in mind, and measure and manage the application to make sure you are actually achieving your objectives. Of course, there’s nothing particularly radical or new about this idea, but the added complexity of RIA’s increases the risk of failure if we don’t actually follow these well-established best practices.

The RIA Development Process
If you want to implement a really usable Rich Internet Application, you’re probably going to need to beef up your site development processes to get everyone on the same page. That's because up to now the Web Usability (with a capital 'U') profession has concentrated on the dimension I call Clarity.

Their exclusive focus on Web page design and Information Architecture was possible only because the traditional structure of a Web applications was so constrained by the limitations of the Web browser technology they had at their disposal. When all Web applications implemented the same simple thin-client model, they did not have to worry too much about application complexity.

But as we evolve to building RIAs with a separate client-side engine, the site development process must deal with the kinds of complex issues that arise during the design of distributed applications. In the past, Web designers have not had to deal with anything as technical as this. They certainly can’t buy a book on 'Web Usability' and read about these kinds of issues.

Confirming this perspective, I recently heard Scott Dietzen, President and CTO of Zimbra speaking about Ajax at an MIT/Stanford VLAB meeting. To illustrate his statements that 'Ajax is still hard' and 'You can't crank out a good Ajax application quickly' he revealed that Zimbra has had to interview 40 JavaScript developers for every one they have hired. This is telling evidence of the difference between traditional Web applications and RIAs.

And you really do have to work hard to get everyone on the same page, because typically the people in the various participating departments usually don’t have a clear picture of just how much it takes to build a successful Web application.

As illustrated here, they naturally focus first on their own problems and challenges. The graphic is meant to be a bit tongue in cheek, but it does reflect the political realities of application development.

Several groups contribute to the success of any site, and each group thinks that their contribution is the most important! Apart maybe from those who work in IT, who tend to complain that they could deliver much better service levels if only the developers would hand them more robust and stable applications to manage.

These attitudes are not just the result of people having an inflated sense of their own importance. Most people honestly don’t understand just how much work the other players have to do.

This is already true today, and with the added compexity introduced by Rich Internet Applications everyone will have even more to think about. This is likely to increase their isolation from the other participants, unless the development process requires close cooperation.

So the first lessons your development processes must drive home are that all participants play an equally vital role in delivering a usable application, because all four dimensions of usability are equally vital to the success of the application.

The Importance of Being Agile
Building successful RIAs will also require more communication. We’ve been talking about the importance of SLM within the life cycle. But the problem with describing the software life cycle using a list activities like the one in the third illustration above (introducing the development life cycle) is that such lists tend to imply a step-by-step process –- a waterfall methodology –- in which we must complete one activity before moving on to the next. That approach to the development process is certainly not going to be an effective way to develop Rich Internet Applications.

Previously I introduced the RIA Behavior Model shown here. Consider in particular the grey boxes along the top, representing the application's design and usage environment, or context. As I wrote at the time:

A user's satisfaction with any application depends on ... how well the application design matches their needs at the time, their way of thinking, and their behavior when they are using it.

Their experience of response time depends on the combined behaviors of the client and server components of the application, which in turn depend on the application design, the underlying server infrastructure design, and of course the user's Internet connection speed. The most effective RIA will be one whose creators took into account these factors at each stage of its development life cycle, and who created the necessary management processes to ensure its success when in production.

Now imagine trying to focus on any one of the four aspects of usability without paying attention to the others. It can't be done. Application utility is tied to user behavior, which is driven by the clarity of the site, the design of the client-side engine, and the responsiveness of interactions between the client engine and the server components. And the design and implementation of those components will ultimately also determine application availability.

Since the four dimensions of usability are so intertwined, I conclude that to ensure a successful outcome, we must adopt a development process that keeps all four usability goals in focus, all the way through the life cycle. This approach is called agile software development.

Agile methods use an iterative approach to design, development and testing that keeps all the different participants ('stakeholders') involved -- marketing or business owners, Web designers, application developers, and the IT staff who understand what it takes to manage a site in production. They all have to communicate and share their different areas of expertise throughout the process.

In the Web design literature, methods similar to this have sometimes been referred to as customer-centered design. For example, that terminology is used in The Design of Sites by Doug van Duyne et al (see Chapter 1). However, Customer-Centered Design: A New Approach To Web Usability by Kreta Chandler and Karen Hyatt, while interesting (see Chapter 1), does not have much to say about the development process.

My graphic here is based on Chapter 5 of 'The Design of Sites', entitled 'Processes for Developing Customer-Centered Sites'. That book, having been published in 2003, is not about RIAs. Nor does it mention agile development per se, which is hardly surprising considering that the agile manifesto was crafted during the time when van Duyne et al were writing their manuscript.

However, it does suggest the kind of iterative approach to the development process that I believe is essential when developing RIAs. Chapter 5 concludes:

The principles and techniques of customer-centered design and iterative prototyping are embedded in every stage. Many firms have similar processes, though the stages and deliverables might have slightly different names.

Another book I have recommended previously, Usability for the Web by Tom Brinck, Darren Gergle, and Scott D. Wood, is structured around the major stages of an iterative development process. In Chapter 1 they write:

At each stage we want to cycle between refining our design and evaluating our latest refinement, iterating until we've achieved a level of usability that we're satisfied with before continuing to the next stage. Evaluation at each stage allows us to incorporate user and client feedback loops to optimize the design.

At each evaluation, we determine whether our design is adequate for continuing on. We do this by establishing benchmarks, or target usability goals.

These ideas did not emerge for the first time with the advent of the Rich Internet Application -- they are merely descriptions of good development practices. But the added complexity of the RIA model makes it essential that we actually put these kinds of methods into practice.

Managing Rich Internet Applications [6]

2006-03-25T19:14:00.000+00:00

This is the sixth post in a series devoted to the challenges of Service Level Management (SLM) for Rich Internet Applications (RIAs). In these applications, some processing is transferred to the Web client while some remains on the application server. Previous posts introduced the subject and the SLM topics I plan to address, reviewed RIA technologies, introduced The RIA Behavior Model, and introduced the application measurement topic.

Measuring RIA Responsiveness: Complications

To explain the challenges of measuring Rich Internet Applications, I will begin by reviewing three significant complications introduced by RIAs. From this discussion, I will draw conclusions about how measuring RIAs will differ from measuring traditional Web applications.

First Complication: Variety of Possible Behaviors
To make meaningful statements about an application's performance, you must first define what you need to measure. Typically you must measure common application usage patterns (or 'scenarios'), or patterns that are important by reason of their business value. When an application uses only a standard browser, its behavior is simple, known, and predictable, limiting the number of interesting usage scenarios to be measured.

Adding a client-side engine separates user interface actions from server requests and responses, giving application designers many more options. It allows developers to build an application that uses creative, clever ways to transfer display elements and portions of the processing to the client. The custom application behaviors encoded in the client-side engine make the result more complex and its usage less predictable than a traditional browser-based application. This increases the number of possible usage patterns, complicating the task of determining the important scenarios and measuring their performance.

Having many more design and implementation options also creates new opportunities for developers to make performance-related mistakes. They can (accidentally or deliberately) implement "chatty" client/server communication styles, which under some workload conditions may perform poorly. Even with thorough testing, some of these problems may remain undiscovered until after the application is deployed. A systematic SLM process must include measurement capabilities that provide a way to identify, investigate, and fix these types of problems.

Second Complication: Concurrent Activity
There are two reasons why it takes a while to download a Web page comprised of 50 separate content elements, no matter how fast your connection. First HTTP limits the rate at which clients can request objects from servers, then TCP limits the rate at which the data packets can deliver those objects from server to client. In particular, although the HTTP 1.1 standard allows clients to establish persistent connections with servers, RFC2616 defining the standard restricts the number of parallel connections the client (usually a browser) can use:

[Section 8.1.4] Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy. ... These guidelines are intended to improve HTTP response times and avoid congestion.

Avoiding bottlenecks is a desirable goal in all computer systems, and so the Internet protocols are designed to protect shared network and server resources, no matter how selfishly their users might behave. But flow controls may not always be needed. Consider the metering lights that permit only two cars onto the highway every 30 seconds. During the rush hour they can smooth the flow of traffic and reduce freeway congestion, but if left on at other times, they would delay drivers for no reason. Similarly, when there is plenty of available bandwidth and no server congestion, the limit of two connections per domain is just a governor that restricts the potential rate of data transfer from server to client.

Nonetheless, modern browsers (including IE) do adhere to this guideline. And it is safest to do so, because DoS logic in some proxy servers might reject connections that do not obey the RFC2616 standard. For a lively discussion of IE behavior and the meaning of 'SHOULD' in the RFC, see this msdn blog, which also points out a relatively simple technique any Web site designer can use to circumvent this limitation, if they feel it is important enough:

Savvy web developers can take this connection limit into account and deliver their data from multiple domains, since the browser will open up to two connections per domain.

Although this solution may be fast enough for many applications, most Ajax developers are looking for clever ways to make the client even more responsive. And because Ajax offers almost unlimited application design possibilities, today there is little agreement on the best ways to achieve that goal. This debate was well summarized by Jep Castelein in AJAX Latency problems: myth or reality?, a discussion that includes the important reminder that 'IE will have a maximum of 2 simultaneous connections (per domain, actually -- C.L.), whether you use XMLHttpRequest or not'. In other words, Ajax implementations are not above the law of HTTP.

Even so, a primary objective of many RIA designs is to work around the two-connection obstacle to improve responsiveness for the user. The resulting implementations will use many different techniques to communicate with one or more servers. As one data point, consider the opinion of Drew McClellan, who surely qualifies as an expert in developing Web applications. In his tutorial about JavaScript and the XMLHttpRequest object -- Very Dynamic Web Interfaces -- Drew concludes that the real challenge here is not figuring out how to make the code work but thinking of interesting ways in which it can be utilized.

Such freedom to improvise inevitably complicates measurement -- especially when requests originating from what appear to be separate units of work on a client are really all part of a single logical application. It is easy to sit on a server and measure the traffic, or the demands placed on various server-side resources. And this kind of measurement has its uses, especially when load testing or investigating bottlenecks. But the biggest challenge when measuring applications is correlating those seemingly separate pieces of information with a particular application activity, task, or phase. The more complex the client/server relationship, especially when it involves concurrent interactions, the trickier it becomes for measurement and analysis tools to perform that correlation properly.

Third Complication: Asynchronous and Synchronous Communications
An earlier post discussed RIA technologies. It described how Flash and Ajax use client-side engines that can be programmed to communicate asynchronously with server(s), independent of a user's actions. This notion is captured in name 'Ajax', in which the initial 'A' stands for 'Asynchronous', and Figure 2 in that post was taken from Jesse James Garrett's seminal article on Ajax. Although Garrett's figure does show how an engine enables asynchronous application behaviors, and differs from a traditional Web app, it does not illustrate the full range of possibilities, which I discussed further in RIA post #4. In general, a user action within a Rich Internet Application can trigger zero, one, or many server requests.

Also, most discussions of RIA architecture or technology focus on how asynchronous application behaviors can improve usability. But they don't question the fact that communication between browser and server is still synchronous. That is, communication is always initiated by the browser (or the client-side engine operating as an extension of the browser), and follows the standard synchronous HTTP request and response protocol. But in a recent blog post and presentation, Alex Russell of dojo proposed the name Comet for a collection of clever techniques that exploit HTTP persistent connections to implement a 'push' model of communication.

He used an adaptation of Garrett's original Ajax figure to show how Comet extends the Ajax model; this smaller version comes from brainblog:

Using the Comet techniques, a server uses long-lived persistent connections to send updates to many clients, without even receiving a request (a 'poll') from the client. According to Russell:

As is illustrated above, Comet applications can deliver data to the client at any time, not only in response to user input. The data is delivered over a single, previously-opened connection. This approach reduces the latency for data delivery significantly.

The architecture relies on a view of data which is event driven on both sides of the HTTP connection. Engineers familiar with SOA or message oriented middleware will find this diagram to be amazingly familiar. The only substantive change is that the endpoint is the browser.

While Comet is similar to Ajax in that it's asynchronous, applications that implement the Comet style can communicate state changes with almost negligible latency. This makes it suitable for many types of monitoring and multi-user collaboration applications which would otherwise be difficult or impossible to handle in a browser without plugins.

Like Ajax, Comet is not a new technology, but a new name for some clever ways to implement an old communication style using standard Web protocols. But as happened with Ajax in 2005, the new name has triggered a lot of interest among developers -- for evidence, just search on 'Ajax', 'Comet', and 'Web' (the last term should eliminate most links to Greek legends, soccer legends, and legendary cleaning products). Especially useful are Russell's slides from his talk (Comet: Low Latency Data For Browsers) at the recent O'Reilly ETech Conference, and Phil Windley's excellent write-up of the talk.

The complexity of a push architecture is justified only for applications that manage highly volatile shared data. But when it is implemented, it creates an entirely new set of measurement challenges. If information pushed to the client does help the user to work faster, its benefits will be reflected in normal measurements of application responsiveness. But you can't use normal responsiveness metrics to evaluate a server-initiated activity that simply updates information a user is already working with.

New metrics must be devised. Depending on the application, we may need to measure the currency or staleness of information available to the user, or maybe the percentage of times a user's action is invalidated by a newly received context update from the server. This kind of "hiccup" metric is conceptually similar to the frequency of rebuffering incidents, one measure of streaming quality. Server capacity will also be a major issue requiring careful design and load testing. These are new SLM challenges facing anyone who decides to implement the Comet approach.

Measuring RIA Responsiveness: Conclusions

While discussing the three major complications introduced by RIAs, I have already noted some consequences: measurements may become harder to specify, more difficult to correlate, and may even require the invention of new metrics. But I have been saving the punch-line until the end. I will now discuss my three most important and far-reaching conclusions about measuring RIAs, each of which involves a significant change from the way Web applications are measured today. These changes deal with the issues of what, where, and how to measure.

RIAs: What to Measure?
The short answer? Not individual Web pages. First and foremost, because the asynchronous aspects of the RIA model undermine two fundamental assumptions about Web applications and what to measure:

We can no longer think of a Web application as comprising a series of Web pages.
We can no longer assume that the time it takes to complete a Web page download corresponds to something a user perceives as important.

In fact, when a client-side engine can be programmed to continually download new content, or a server-side engine can keep pushing new content to the browser over a connectionn that never closes, some page downloads may never complete.

Therefore to report useful measurements of the user experience of response times, instead of relying on the definition of physical Web pages to drive the subdivision of application response times, we must break the application into what we might call 'logical pages'. To do this, a measurement tool must recognize meaningful application milestones or markers that signal logical boundaries of interest for reporting, and thus subdivide the application so that we can identify and report response times by logical page.

Because (as I noted earlier) it is usually hard to correlate seemingly separate pieces of measurement data after the fact, I conclude that these milestones will have to be identified before measurement takes place. They could be key events or downloads that always occur naturally within the application. Or they could require proactive instrumentation, for example using little downloadable content elements that developers deliberately imbed at logical boundaries in the application's flow, to enable its subsequent measurement.

The former method places the burden on those setting up measurements to first identify application-specific milestones. The latter frees the measurement tool from the need to know anything about the application, but places the burden on application developers to instrument their code by inserting markers at key points.

Comparing Apples and Oranges?
Second, when setting out to measure a RIA, you must think carefully about the purpose of the measurement, especially if the RIA is replacing a traditional Web application, or being compared with one. You must not let the traditional Web approach to a business task determine what measurements are taken. Aleks Šušnjar (see post [2] in this series) provided the following insights:

We can't compare apples and oranges by measuring either an apple or an orange. We need to approach measurement as if we were comparing two applications built by different developers -- one a traditional Webapp and the other a client/server app with a completely different UI.

In this situation, we cannot measure only at the level of single Web pages or server interactions. Finding that it takes so many milliseconds to get a server response for request-type X is meaningless if that client/server interaction does not occur in both versions of the application.

In my experience, a typical example concerned how long it took to upload or download a document. But those metrics were sometimes irrelevant, depending on the application context. So to make really useful performance comparisons, we had to approach the problem at a higher level -- for example, 'how long does it take to move a document from folder A to folder B?' In a traditional Web app that task would likely require multiple clicks on separate pages, whereas with an RIA/Ajax implementation, we could do it with a single drag and drop.

So to make valid comparisons, we had to measure and compare the net effect of two rather different aspects of performance -- one concerning only the computer (how many operations of type X can a machine perform per hour), the other involving human performance (how many documents can an employee move per hour). But both affected the overall conclusion. Generalizing, I would say that:

The server that has to generate additional HTML for multiple responses in a traditional Web app will likely use many more processor cycles than the one supporting an RIA/Ajax implementation, where all the user interactions are handled on the client and the server just has to provide a service at the end.
If the designer takes care to avoid 'chatty' client/server communications, network utilization will probably also be significantly lower in the second case, further improving server performance.
Finally, if the client-side interface is well designed, a RIA should allow users to think and work faster.
In the final analysis, all these components of application performance must be weighed.

Aleks' insights come from his own experience developing a Rich Internet Application -- for more details see his Wikipedia page about RIA and AJAX.

RIAs: Where to Measure?
You might think that to measure a user's experience of responsiveness, you would have to use a tool that collects measurements from the user's workstation, or at least from a measurement computer that is programmed to generate synthetic actions that imitate the behavior of a typical user. Surprisingly, this is not actually the case for traditional Web applications.

While synthetic measurements require computers to mimic both a user's actions and their geographical location, software that collects passive measurements of real user actions can in fact reside either on the client machine or on a machine that is close to the server, usually just behind the firewall -- just so long as that machine can observe the flow of network traffic at the TCP and HTTP levels. Because of the synchronous and predictable nature of these protocols, a measurement tool that can read and interpret the flow of packets can actually deduce the user's experience of response time by tracking HTTP message traffic and the lower-level timings of TCP data packets and (crucially) TCP acknowledgements.

Such a tool is called a packet sniffer, or protocol analyzer. Packet sniffing has a bad name in some quarters, being associated with malicious snooping by hackers. But in the right hands, it is a legitimate analysis technique used by some Web measurement tools to deduce client-side performance without actually installing any components, hardware or software, anywhere near the users.

Unfortunately for these tools, the growth of RIAs will make it progressively more difficult to exploit this clever approach. The RIA Behavior Reference Model (this figure) I introduced previously makes it clear that RIAs -- even without the further complications introduced by a Comet push architecture severely limit the power of the packet sniffing approach. That's because we can no longer characterize response time as the time to complete the synchronous round trip of:

Click(C) => Browser(B) => Request(Q) => Server(S) => Response(R) => Display(D)

Instead the client-side engine in the RIA architecture breaks apart this cycle into two separate cycles operating asynchronously:

The user-client cycle: Click(C) => Engine(E) => Display(D) -- [CED, for short]
The client-server cycle: Request(Q) => Server(S) => Response(R) -- [QSR, for short]

Another way of describing these cycles might be as 'foreground' (CED) and 'background' (QSR). Both cycles are important, because neither stands alone; it is their relationship that defines application behavior. But that relationship depends only on the application design, which cannot (in general) be inferred by a measurement tool, especially one that can observe only one of the two cycles.

I conclude therefore that to measure RIAs, tools will have to reside on the client machine, where they can see both the level of responsiveness experienced by the browser (the QSR cycle) and the user's experience of responsiveness (the CED cycle). Granted, QSR cycles can still be monitored by the traditional packet sniffing methods, but tracking them will permit only limited inferences to be made about the CED cycle, which is separately managed by the client-side engine.

One might imagine that tracking the times of certain previously identified 'marker' objects within the QSR stream could solve this problem, especially since I already concluded (above) that marker objects will be needed to delimit the logical pages of RIAs. But in order to be able to draw conclusions about CED times by seeing those markers in the QSR stream, a measurement tool must impose a lot of constraints on the design of the client-side engine. An engine that implemented truly asynchronous behaviors (such as anticipatory buffering) would make it difficult or impossible to assess the user's actual experience without a measurement presence on the client side to observe the CED cycle.

Either that, or the marker objects would themselves need to be active scripts that triggered timing events that were somehow transmitted to the server (in a manner similar to ARM), rather than simply being passive milestones. But once again, this approach is tantamount to placing a measurement capability on the client. (Indeed, in the context of RIAs, dynamically distributing little measurement scripts that function as extensions to the client-side engine would be a natural approach). I therefore conclude that an approach comprising only passive listening on the server side will be insufficient to measure RIA responsiveness.

RIAs: How to Measure?
We have seen that RIAs will affect where a passive measurement tool can be used. Active measurement tools, because their modus operandi is to simulate the user's behavior, are not affected by this issue -- since they mimic the client, they naturally reside there. For these measurement tools, the issue raised by RIAs is how closely a user's behavior needs to be emulated.

User Actions: First, note that RIAs can generate back-end traffic in response to any user action, and not only when the user clicks. For example, the Google maps application can trigger preloading of adjacent map segments based on the direction of a user's cursor movement within the map display. Therefore to measure RIA responsiveness, an active measurement tool must simulate the user's actions, not the engine's actions. This further explains why active tools must reside at the client.

In other words, using the terminology introduced earlier, active measurement tools must drive the CED cycle, not the QSR cycle. The former involves driving the client-side engine to obtain its backend behavior; the latter would require the tool user to supply a complete script of the engine's behavior to the active measurement tool. The latter involves more work and is inherently more difficult and mistake prone, and therefore much less useful.

Think Times: Second, an active measurement tool must properly reflect then fact that a client-side engine may be doing useful work in the background, while a user is reading the previous response or deciding what to do next. It may, for example, be prefetching content in anticipation of the user's next action. Therefore the time a user devotes to these activities -- jointly represented as 'think time' in the RIA Behavior Model -- may affect their perception of the application's responsiveness.

For passive measurements of real user traffic, this is not a problem, because their measurement data always includes the effects of the users' actual think times. But for traditional Web apps, synthetic measurement tools have not needed to simulate think times by pausing between simulated actions, because introducing such delays would not have altered the result. When measuring an RIA however, setting think time to zero (as is typically done today) could have the effect of eliminating or minimizing potential background preloading activity, thus maximizing the perceived response times of later user actions.

And because the engine's behavior during think time varies by application, a measurement tool cannot simply measure content download times then introduce simulated think times during the reporting phase. Combining individual component measurements after the fact to produce a realistic estimate of user experience would be like trying to construct a PERT chart for a volatile project when you are not sure you know all the tasks and also cannot be sure about all their interdependencies -- in other words, impossible.

While people familiar with the project could probably construct the chart and draw conclusions about the project's duration, a general-purpose tool cannot. But in software performance and analysis work, the most difficult and error-prone aspect is combining low level measurements to draw conclusions about user-related metrics like transaction response time. So most users of application measurement tools want to be told the bottom line, namely the user experience, not just the underlying component times.

Therefore I conclude that to reflect a user's experience, an active measurement tool will have to simulate user think times during the measurement process. Using realistic think times (as the best load testing tools already do today) will produce the most realistic measure of the response times a user perceives during a particular application use case.

Summary
Since this has been a long post I will now summarize my conclusions, with links back to the discussion behind each:

The variety of possible RIA behaviors creates new opportunities for developers to make performance-related mistakes, requiring more systematic approaches to measurement.
Concurrent client/server interactions make it difficult for measurement and analysis tools to correlate seemingly separate pieces of data with a particular application activity.
RIA push models (like 'Comet') will require the invention of new metrics to measure effectiveness.
Milestones must be specified in advance to allow measurement tools to group RIA measurements into 'logical pages' or tasks.
To compare the performance of a RIA with a traditional Webapp, you must measure equally important activities within each.
Passive monitoring of server requests will be insufficient to determine a user's perception of RIA responsiveness.
Active measurement tools must simulate user actions, not just engine actions, to measure RIA responsiveness.
Active measurement tools must simulate user think times during the measurement process, to reflect a user's experience accurately.

Next ...
This completes (for now) my analysis of the challenges of measuring RIAs; any further ideas will have to wait until future posts. Next I will consider how the introduction of RIAs may affect SLM processes that were designed and fine-tuned to manage traditional Web applications.

Managing Rich Internet Applications [5]

2006-03-17T08:32:00.000+00:00

This is the fifth post in a series devoted to the challenges of Service Level Management (SLM) for Rich Internet Applications (RIAs). In these applications, some processing is transferred to the Web client while some remains on the application server. Previous posts introduced the subject and the SLM topics I plan to address, reviewed RIA technologies, and introduced The RIA Behavior Model.

Measuring RIA Responsiveness: Introduction

Having been sidetracked by my interesting but futile search for the origin of the saying 'you can't manage what you can't measure', I am now almost ready to examine some of the challenges of measuring Rich Internet Applications. But first I must spell out the motivations for measuring an application, and explain why the choice of an active or passive measurement methodology is of only secondary importance to this discussion.

Why Measure Applications?
Reasons for measuring application performance fall into two classes, one client-centric, the other server-centric. They are:

MEASUREMENT: Determining how quickly the users can achieve their goals:
- (a) Verifying that the application meets its service level objectives
- (b) Detecting abnormal application behavior (typically, slow response)
- (c) Identifying bottlenecks in application components
- (d) Comparing builds, versions, or releases of an application
- (e) Comparing two applications (e.g. our app vs. competitor's version)
LOAD TESTING: Determining how a system behaves under increasing load:
- (a) Verifying that the system will be able to handle the projected traffic
- (b) Determining how many users a given server environment can handle
- (c) Predicting bottleneck components as workload levels grow
- (d) Comparing builds, versions, or releases of an application
- (e) Identifying components that fail after extended use

As I have indicated, activities in the first class are conventionally called 'measurement', while those in the second are referred to as 'load testing', or simply 'testing'. So although both classes require us to address many common measurement problems, I will simplify my description of these problems by using the conventional terminology to distinguish the two classes of activties.

Active vs. Passive Measurement
A few years ago I edited a version of the Network Measurement FAQ originally produced by a CAIDA working group, for publication by CMG. The online FAQ seems to have gone into hiding, but you can still download the CMG paper, Fundamentals of Internet Measurement: A Tutorial. Although it is about network measurement, a couple of paragraphs introduce the concepts of active and passive measurements:

Passive measurements are carried out by observing normal network traffic, so they do not perturb the network. They are commonly used to measure traffic flows, i.e. counting the number of packets and bytes travelling through routers or links between specified sources and destinations.

Active measurements, on the other hand, are performed by sending test traffic into the network. For example, one might measure a network's maximum carrying capacity by sending packets through it and increasing the sending rate until the network is saturated. Clearly one needs to be aware that active measurements impose extra traffic onto a network and can distort its behavior in the process, thereby affecting measurement results.

When measuring Web applications, similar definitions and concerns apply, except at the level of application traffic and the server infrastuctures that deliver applications. While load testing requires an active measurement approach, application responsiveness can be measured using either active or passive methods.

However, no matter which is used, the passive and active measurement approaches differ only in the way application traffic is generated -- both still require mechanisms to measure that traffic. Passive measurements must capture the behavior and experience of real application users, while active measurements must do the same for synthetic users, that is, computers mimicing the behavior of real users.

This is fortunate, because to discuss the pros and cons of real and synthetic monitoring would require a major blog post. In fact I gave a short presentation on that topic at Interop last year, which would provide the structure required. But for now I will simply note that both real and synthetic application users have to be measured, and RIAs present many common measurement challenges, which I will be describing in my next post.

Deep Thoughts on Management

2006-03-14T21:48:00.000+00:00

Since I am writing a series of posts about managing Rich Internet Applications, and working on a post about the difficulties of measuring them, I thought I should begin with the popular management aphorism that you can't manage what you can't measure. Well, was that ever a diversion! Everyone is familiar with this saying, but interestingly, despite a ton of digging on the Web, the precise origin of this saying remains obscure (at least, to me).

Perhaps because of my training in statistics, I had always assumed it was just a simplification of a famous statement by Lord Kelvin, often cited by statisticians:

When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind. It may be the beginning of knowledge, but you have scarcely advanced to the stage of science.

Unfortunately, there does not seem to be a lot of support for my assumption. Instead, my Web searches have revealed that the statement 'you can't manage what you can't measure' is most often attributed either to the pioneer of quality control, W. Edwards Deming, or to 'the father of modern management', Peter Drucker. Moreover, neither of these popular attributions seems to be correct.

In Deming's case, one of his most well-known statements is almost the exact opposite, namely that management must take account of many things that cannot be measured, and that running a company on visible figures alone was one of the seven deadly diseases of management. All the same, I'm sure Deming, given his interest in quality control, would have also agreed that things which can be measured should be measured, And all SLM processes are based on measurements.

Attribution to Peter Drucker also seems to be a misconception, since it does not appear either in the wikiquote list of quotes from Drucker, or in other extensive lists of his well-known sayings. But doing this research reminded me of the many insightful statements Drucker actually did make, including:

Management is doing things right; leadership is doing the right things.
There is nothing so useless as doing efficiently that which should not be done at all.
A manager's task is to make the strengths of people effective and their weakness irrelevant--and that applies fully as much to the manager's boss as it applies to the manager's subordinates.
Management is so much more than exercising rank and privilege, ... it is much more than "making deals." Management affects people and their lives.
Rank does not confer privilege or give power. It imposes responsibility.
Executives owe it to the organization and to their fellow workers not to tolerate nonperforming individuals in important jobs.
Most of what we call management consists of making it difficult for people to get their work done.
The most efficient way to produce anything is to bring together under one management as many as possible of the activities needed to turn out the product.
Increasingly, politics is not about "who gets what, when, how" but about values, each of them considered to be absolute. Politics is about "the right to life"...It is about the environment. It is about gaining equality for groups alleged to be oppressed...None of these issues is economic. All are fundamentally moral.
What's absolutely unforgivable is the financial benefit top management people get for laying off people. There is no excuse for it. No justification. This is morally and socially unforgivable, and we will pay a heavy price for it.
Unless commitment is made, there are only promises and hopes... but no plans.
The best way to predict the future is to create it.

This post is an example of what happens when I do research on the Web -- I find a lot of interesting information I was not even looking for. So this post became a diversion about the management insights of Edward Deming and Peter Drucker, and in my next one I will get back to the subject of measuring RIAs.

Managing Rich Internet Applications [4]

2006-03-10T05:25:00.000+00:00

This is the fourth post in a series devoted to the challenges of Service Level Management (SLM) for Rich Internet Applications (RIAs). In these applications, some processing is transferred to the Web client while some remains on the application server. Previous posts introduced the subject, listed topics I plan to address, and reviewed RIA technologies.

The RIA Behavior Model

In an earlier post I discussed the idea of using a reference model to establish a shared frame of reference, or conceptual framework, to structure subsequent discussions of a subject. Today I introduce a new reference model -- The RIA Behavior Model -- illustrated by the diagram below.

I intend this model to represent the principal elements of the behavior of RIAs, elements that must be considered in any discussion of RIA performance and management. Note however that I am not attempting to address the complex human forces that determine perceptual, cognitive, and motor behaviors. I am merely seeking to represent a few generalized behavioral outcomes that are relevant in the context of an interaction between a human user and a Rich Internet Application.

As you can see, this model embraces more concepts than the two figures created by Jesse Garrett to introduce Ajax, which I included in my previous post. Those figures are ideal for explaining the differences between traditional Web applications and RIAs. But to discuss how the combination of a Rich Internet Application and a user will actually behave, I have included several more elements that interact to determine behavior, which I will now describe.

At the highest level, the model comprises three major aspects (indicated by the color coding in the figure), each of which influences application performance:

The application's design and usage environment, or context (upper row, grey)
The user's expectations and behavior (lower left, blue)
The application's behavior when used (lower right, yellow)

Browser/Server Interaction
If we consider a Web browser to be the simplest form of client engine, then the solid black arrows trace the flow of a traditional Web page download. The user clicks on a link in their browser, the browser sends requests to one or more servers. Servers respond to client requests, and when there is enough of the requested content on the client (in the browser cache), the browser renders it ('paints' the screen), and the user can view it. The user's experience of response time is the elapsed time of the entire process, 'from click to paint'.

Even a single Web page download will normally involve many round trips between client (browser) and server, because most Web pages are an assemblage of content elements such as CSS files, script files, and embedded images, each of which is separately downloaded by the browser.

In the traditional synchronous Web application, illustrated in the upper half of Garrett's Figure 2, this process is repeated several times. Because applications usually require an exchange of information, at least one of the requests the browser sends to the server will normally be an HTTP POST (as opposed to the much more common HTTP GET request), to upload some data a user has entered into a form. Consider, for example, shopping at amazon.com as a return visitor. At minimum, even if the application recognizes you from a cookie, you must enter your password again to confirm your identity. But after that, the site already has all your personal information, and you can complete your transaction just by clicking on the right buttons on each page as it is presented to you.

Server-Side Elements
We are all familiar with this kind of Web application and its behavior. But unless you are responsible for site management or performance, you may be less aware of some of the other server-side elements of the model. Servers must field requests concurrently from many users. No matter how powerful the server, every concurrent user consumes their small share of the server's resources: memory, processor, and database.

Web servers can respond rapidly to stateless requests for information from many concurrent users, making catalog browsing a relatively fast and efficient activity. But when a user's action (such as clicking the 'add item to shopping cart' button) requires the server to update something, more of those server resources are consumed. So the number of concurrent transactions -- server interactions that update a customer's stored information -- plays a vital role in determining server performance.

In the model, the grey arrows and the boxes labeled Users and Transactions represent the fact that server performance is strongly influenced by these two concurrency factors. Servers typically perform uniformly well up to a certain concurrency level, but above that level ('the knee of the curve') transaction performance quickly degrades, as one of the underlying resources becomes a bottleneck. Because of this characteristic, seemingly small design changes in an application or in the infrastructure serving the application may, if they extend the duration of transactions, have a significant effect on the user's experience of response time.

Adding a Client-Side Engine
Adding a client-side engine does not prevent an application from implementing the traditional synchronous design described above. But RIAs aim to improve the user's experience, and a client-side engine allows the application designer to consider many additional possibilities, such as:

Prefetching of content to client
Lazy loading of content
Just In Time fetching of content
Client-side validation of user input
Client-side only responses to user input
Batching of server inputs on the client
Offloading of server function to client machines

If any of these techniques are employed, the resulting application will inevitably be more complex than the traditional synchronous application. The challenge of SLM is to ensure that a more complex design actually does produce a more responsive user experience, because -- despite the optimistic claims being made for Ajax and Flash -- there are no guarantees.

Improving the User Experience
For example, a common method of accelerating client-side response is to anticipate the user's next action(s) and program the engine to preload (or prefetch) some content 'in the background', while the user is thinking. Depending on their think time, when the user clicks, part or all of the desired response can be already available on the client. This technique has long been used in client/server systems to improve client responsiveness; a version (called Link Prefetching) is implemented by Mozilla browsers.

Preloading will certainly create a more responsive experience -- if the user actually follows the anticipated path through the application. But what if they don't? Then the client engine has made extra server requests that turned out to be unnecessary, and it still has to react to the user's input with yet more server requests. So the design has placed extra load on the servers, for no benefit.

The extra load serving these 'background' requests, on behalf of what may be hundreds or even thousands of clients running the preloading application, can slow down a server's responses to requests that users are actually waiting for. Slower responses lengthen transaction times, which drives up the number of concurrent users, clogging up the servers even more, and further slowing down responses. It's a vicious circle.

Even if the application serves some users quickly, those whose usage patterns do not match the profile the application designer had in mind will probably not receive such good service, and (in the worst case) may abandon transactions before completing them. Apart from the lost opportunity to serve a customer, abandonment usually also wastes scarce server resources, because the allocations earmarked for now-abandoned transactions languish unused until finally freed by some type of timeout mechanism.

People who design and test back-end systems already know that behavioral variables like user think-time distributions and abandonment rates per page have a significant influence on the capacity and responsiveness of servers under load. Today, RIAs (as indicated by the dotted lines in the diagram) give application designers the flexibility to create application designs that attempt to take account of such behavioral variables. But a consequence is that RIAs also magnify the risk of failure should the designer miscalculate: a simple design applied well beats a clever design misapplied.

The Application Environment
This brings us to the crucial importance of the design and usage environment, represented by the grey boxes in the model. A user's satisfaction with any application depends on their usage context and environment; in other words, how well the application design matches their needs at the time, their way of thinking, and their behavior when they are using it.

Their experience of response time depends on the combined behaviors of the client and server components of the application, which in turn depend on the application design, the underlying server infrastructure design, and of course the user's Internet connection speed. The most effective RIA will be one whose creators took into account these factors at each stage of its development life cycle, and who created the necessary management processes to ensure its success when in production.

Future Posts
Using The RIA Behavior Model as one starting point, future posts will discuss how to build robust and responsive RIAs, and explore some of the technical and management challenges they pose, especially in the area of response time measurement. If I can organize my thoughts sufficiently to keep the writer's block at bay, I expect my next post to focus on that topic.

Managing Rich Internet Applications [3]

2006-03-07T04:56:00.000+00:00

This is the third post in a series devoted to the challenges of Service Level Management (SLM) for Rich Internet Applications (RIAs). In these applications, some processing is transferred to the Web client while some remains on the application server. Previous posts introduced the subject, and listed the topics I plan to address.

RIA Technologies
Before diving into the management issues posed by Rich Internet Applications, I will introduce the two principal technologies used to implement RIAs -- Ajax and Flash -- and provide a few links I have found useful.

I am going to begin with Ajax, because the seminal article defining the term "Ajax," Jesse James Garrett provides such a clear introduction to the subject. After explaining that ...

Ajax isn’t a technology. It’s really several technologies, each flourishing in its own right, coming together in powerful new ways. Ajax incorporates:
standards-based presentation using XHTML and CSS;
dynamic display and interaction using the Document Object Model;
data interchange and manipulation using XML and XSLT;
asynchronous data retrieval using XMLHttpRequest;
and JavaScript binding everything together.

... he uses two figures that reveal the essential differences between a classic Web application and one implemented using Ajax:

The classic web application model works like this: Most user actions in the interface trigger an HTTP request back to a web server. The server does some processing — retrieving data, crunching numbers, talking to various legacy systems — and then returns an HTML page to the client... This approach makes a lot of technical sense, but it doesn’t make for a great user experience. While the server is doing its thing, what’s the user doing? That’s right, waiting. And at every step in a task, the user waits some more.

Obviously, if we were designing the Web from scratch for applications, we wouldn’t make users wait around. Once an interface is loaded, why should the user interaction come to a halt every time the application needs something from the server? In fact, why should the user see the application go to the server at all?

Figure 1 illustrates how Ajax is different:

Figure 1: The traditional model for web applications (left) compared to the Ajax model (right).

While this diagram refers to Ajax technology, its structure describes RIAs in general. Garrett explains that the RIA model:

... eliminates the start-stop-start-stop nature of interaction on the Web by introducing an intermediary — an Ajax engine — between the user and the server. ... At the start of the session, the browser loads an Ajax engine — written in JavaScript and usually tucked away in a hidden frame. This engine is responsible for both rendering the interface the user sees and communicating with the server on the user’s behalf".

Figure 2 (below) illustrates how the addition of a client-side engine ... "allows the user’s interaction with the application to happen asynchronously — independent of communication with the server".

Figure 2: The synchronous interaction pattern of a traditional web application (top) compared with the asynchronous pattern of an Ajax application (bottom)

In the best case, a client-side engine can mean that users spend less time waiting for the server to respond. Like all writers making the case for Ajax and RIAs, Garrett assumes that this architecture guarantees a more responsive user experience -- but the reality is more complicated. In practice, an RIA's responsiveness will depend on several factors that I will be exploring in this series of posts.

For more detailed discussion of the history of RIAs and Ajax, I recommend Aleksandar Šušnjar's Wikipedia page about RIA and AJAX. For more technical details, see these Sitepoint articles:

What about Flash?

You may feel I should have started with Flash, because it came first, but opinion is divided on that point. It is true that Macromedia announced their Flash MX product line in 2002 (see Developing Rich Internet Applications with Macromedia MX for a good summary), whereas the term Ajax was coined only in February 2005. However, the underlying RIA techniques have been in use for much longer. For example, on his Wikipedia page Aleks cites the pioneering work of the now defunct company, Desktop.com. In 1999, Dave Winer wrote in his blog:

Want a vision of where the Web is headed? Check out Desktop.Com. Launched in beta a couple of weeks ago, this stunning site changed my point of view on what can be accomplished with JavaScript, ActiveX and whatever other kinds of mysterious code-doo-dads they're using... Desktop.Com says the Web is a desktop, just like the desktops on the Mac and Windows. Icons down the left edge of the browser window, menus at the top of the window, double-click to open a directory, double-click to edit a file. -- Dave Winer

If you need to compare Ajax and Flash, here are four links you may find interesting:

Any search engine will turn up plenty more material on this subject, and I will return to it when discussing aspects of RIA measurement and management.

You can download many reports and papers about Flash and RIAs from the Adobe (formerly Macromedia) Web site -- for example, these. Not included in that list is one of the most readable introductions to Rich Internet Applications, a 2003 paper sponsored by Macromedia and Intel and written by Joshua Duhl of IDC, an excellent writer who I once worked with (in the early 90's) at ONTOS, a Boston-based Object DB company. Another former colleague from Boston, Alan Sarasohn, used to claim that the computer industry is run by just 300 people, but they keep moving around. I'm starting to believe him!

Managing Rich Internet Applications [2]

2006-03-02T08:20:00.000+00:00

This is the second post in a series devoted to the challenges of Service Level Management (SLM) for Rich Internet Applications (RIAs). In these applications, some processing is transferred to the Web client while some remains on the application server.

In the first post in this series, I introduced the concept of a Rich Internet Application, and asserted that anyone planning to deploy RIA technology must be prepared to invest more in design, development, testing and management to make it successful. By that I meant that deploying an RIA successfully will demand more resources -- skills, tools, process, time, and (of course) money -- than would be required to deploy a more traditional version of a Web-based application in which all application logic is executed on the server.

I have reached this conclusion after studying this subject for the last three months, in the light of my prior experience in the field of software performance management. Factors leading to this conclusion include:

the nature of much of the technology being promoted for building RIAs today (primitive)
the stage of evolution of this new application architecture (early)
the toolsets available to support RIA developers (incomplete), and
the experiences of others developing RIAs (hard work).

This comment by Todd Spangler in Baseline Magazine neatly sums up the situation today:

Creating an Ajax application from scratch is like having to build a brick wall but first having to figure out how to create the bricks.

This reminds me of the days when I used to program in IBM Assembler -- funny how life in the world of computing seems to repeat itself every few years! (And I'll have more to say about that later). But while this is all fascinating stuff, it is not my primary focus; it is just the starting point for the SLM topics I want to discuss. So although I will devote some space to reviewing the state of RIA technology, I will try to keep those posts concise and provide useful links. If you want more, a few searches will turn up tons of reference material.

The questions I plan to explore in more depth in this series are:

Why are RIAs so much more difficult to design, develop, and manage?
How do you measure an RIA user's experience of response time?
How do you test whether an RIA will perform acceptably in the production environment?
How do you break apart the time it takes to complete a business transaction using an RIA into meaningful components?
How do you monitor the performance of an RIA in the production environment, and alert when its behavior is abnormal?
What kinds of development and systems management processes will maximize the chances of implementing an RIA successfully?

Before I get into any of these details, I'd like to acknowledge the contributions of two people who have helped shed light on these issues. This first is Victor Pavlov, a principal engineer and colleague at Keynote, who knows Web technology inside out, and can figure out how to measure anything. I am lucky that when I am noodling on a tricky question, I can sometimes run into Victor at the coffee machine.

The second is Aleksandar Šušnjar, who I met online after reading his contributions to the discussion of RIAs in Wikipedia, where he shared insights gained from his experience building a product first released in 2002 by his company, Hummingbird. This product, DM Webtop (later renamed DM Web Server) is a component of their Enterprise Document Management suite. In delivering desktop capabilities via the Web, it clearly predated much of today's thinking about the purpose of RIAs and how to build them.

Naturally, Aleks also discovered some potential pitfalls an RIA designer might run into, which I will be mentioning in later posts. Regrettably, many of his insightful contributions to Wikipedia (which can still be found in its history pages) were subsequently removed by other contributors who lacked either his intimate knowledge of the technology or his historical perspective. Rather than waging Wikipedia edit wars, which for a hot topic can continue interminably, he has simply posted his own page about RIA and AJAX.

I am not surprised by these events, because I have found that the computing world tends to attract people who have little sense of history. This explains why it is forever reinventing the wheel, repeating yesterday's mistakes, and tripping over previously solved problems. Rich Internet Applications are a case in point. Mainframe (thin client) computing was replaced by the client/server (fat client) model, which in turn was displaced by the Web-based (thin client) applications. Now the emergence of RIAs signals a return to the fat client model again. The only difference between the client/server and RIA models is that instead of Sneakernets and static installation protocols, Internet and Web technologies now provide an almost universally available platform for distributing function to client desktops dynamically. But many of the over-optimistic claims being made for RIA technology today mirror those popular 15-20 years ago, when everyone was jumping on the client/server bandwagon and predicting the imminent demise of the mainframe -- which, by the way, never happened.

So I'll end by reminding you of the famous saying of philosopher George Santayana: Those who cannot remember the past are condemned to repeat it. One of my goals in this series of posts about RIAs is to keep you from that fate, so I'm glad to be collaborating with people like Victor and Aleks whose common sense is grounded in a strong sense of history!

Performance Matters

Index of Migrated Posts

Foundations

Performance as an Aspect of Usability

Web Usability Books

SLM Overview

Managing Rich Internet Applications

Web Performance Engineering: Building in Performance

Setting Performance Objectives

Capacity Planning and Load Testing

Measuring Performance

Detecting, Diagnosing, and Fixing Problems

Reporting on Performance

SLM Cost/Benefit Analysis

Performance in the News

Recent posts now migrated