Last night I joined the Beyond Web Analytics panel for a podcast on going – well - beyond. In this case, going beyond reporting to analysis. If you haven’t heard about Beyond Web Analytics, it’s put together by Rudi Schumpert who is joined by Adam Greco (always terrific) and James Dutton for what’s planned to be a regular series of podcasts on all things analytics. I was a guest last night on going beyond reporting to analysis - a topic that’s definitely meat and potatoes to me. I don’t think most organizations do nearly enough analysis on their data. Even among Semphonic clients, most could be much more aggressive about analysis not just reporting.
It’s a nice forum and the panel format – not my favorite for conferences – works well for podcasts. Listening to one person talk always feels a little tiresome after awhile – even when that person talks very well. The conversational style is easier to listen to (at least for me). And since they are a regular panel, I can see them really beginning to interact and converse, removing my main objection to panels which is that the participants always seem to be talking to an audience in their head not to each other.
The panel has a nice balance and it could end up working quite well indeed. I’m also going to be on the next discussion (about staffing) and I’d be a guest anytime they’d have me. I’d like to see it really work and build an audience.
You can hear our discussion and check out the ongoing series here.It’s oh so typical that just when most of us start to feel pretty good about our knowledge and methods for measuring our own web sites, everyone starts asking us about what’s going on in the greater online space. Measuring communities, conversation, and social media is at once fascinating and absurdly challenging. The tools are primitive, the infrastructure almost non-existent, the questions largely undeveloped and even the business goals are poorly understood. It’s just like old times! So I’m delighted to be doing a webinar on measuring social media with my friend Scott Wilder.
Scott and I worked together for the last few years as he built-up Intuit’s impressive online communities. He’s that rarest of breeds – someone who uses social media and technology with true passion and combines that with real business acumen. Like lovers of ballet and football, neither one nor the other is particularly rare, but the two almost never to comingle!
Our plan for the webinar is to start with an examination of online communities and measurement. We’ll cover some of the basics of community measurement then dive into measurement based on both audience and organizational segmentation. This is interesting stuff. If you have or are thinking about a community, it should really help you understand how to measure your success. We also hope to provide some perspective on whether – if you are on the outside looking in – building a community is right for your business.
For many organizations, it’s cheaper and more practical to take the conversation to where your customers already live. The second part of the webinar will focus on the different uses of measurement in this broader online space. Scott and I will talk over how to find your audience online, how to measure the success of your brand-monitoring and customer support efforts, how to measure influencer programs, and how to track impacts on your own sites. Don’t expect miracles – this stuff is hard. But we’ll try to provide real solutions and along the way we’ll cover everything from old-school techniques like opinion research to new fangled tools like Radian6.
At this early stage in the game, it isn’t really sensible to talk about measurement without regard to strategy. Social Media strategies haven’t developed to the point where tactics are the primary focus of measurement. So our focus will be on how to use analytics to target your social efforts and establish whether you’re effective enough to continue.
For a take-way, we’re putting together a “Metrics Menu” that captures what we think are the most important types of measurement to focus on when you’re getting started.
Scott is the most effective and thoughtful strategist and practitioner in this space that I’ve yet encountered. I think you’ll really enjoy hearing him. If you’re thinking about a social marketing strategy or are responsible for social media measurement, do join us!
Online Applications and highly-interactive web sites written in full-on programming languages are becoming the norm on the web, not the rare exception. Measuring these applications is challenging for a whole host of reasons. In my last post, I talked about the inevitable trade-offs between the desire for extensive measurement and the cost-of-measurement in a SaaS environment. In today’s post, I’m going to discuss some of the ways that analyzing applications is a little different than analyzing more traditional we sites.
One of the biggest differences I see between measuring applications and measuring websites is that application developers are far more interested in measuring the efficiency of their GUI. Traditional web sites have far fewer moving pieces – and the main concern is how well each page (as a unit) functions. But applications have many more options to consider and GUI developers are almost always deeply interested in understanding what users need most to accomplish any given task.
Much of the framework I suggested for measuring applications is driven by this recognition that applications can’t be measured quite like web sites. In my framework, units-of-work replace page views as the “core” ingredient in measurement. In addition, I added the concepts of “state” and “performance” to the unit of work.
From these concepts, a fairly powerful set of GUI analysis reports fall out quite naturally. It’s easy to report on which tasks are most commonly used; which tasks are done first in an application; the order of tasks and their associations (which are done together). In addition, the analyst can easily tell which application options are used most often (from application states); how application states are changed for each task; and which application states are most likely to result in task success. Finally, the performance measurements lets application developers compare how much time each task takes in total to how much of that is human time vs. machine time. It also makes it easy to see if time is impacting task success rates.
This is all good stuff and tremendously useful for GUI designers. And as I’ve said before, this type of measurement CAN be done with current web analytics systems that lack my suggested framework; it just requires quite a bit more work setting up the measurement infrastructure and hacking the reports.
There are some other types of application analysis that fall out a little less obviously from the framework and are distinct from traditional web analysis.
Unnecessary Clicks: Part of improving any GUI is making the path to task completion as swift and short as possible. The same forces are at work in traditional web sites. In a Functional analysis, we’re always looking for unnecessary pages – things like Router pages where the vast majority of users choose a single route. But use-cases are harder to apply to the open-ended non-task based paradigm that most web sites live in. With applications, however, use-case analysis (as embodied in my units-of-work concept) is probably the core methodology. For this analysis, the goal is to understand the swiftest path through any task and then identify and count all of the types of actions that deviate from that path. By identifying common deviations, designers may be able to bake-in settings, information or functionality to a process so that the user doesn’t have to – improving the overall performance of the application.
Human Performance Curves: One of the most difficult trade-offs an application designer faces is structuring a GUI that works well for beginners and experts alike. There is almost never a perfect solution – most interfaces will tend to optimize one experience or the other. But understanding the trade-offs doesn’t guarantee that your design matches your expectations. Analyzing this is non-trivial and quite different than most web site analysis. The best way to approach the problem is with performance curves. In the framework I suggested, performance times are automatically captured for each unit-of-work. These performance times measure both human time and machine-time. For this analysis, you need to plot the performance times against the number of times the user has done the task. This will yield a curve of times by usage. In an application that draws lots of regular users, you’d probably hope to see times drop sharply on repeat usage since the application should be designed to reward regulars. For infrequently-used applications, you’d probably hope to see a much flatter curve – the GUI should focus on getting all visitors through the process and not so much on speed improvements and options for heavy users.
Onboarding Success / Time: This is an analysis closely related to Performance Curves – the idea is to focus on the success of first time users. First use is critical to any application – first impressions do matter – and converting triers to regulars is vital. But the performance of first time users can be obscured if you focus on averages; especially if you have a significant population of regular users. These regulars are likely to be so good at the application that they make every measurement look good – performance times and completion rates will all be outstanding. By focusing on only first-time user performance, you can correct against that bias and better understand potential usability issues.
Help Rates: In traditional web analytics, people rarely go look for help. Well, they don’t do it that often in applications either! But there is a traditional analysis equivalent – we often analyze the internal search rate from pages, especially router pages. This rate is nearly always fairly low. But there are significant variations by page that highlight usability issues. Pages with high-rates of internal searching often lack pointers to types of content the user is expecting. "Help" usage in applications is similar. Tracking context sensitive help by unit-of-work is a good way to identify trouble spots. Remember, it’s quite unlikely that 50% of visitors are going to look for help ANYWHERE. You’re looking for tasks where the help rate is significantly higher even if the difference is between a rate of 3% and a rate of 6%. It’s almost like counting Voice of Customer complaints by task – the evidence is anecdotal but each complaint is likely a stand-in for many other users.
Common Variations Analysis: Applications are built because they provide a user with the flexibility to try different combinations of factors. It doesn’t matter whether you are talking about configuring options on a car, selecting a life-insurance policy, designing a portfolio, or choosing a vacation – the underlying goal of a good application is to make it as easy as possible for users to explore the options that might interest them. Out of any set of options, however, there are bound to be strong patterns and relationships between individual options that the application designer might take advantage of to make the application more intuitive and powerful. If you know which combinations users tend to explore in sequence or together, the application can be designed to automatically create comparisons or at least to suggest and short-cut navigation to these associated options. In my measurement framework, applications states would be used to capture this type of information. But the same analysis can be done within current measurement frameworks though you’ll probably have to export the data to a SQL-Database or statistical package after capturing use states in custom variables.
There are, of course, an unlimited number of different and often highly specific analysis methods for applications. But just as certain methods of analysis have proven routinely fruitful for web sites (functional analysis, real-estate analysis, funnel analysis, etc.), these analysis methods fit many a different application. People often assume that having a good tool is the same as having good measurement. But as I've said repeatedly in this series, tight integration of measurement into an online application development environment is just a piece of the solution.
I’ve written this series as a sort of open-letter about what I’d like to see companies like Adobe, Google and Microsoft do to integrate measurement with online applications. But along the way, I’ve tried to point out how most of what I’m recommending can be done in today’s measurement system. It takes a lot more work than it should, but it’s still quite possible. Since nobody ever did useful work by waiting for tools to improve, I want to close by emphasizing this last point. People are already building application and dynamic web sites. Nobody’s waiting for good measurement paradigms to emerge and be incorporated into their tools.
By getting involved with the application development cycle early on, understanding what kinds of concepts are important to application measurement, doing your best to make your measurement infrastructure clean and testable, and adapting your analysis plan to meet the needs of applications, you can bring measurement value to the table right now.Online Applications and highly-interactive web sites written in full-on programming languages are becoming the norm on the web, not the rare exception. Measuring these applications is challenging for a whole host of reasons. In my last post, I talked about the importance of automated testing to bring measurement fully into the application development life-cycle. In today’s post, I’m going to cover an issue that plagues application measurement, video and most other rich media measurement – how to balance cost vs. coverage when it comes to measurement.
The problem of cost in measurement springs from our basic model. With SaaS implementations, each server call you make to your measurement system costs money. Not a lot of money, of course. And for web sites, there has been little doubt that the knowledge gained from page measurement more than offsets the per-page cost of collection. On most web sites, a typical visitor might view six to ten pages. A really engaged visitor might view twenty to thirty pages. In an online application, however, the number of user interactions (every click, zoom, and setting) can be much, much higher.
Without a clear paradigm for a unit of measurement (such as a page view), it’s unclear which of these interactions should be measured. Capture every one, and you’ll likely find that the cost of measurement outweighs the benefit.
As I mentioned, we’ve seen similar issues with video measurement. Many of our media clients went from measuring almost nothing about video to detailed measurement of how far viewers penetrated into each and every visitor played. But to get that measurement, they usually had to make multiple server calls per video – sometimes as often as every 10 seconds. For a 2 minute video, that means a full-play results in 12 server calls. That’s a lot of measurement expense relative to the value of tracking abandonment per 10 second interval.
In fact, a lot of that more detailed measurement ended up getting pulled.
We’ve seen similar pull-backs when clients over-measured flash experiences – tracking roll-overs for example. After awhile, most companies find they aren’t using that level of detail about user interactions very effectively and the cost of measurement isn’t justified.
In an earlier post, I described a framework for measuring applications that included four core concepts: units-of-work, application state, rest-states, and performance. Part of the reason I settled on these concepts is that they provide a reasonable balance of measurement vs. cost.
A unit-of-work is a function within the application. The unit-of-work would typically encapsulate a single use-case (like getting driving directions for a mapping program or paying a bill in an online bill-pay system). The measurement system would make a call whenever a unit-of-work is initiated. With this call (and all calls) it would pass an application state.
The application state is a set of variables that capture all the important settings a user might have (like sort-order, filters, zoom level, tab open, etc.). The goal of application states was to be able to tie application settings to important actions (units-of-work) without having to pass measurement calls every time a state is changed (like a sort).
Now I’m going to add a new concept – milestones. Milestones would be way-markers within units-of-work. A milestone could be named and sequenced if desired. Whenever a milestone within a unit-of-work occurs, the measurement system would fire off a call. Every unit-of-work would have at least two associated milestones – an abandon milestone and a completion milestone.
Using the milestone concept, developers could control how much measurement is being done within a unit-of-work.
When a milestone is reached, the measurement system would automatically get a new application state and it would generate and pass a performance measurement for the previous milestone.
With this type of framework, there would be little need to track every user change or setting with a server call. Each measurement call would capture some significant action along with the associate states. On the measurement end, comparing states between milestones would make it easy to produce reports about what settings users changed when performing a unit-of-work.
The unit-of-work/milestone combination provides developers with a very granular way to control the level of measurement in the application (that’s important) and, potentially, to be very sparing about the cost of measurement.
It’s quite possible to duplicate some of this functionality in existing systems today. In Omniture, for example, you can code units-of-work using an s.prop variable. By turning on pathing, you can track movement between units-of-work. By using a combination of unit-of-work and milestone in a second prop, you can enable pathing within milestones. You can use a list-prop to capture application state or you can use a carefully formatted and delimited string in a normal string s.prop.
If you correlate that application state variable with the units-of-work s.prop and the milestones s.prop, you have basic reporting on how application states changed across milestones. Of course, you have to do most of the work to use those reports since the application isn’t going to really help you any. And you also have to do ALL the work to setup and capture this stuff.
Of course, that’s the whole point of this series. You CAN measure applications right now, it’s just not nearly as easy or powerful as it ought to be.
With that in mind, my next (and final) post in this series will tackle the last and probably the most difficult part of measuring applications – figuring out how to analyze the information to make it useful!When we talk with the development teams these days, I’m often the one who has to break the bad news that there is no reasonable automation strategy for testing measurement. Not only that, but the testing is extraordinarily difficult to integrate into basic development cycles. Someone has to run through a test plan by hand, then go to the web analytics tool and try and shift through the data to isolate the test results. It’s kludgy, manual, slow and error prone. A perfect storm for testers.
This is especially frustrating because measurement isn’t “critical” functionality. Nobody wants to slip deadlines or delay release for measurement.The changes I’ve talked about so far – especially in the measurement interface – would greatly reduce the burden on programmers to get measurement INTO the application. But they really wouldn’t do much to make it easier to test.
I’d like to see Omniture and Microsoft and other application and measurement vendors integrate testing functionality directly into the measurement system. Ideally, developers should be able to configure “testing” as an overlay on the existing measurement interface. In other words, I wouldn’t add any code to test, I’d simply set a variable or an object that controlled the “testing” state.
When testing was turned on, every measurement call would be mirrored to an additional destination. Since online applications typically run locally, I see several ways this might be handled. First, the application could potentially save the measurement in a flat file on the local system (given permission). Alternatively, the application could save the measurement calls to an interface object in some easily parsed format. Finally, the application could pass the calls back to a secondary server that simply recorded them in a flat file on the server side or was part of a testing automation suite.
With this type of setup, testing automation tools could drive through a series of application steps which would produce a measurement file that could be quickly “diffed” with previous versions to establish correctness. This would dramatically reduce the amount of time new testing cycles took while greatly improving accuracy. Even if each test cycle had to be done manually, programmers would only have to worry about checking the measurement file once and getting a correct version.
Even without the automation aspects, this type of system would make testing applications in secure and mobile environments much easier. We often do the bulk of our testing using HTTP sniffers like Firebug, Charles and Fiddler. But these sometimes don’t work or conflict with the application on development networks and in secure environments.
As I think about this, I realize that in some ways this “testing” capability mirrors a universal tag. If the measurement library makes it easy to reflect measurement calls to multiple locations, you have the basic infrastructure for universal measurement.
That seems like a pretty good idea – something that would be almost trivial to add into a measurement framework of the sort I described in the first two posts. Pegging testing on a universal tag capability has the additional benefit that people may pay more attention to it that way. Testing is one of the most time-consuming, manual, and expensive tasks in software development. Unfortunately, as the least sexy task is a discipline with very little sexiness in general, measurement testing gets less real-world attention than the “best script” honors at an adult video awards program.
But if you’re going to build a really good measurement interface, building support for automated testing right into the system would go a long way toward making the incorporation of measurement into online applications routine instead of esoteric and painful.It’s obvious that online applications don’t easily fit the page view paradigm of static web sites. What’s less obvious is what should replace that paradigm. I’m not prepared to give a complete answer to that question. However, in our work actually building measurement into applications, there are four themes that have emerged and seem necessary to any good framework.
Measuring the Application StateWhen we build application measurement into traditional measurement systems, one of the most important concepts we’ve evolved is the idea of an application state. The idea behind an application state is that it captures all of the “settings” that are in place while a user accomplishes some action. Typical elements of an application state are things like “sort type,” “filter-type,” “zoom-level,” and “view-type.” In my last post, I used the example of a mapping application to illustrate some of the concepts involved in a good programmatic interface. It’s a useful example, because it’s a type of online application that virtually everyone is familiar with. In a mapping application, the application state is likely to include variables like “map type,” “overlay-type,” “zoom-level,” and “route-type.”
The idea behind an application state is simple. It allows you to track key combinations of interface elements by application function. It can also be used to reduce the number of calls you need to make to the measurement service (a critical factor for any high-volume application). You don’t need or want to capture every single mouse click in an application.
If a user clicks on zoom or sort, sending a measurement click is generally too expensive. Instead, we capture the entire set of application variables whenever key actions happen. In the map example, this would allow us to say what the most popular zoom levels are when viewing a route or adding a restaurant overlay vs. a hotel overlay. That’s incredibly useful information for GUI designers and its captured much more effectively in an application state than in any other fashion.
When we build an application state into a current measurement system, we generally pass it as a List Prop or a simple string variable. However, it takes some work to get it to tie to application functions in an intelligible manner. I’d like to see an application framework capture the state as a type of thing and allow it to be viewed and counted and analyzed in conjunction with different application functions.Rest States
One of the trickiest aspects of application measurement is what to do about applications where the interaction isn’t discrete. In our mapping example, panning and zooming are a good example of what I mean when I say not discrete. It’s easy to fire off a measurement call when someone clicks a button to change from a street view to a topo view. But when do you fire off a measurement call if someone is zooming? How do you know they are done?Ignoring these non-discrete interactions works for some applications but cripples measurement for others. So what I’d like to see is a measurement framework that includes a threaded callback mechanism that can be attached to various interactions and basically just checks to see if the interaction has stopped for some period of time. I call this stop-time a rest state.
Using my map example, when a user clicks on the zoom control of a map, it would fire a measurement function that started a thread (call it zoomwait). The function would be passed three things – the action it’s checking for, the wait period required, and the measurement function to call when the wait period has been realized. Let’s say we are checking zoom and we have a 3 second wait period. The measurement thread would check to see if a zoom had been done in the last 3 seconds. If it had, it would until at least 3 seconds after the last registered zoom time and try again. It would do this until it detected that 3 seconds had passed since a zoom. It would then call the measurement function specified which would make the actual call to the measurement system and pass the new application state.
The idea behind rest states is to make it possible to measure the application state AFTER the user has finished some set of actions. This makes it possible to capture many key application states without necessarily trying to measure every application state when a user is zooming, panning, or doing some other non-discrete interaction.
Measuring Performance
The current static web site measurement framework largely ignores performance. That’s a serious drawback, but one that’s hard to solve given the limitations on measurement with our current infrastructure. But for applications, measuring app performance is really, really important.
Application frameworks like .Net already include some low-level calls designed to make performance tracking around key functions possible. But not every development framework has this (many do not), and I think it’s a natural part of a measurement system. So I’d like a measurement framework to include this functionality.To measure performance, the system should have a way of starting a unit of work and defining it as being open-ended (not terminated by another start) or closed (terminated automatically if another unit of work starts). Units of work would be named and could be closed explicitly by the application sending a close call with the corresponding name.
The measurement system could use this concept in a variety of ways including measuring total time in application (by firing off an open ended unit of work named application start that is only closed on app clean-up), measuring total time in app areas, and measuring specific application times for functions like “sort” or “synch.”
Depending on how this was used, it could just as easily measure how long a user takes for an application (like filling out a form) or how long an application takes to do processing (like placing an order). In the current web environment, these two numbers are always conflated into page times. Providing the capability to break them out would be significant win in application measurement.
Units of Work
In my discussion on performance, I touched on “units of work,” the last and, in some ways, the most important of the application measurement framework themes I’d like to see a measurement system address.
When we measure static web sites, we often employ a methodology that we call functionalism. It works by assigning specific functions to each page and then analyzing their performance relative to those functions. It doesn’t work well for applications, however. The functional layer in an application isn’t page based at all.
I’d like to see this functional concept extended directly into the measurement framework. The idea would be to encapsulate all of the major application use-cases as units of work. The framework should make it easy for the developer to kick-off a “unit-of-work” and, as with performance, control the interactions that can occur (in other words, to be able to write logic that determines what to do when one unit of work interrupts another or takes place inside it).
Instead of page-based tracking, I think the basic level of tracking in an application should be at the unit-of-work level. Functions inside the unit-of-work could be tracked as “steps” or “tasks” depending on whether the unit-of-work was ordered or unordered.How would this work? Going back to our mapping application, a unit-of-work might be triggered when a user does an address search.
This “search unit” would persist as a user pans, zooms, and adds an overlay for hotels, and then clicks on a hotel or two. The unit would end when the user enters a new search or exits the application (and possibly on other actions).
The measurement system would track how often a visitor did each type of unit of work (how often the “search unit” was triggered, how long the user spent in the search unit, how long key actions (like the overlay fill) took to perform, the use-state at each point in the unit or work, and what the visitor did to exit the unit.Summary
I’m not sure that these four concepts (unit of work, performance, rest state, application state) are a complete description of an application measurement framework, but they would go a long way to providing logical, rich, and appropriate measurement to applications. Some can (and should) be done even if you are coding in current measurement systems – though all require significant hacks to do right now. But if Microsoft and Omniture are working to build robust application measurement, creating a measurement framework that embedded these concepts would truly advance the state-of-the-art. In my next post, I’ll take up one of the true pain-points in application measurement – testing.
Most measurement systems now work similarly – they rely on a small javascript program (the tag) that allows a coder to customize the values for certain variables and then call a function that assembles those values, adds a set of environmental variables (like browser and referring site), creates an image request and then executes that image request. All of the custom and environmental variables are passed as parameters in the image request and are decoded by the vendor when the request is logged on their end.
Think about it – all the fuss and muss of tagging – just to make a call to another server and pass a few values. In the world of applications, there are many ways of handling this basic task, by far the most common being to use a web service. Web services are vastly more convenient for the application developer than the current method (which is well suited to static HTML). Indeed, it's hard to imagine a more primitive interface for web communications than passing parameters in image requests.The virtues of web services include the fact that they are directly supported within every significant programming language (including direct support within the programmers development environment), they provide methods that allow for back-and-forth communication and error-handling when necessary, they provide more control over the nature and amount of information passed, and they typically encapsulate the information passing into much more logical and understandable programmatic units.
It’s true that some measurement vendors provide data insertion APIs. Omniture, for example, has one. But the Omniture Data Insertion API is not a real web-service. It’s just a ”post” based mechanism for sending data. It’s might be the right method to select if you’re coding SiteCatalyst measurement in an application right now – but it’s hardly a robust interface.
But to me, the most important part of building a good interface between application system and measurement system isn’t the handshake mechanism, it’s the structure of the programmatic interface.
Here’s some sample SiteCatalyst code that might get attached to a click-handler when an event is firing:
var s=s_gi('rsid');s.linkTrackVars='eVar34,prop8,eVar8,prop9,eVar9,prop32,eVar32,eVar50,events';s.linkTrackEvents='event3';These 13 lines of code create the basic SiteCatalyst measurement object (s), tell SiteCatalyst which variables to pass (linkTrackVars), which events to pass (linkTrackEvents), assign seven different variables and one event, and then call the s.tl function that generates and fires the image request.
Strewing chunks of code like this – even if it’s translated into C# or java - all over your application makes for an incredibly messy situation: it’s hard to document, hard to code, hard to test, and very hard to maintain. And like most poor coding mechanisms, it means that if you want to change measurement systems, you’re faced with a significant re-wiring task.
If this is the wrong way to integrate measurement into an application, what’s the right way?
Well, it isn’t to give the programmer a function call like this:
doPageView(“pagename”, “events=1,3,5”, “evar1=test;evar2=test2”, “prop1=red”);In truth, even this would be better than what we have now (though it’s easy enough to create this sort of wrapper). But this simple function library approach doesn’t create a structure that embeds measurement logically inside the application.
What I’d like to see is a system where you started with a basic measurement object. This object could be instantiated once by the application (though applications could use more than one) and would contain, at minimum, the target account information and the configuration of environmental variables (what needs to be pulled and passed).
Ideally, environmental variables would only be passed once on invocation of the application (since apps aren’t stateless like the web, the challenge of inferring sessions and the necessity for re-passing environmental information with every request doesn’t exist). This would dramatically reduce the amount of information required with each call and the corresponding network bandwidth required.
Underneath this global measurement object would exist one or more measurement objects that inherit the global state and allow for custom mappings of application variables. The custom mappings would let the developer populate measurement variables from object properties (like Name, ID, Value or isVisible), application variables, static values or special function.
In this way, a measurement variable could be populated with the Name Property (for example) of any object to which to which it was attached. If attached to a Form Object, it could automatically create something like a pagename variable for an application by mapping the Form's "name" property to the measurement pagename variable.
The measurement object could also be configured to support any or all of the default or specific methods that objects in the development environment have. In other words, a measurement object could be configured with a specification for “load”, “unload”, “click”, etc. This would allow the developer to automatically fire a measurement call in response to either specific functions (like a search) or general functions (like a form load).
With this approach, the developer can take advantage of built-in UI constructs and also manage a small set of application specific variables that will automatically be translated into the measurement system equivalents whenever an actual measurement call is made.
When I create an object in the interface, I would then attach my measurement object to it. When the object is loaded, any handlers for the measurement object would be fired if present. It would automatically set its values, give the programmer an opportunity for a call-back (which should be rare) and then fire the request.
This is rather abstract, so let me show how this work in a more concrete fashion.I’d start with a measurement object in which I would set the global information about the account (and maybe the vendor) I’m sending information to along with the environmental variables I want to pass on invocation.
Next, I’d create a custom variable object that described the specific variables I want to pass. Let’s suppose that I’m working on a Mapping Application and I want to collect the viewed area (the region/country/city shown in the Map Title), the map view (road, topo, satellite, etc.), the map state (overlays, zoom level), the language of the map, and the visitor identity. I would map each of these application variables to a specific measurement object variable. Next, I’d assign the click-handlers I want the measurement to fire on (I may want it on every re-draw but I might also want it on a more limited set of actions such as onSearch and onMapChange). Finally, I’d attach the measurement object to the underlying UI object.
That’s pretty much it. Once the object is attached, the programmer is done. When the object to which the measurement is attached fires an appropriate handler, the object would inherit its default state, populate the variables (like viewed area, map state, etc.), and then fire the measurement request.
Depending on my application, I might need only one custom measurement object or I might need quite a few. But with this type of system, the measurement objects would be embedded into the underlying GUI objects and their handlers. The measurement layer would deeply in-grained, it would provide inheritance, and it would provide configurability. It would be much, much easier to maintain such a system. The measurement objects themselves would be compact and discrete and they would be directly attached to the objects in which they reside.
Having a measurement integration at this level inside an application development framework would make the development process dramatically easier. Indeed, it would be possible to achieve a very high-level of measurement with a simple default configuration of the measurement object that might simply “fall-out” of a single drag-and-drop of the measurement object into the application. Such a default configuration might capture the object name and title of every form on load and unload, the object name and title of every “play” movie action, and the selected value from any Combo-Box. But it would also provide a high-level of control within the application for measurement at a very fine-grained object level.
As I mentioned in my first post, there really aren’t any serious barriers to building this type of system. There are probably different ways of achieving a deep measurement integration or designing the measurement objects – and I’m not confident that what I’ve suggested is the exactly right answer. But some form of deeply embedded measurement at the object level within the app framework seems both inevitable and right. Having an embedded measurement system won’t solve all the problems in application measurement, but it would be a big step in the right direction.
In my next post, I’ll take up a thornier issue – the right measurement paradigm in a world where page views and link clicks aren’t the ultimate measurement objects!