Analysis into Action

Mobile Measurement Infrastructure and the Travel Sector

2011-12-05T09:55:08-05:00

I was recently interviewed by Ritesh Gupta over at EyeForTravel.com and wanted to share the contents of it with my readers here as well.

Q: As an expert in mobile measurement and enterprise analytics, what do you recommend when it comes to making the most of mobile strategy especially in the travel sector? How should travel companies go about making the most of their investments in this arena?

A: When thinking about mobile strategy in the travel sector, the primary place to look for direction is within your existing site analytics. Start by looking at which specific mobile devices and operating systems are visiting your site - where are they coming from, what are they doing, and what content are they consuming. By understanding the mobile usage of your current site and the areas where users are succeeding or failing, along with what devices they are using, you will be able to inform your mobile product strategy greatly. Understanding what site content or features resonate with mobile users will allow organisations to address the primary need of their site visitors and help direct the mobile strategy. Determining your overall product suitability to a mobile offering and which devices or experiences you need to support will ultimately drive mobile strategy and identify where you need to focus your mobile efforts.

Q: It is highlighted that understanding how customers behave when using mobile devices is a major challenge. What are your observations and how do you think understanding behaviour is significant in assessing the overall measurement of any mobile product strategy and related initiatives?

A: Measuring mobile has historically been very hard to do given the fragmentation of devices, operating systems, and carrier restrictions. The majority of these hurdles have been eliminated over recent years with the advent of the smartphone and mobile application measurement SDKs. Previously, feature phones incapable of executing JavaScript posed a significant measurement problem as they relied heavily on manually (or server) generated image requests making measurement of mobile initiatives and device side experiences difficult. Current smartphones can execute JavaScript and device resident applications making traditional web measurement methods viable for mobile websites and applications. Device and operating system fragmentation still persists complicating measurement deployment and organisations need to focus on their current user technographic to understand which devices they need to support from a measurement perspective in order to ensure measurability.

Understanding mobile user behaviour is critical to evaluating and optimising mobile strategy. After all, you can't improve what you don't measure.

Q: The expectations and demands of smartphone customers are significantly higher than website visitors. What sort of benchmark should travel companies set for themselves when it comes to measurement of mobile web and app-related initiatives vis-à-vis any other component of digital strategy?

A: The mobile user expects immediate satisfaction and demands simple and clean interfaces. It is the nature of the mobile user. They are not 'surfing' (generally) and are attempting to complete a specific task when engaging with a mobile website or application. This can be booking a flight, hotel room, or dinner reservations as well as researching destination highlights but all of these have a singular focus - completing a task. For all intents and purposes, mobile users should be converting on these tasks at higher rates and engaging more frequently than traditional website visitors. Often this is not the case due to complicated checkout flows and of course this varies with the utility and appropriateness of the mobile experience. Organisations should focus mobile benchmarks at or above their traditional website rates and optimise the user experience and conversion flow of their mobile initiatives accordingly.

Q: How do you assess the maturity level of mobile application measurement framework at this juncture? What should travel companies learn when it comes to measurement?

A: Historically, mobile applications were developed and released without any inherit device side measurement and the primary metric was downloads. While downloads are an important metric, we have come to realise that user engagement along with recency, frequency, intensity and duration are more actionable metrics providing a holistic view of application success. Mobile application measurement has improved greatly over the last two years and releasing a mobile application today without measurement is heresy. Niche mobile application analytics vendors held a lock on this market until recently and in some cases still provide a superior solution, however all enterprise analytics vendors currently offer robust SDKs. The SDKs allow for the ability to measure all device side interactions (even when the device is offline) and by instrumenting your application with the appropriate level of tracking to capture metrics such as install date, app launches, daily usage, and key event success organisations will gain a greater understanding of actual application usage not just application downloads.

Q: Considering the varied utility and engagement level of mobile web and mobile apps, how should one approach measurement for both? What should one avoid in order to have unjustified expectations?

A: Mobile web measurement should align with fixed web measurement wherever possible. Existing measurement frameworks and implementation methodologies translate well to the mobile web environment and should be leveraged wherever possible as to not reinvent or double efforts. Technology frameworks aside, the mobile website experience is inherently different than the fixed web experience and organisations should avoid a wholesale migration of all of their content or site functionality to their mobile web experience. Screen size and input methods weigh heavily in mobile product strategy decisions and determining which elements and in what format to display these elements is crucial to mobile website success and ultimately mobile website usability.

Conversely, mobile application measurement shares little with traditional fixed web measurement and only the high level success frameworks and key metrics will translate well to this medium. Aside from the technical differences in implementation from mobile web measurement, application measurement focuses primarily on a subset of achievable tasks. Organisations should avoid porting all website content and functionality into their mobile application and focus on the key aspect or element of their product offering that would make the user want to engage with them in a mobile environment. Simple, direct, and facile task resolution should be the primary component of a mobile application.

Q: What do you make of the “Web Versus Application” debate in the mobile product strategy at this juncture? Would it be right to say that the pros and cons settled down now?

A: Mobile websites and mobile applications both have their appropriate place within a mobile product strategy - it is not an either/or debate. Ensuring the appropriate user experience is present in each is the true debate. All too often organisations will make a technology decision rather than a strategic decision in the form of "We need an iPhone app" or "We need to run an SMS campaign" without considering the product suitability to a mobile environment or why the customer would want to engage with their brand in the first place. At this point, a mobile website is a must and should be at the top of your mobile strategy list if you don't have one. Once that is in place and you have gleaned actionable insights into how customers are interacting with your product offering then, and only then, should you craft a mobile application strategy.

Q: Where do you foresee mobile application measurement framework headed in the next 12 months?

A: Mobile application measurement needs to become much simpler to implement. A few vendors have succeeded in simplifying the development process by enabling automatic capture of device side events and variables but this needs to improve considerably for continued widespread adoption. Currently developers are expending tremendous effort to instrument measurement in their mobile applications significantly increasing time to market for new applications - this needs to change. Additionally, location based services are changing the way we visualise data once collected from mobile applications. Given the fixed nature of traditional web browsing this element has not had much attention until now. Being able to see where and when a customer is interacting with your mobile application creates a whole new dimension to the traditional marketing strategy. It is no longer about getting the right offer to the right person at the right time - but about getting it to them at the right "place".

Q: What do you make of the mobile product strategies especially the usage of mobile apps in the travel sector?

A: The hospitality and travel industries have embraced mobile fairly well and have incorporated mobile strategies into their product marketing and service offerings adequately. However, some of the most innovative mobile product strategies I have seen recently in the travel sector have originated from capitalising on the "location aware" aspect of modern smartphones.

For example, Vail Resorts, using RFID technology, is able to tell when their customers are "on" or "off" mountain and provide appropriate messaging and content to them along with social network interaction opportunities. By capturing this location based data they are able to expand and optimise the on mountain experience for their customers. The ability to track your progress, vertical ascent, capture photos, share achievements with your social network, and have all this in your pocket is truly amazing!

Get SMART with Mobile Applications - Webinar Redux

2011-11-15T09:00:00-05:00

A few weeks back I presented a webinar along with Brian Suthoff from Localytics. We delivered a mobile application measurement framework based on my SMART methodology and I wanted to follow up on the Q&A portion of the webinar. We unfortunately didn't have enough time to address each question during the webinar...

For those who did not attend, you can watch the session and/or download the slides here.

Q: How can I do A/B testing?
Q: If I instrument my iPad app with Google Analytics, and want to do A/B testing, is there a way to use Google Website Optimizer as my A/B testing platform?
Q: For analytics & optimization, A/B testing is essential. Please describe my options. Thank you.

A: Yes, you can build A/B testing logic into your app. For example, you can present different content or options to different users, and tag an event for the path (or option) they saw. Then in Localytics you create funnels that start with that event and end at your goal to easily compare the results of users sent in different directions. and you start the two funnels each with the different event. That way you can compare the results very easily.

Web-style A/B testing in apps is still in it’s infancy, testing and optimization is currently restricted to apps that leverage HTML, CSS, and JavaScript for the majority of their interface (Netflix, for example). By developing different UIs and assigning them to device IDs at time of app installation and activation (persisting them, of course) it is possible to measure user behaviour against specified KPIs by app UI to determine relative lift.

Q: If my app sends people to someone else's website - where the purchase conversion takes place - what are my options for measuring / tracking that conversion?

A: Apps are sandboxed from each other and from the web, so many of the conversion tracking techniques used online are not available. In this case, you could pass referring information (e.g., app ID, Localytics ID, campaign ID) to the website where code on the website would then record the referrer and capture the conversion. Essentially, it is up to the destination web site where the conversion takes place to pass on conversion metrics. Of course, if they will let you place your own tracking codes on their conversion pages - all the better...

Q: Device collects data in chunks and transmits when device becomes online to Localytics data collection servers. Is that a correct assumption?

A: Yes that's correct. During app usage the Localytics client library will cache all event data in local storage, even if the app is online. Then when scheduled by the app developer, but typically at the beginning and end of a session, or as part of a background service, the usage data will be compressed and uploaded to Localytics.

This is true for most vendor SDKs in that some data is sent upon app launch but it is up to the developer to instrument their app to send data at appropriate times during usage depending on current app processes. Best practice dictates that it is not prudent to send data to the collection servers upon each and every action but to batch requests to avoid overtaxing the devices data connection.

While not true batching of requests, Adobe has the ability to force the data collector ‘offline’ during processor intensive app routines and then return it to ‘online’ mode when complete. Additionally, using the offlineThrottleDelay function allows the developer to set the cadence (or delay) when sending requests to mitigate performance impacts once the application is online again.

Google SDKs allow for the batching of requests using the ‘dispatch’ method. It should be noted that with the Google SDK requests are set with a timestamp of when the app sends data to Google NOT when the action occurs.

Q: The fact that every big corporation is jumping into the app sea is not necessarily a measure of success. Are those apps really successful?

A: It certainly varies by company, just as not all websites are successful. How success is measured will also vary. But to offer an anecdotal example: Rue La La announced at Shop.org that 22% of their sales now come from mobile, which feels like a pretty great success. Also, eBay expects to see $5B in mobile revenue in 2011.

Q: How should Mobile Strategy be approached for applications that have already been launched with no or limited measurement

A: Fortunately, users upgrade the apps on their devices very quickly, so adding analytics to the next version of an already launched app is an easy first step. Beyond the basic integration and metrics, you'll also want to think about how your goals and questions translate into event tags for your app. There are many tips and suggestions on the Localytics blog and either Localytics or Semphonic can share additional best practices or consulting services to quickly get you started.

Q: So you are saying that the main difference with mobile analytics compared to traditional online analytics is the implementation?
Q: What is really different from mobile analytics compared to regular online analytics? Is there really a difference?

A: Implementation differences are more effect than cause. The biggest differences are that apps and websites are both built and used very differently. Web analytics can safely assume that users are online (or they couldn't reach the site) and that everything is received in chronological order at the time it happens. Web analytics are also more server-based and were not built to take advantage of PC/device APIs.

Apps may function completely offline, or suddenly lose connectivity, which dramatically changes how the implementation must be done and how data are cached, timestamped, uploaded, de-duplicated, etc. As mentioned on the call, Adobe SiteCatalyst report suites must be configured to support device timestamps, which then affects the rest of your SiteCatalyst deployment should you wish to rollup these report suites. Google analytics assumes that everything happens when it's reported, which is hardly true when 15% to 25% of app usage happens while offline.

Other differences include web time-outs vs app multitasking, location based on source IP or device API, power and network conservation, etc.

Q: Do you have any stats regarding the mobile Canadian market? How much different do you think it is compared to the US market?

A: Canada is experiencing rapid mobile growth, however current adoption is far less than that in the US due to carriers not supporting the broad extent of smartphones found here in the US. This is changing rapidly, however and it is predicted that more than 425 million smarphones and tablets will be sold in 2011. Deloitte has some predictions on the Canadian mobile market here - http://bit.ly/uimnia

Q: To the best practice of separate profiles per channel: what about an iPad user hitting our regular fixed web site, should that go to a dedicated iPad suite or is it OK if that mixes with our desktop/fixed web suite?

A: Regular websites tend to work well on iPads (non-Flash of course!). Depending on your business, you may still want to have a version of your site optimized for iPads or build a cross-platform site with dynamic layouts (the new Boston Globe site is a great example).

It is recommended that you create separate profiles for your various channels so as not to conflate differing device types and/or platforms. Having separate profiles for fixed web, mobile web, and mobile apps is the preferred method and again this is based on device type NOT site type. So in response to the question, it is perfectly acceptable to have iPad traffic to your fixed web site be included within your existing web site profile.

Tracking QR Code Performance

2011-01-07T12:05:44-05:00

Recently, a colleague of mine Allison Hartsoe asked if I had any information related to tracking the performance of QR Codes and I figured that this would be a good opportunity to build upon some previous campaign and social media tracking tips provided by June Dershewitz. June provided some great tips on Tracking Social Media Links as Campaigns as well as URL Shorteners with Analytics that dovetail nicely with tracking the performance of QR Codes. But first a little background.

Background

Mobile barcodes can take on many different shapes, sizes, and colors owning to the limiting factors holding this technology back. QR Codes are rapidly becoming the de-facto standard, but there are many proprietary solutions and several competing standards which has made adoption somewhat slow, especially in the US. Wikipedia details several of these formats:

Essentially, QR Codes (Quick Response Codes):

Refers to the use of 2D barcode images read via a mobile phone for quick access to stored content
Utilize a reader application to scan the image of the QR Code to display text, contact information, connect to a wireless network, or open a web page in the phone’s browser
Limiting factors include interoperability, proprietary barcode technologies and a lack of standards

Readers

Thanks to our friends at PercentMobile, you can get a QR code reader for your phone by visiting http://percentmobile.com/getqr on your mobile device. Or to see a listing by device, visit http://analytics.percentmobile.com/device_qr_code_readers/.

My favorite reader application currently is QuickMark. It can handle many different mobile barcode formats and is super fast at isolating and reading the barcode image.

Tracking

Tracking QR codes falls into two categories:

1) Fully contained systems – Utilizing a mobile barcode generation service you can generate and deploy a mobile barcode and then report on performance within the vendors proprietary system. A short list of vendors include:

2) Manual or Redirect based – By creating your mobile barcode manually and using standard campaign tracking, you can effectively track the performance of your mobile barcode campaigns by leveraging your existing Web analytics vendor.

First create your mobile barcode using any number of free tools:

Delivr

Quirify

Mobile-Barcodes.com

Google Chart API

If you are linking directly to a page within your site that already has tracking on it - you can simply append a campaign code to the URL you are using as the target for your mobile barcode.

If you are linking to content NOT on your own web site you will need to route through a 301 redirect page with your analytics tracking code embedded on the redirect page. Check with your analytics vendor and see if they have a preferred method for redirects.

Then there is the Easy way...

PercentMobile is offering QR Code Tracking for Everyone, currently available by request.

-OR-

bit.ly and goo.gl allow for QR creation and tracking of shortened URLs created on their system. By adding .qr to the end of any shortened link you are taken to a QR code for that link. Simply copy the image and you are good to go!

You can leverage each tools respective tracking system for analytics - but this will only provide basic metrics such as clicks, referrers, and location. However, if you follow June Dershewitz’s suggestions with respect to tracking your social media campaigns and be sure to embed a campaign code within your shortened URLs, you will have visibility into post click behaviors and conversions.

For example, here is the QR Code for this blog post as trafficked through bit.ly with Google Analytics campaign codes. Fire up your QR Code reader and snap the image below to check out the resulting URL.

Please leave a comment below as I would love to hear how you are leveraging QR Codes in your marketing efforts, if you are tracking them, and how they are performing.

Mobile Strategy First - Tactics Second

2010-04-09T13:32:23-04:00

I attended the ThinkMobile conference sponsored by MediaBistro in NYC this week and was reminded how important Strategy, with a 'big S', is to the success of a mobile product offering. A lot of the conversation over the two day event centered on application development and the technical considerations around what platform or mobile operating system to code for. Invariably the iPhone and the iPad had center stage in most of the conversations, but all were cautioned not to undervalue a proper mobile Web site (when will we stop calling these WAP sites?).

Technical considerations like this are critical in ensuring the success of your mobile product offering, but are definitely not the first thoughts that should cross your mind. It's a given that this is the fun stuff to talk about and it makes for good conference fodder (who doesn't like a good OS/platform war) but this takes away from the real question organizations should be asking themselves - what is our mobile strategy?

All too often organizations take a backwards approach to developing a mobile product offering and begin with a technology decision rather than a strategic plan. Statements like "We need an iPhone app" or "Let's do something with SMS" lead to siloed approaches and marketing fragmentation. Success in mobile demands a systematic approach that begins with understanding your customers mobile usage, determining your product suitability to a mobile offering, defining your business objectives, and evaluating your level of commitment. Only once all of these steps are completed should you begin to implement the necessary technologies to achieve your mobile objectives.

In response to the need for a systematic approach to mobile strategy and measurement, Semphonic has put together the SMART Mobile Program.

Strategy - Determine your ‘Mobile Readiness’
Measurement - Define a mobile ‘Measurement Plan’
Analysis - Establish intelligent mobile ‘Correlations’
Reporting - Make strategic information ‘Obvious’
Tactics - Generate ‘Actionable Insights’

The starting point in the SMART Mobile Program is obviously Strategy - with a 'big S'. By defining an overarching mobile strategy informed by primary and secondary research, competitive evaluation, and market intelligence, organizations can establish their 'Mobile Readiness' before jumping headlong into technology and enablement decisions.

'Mobile Readiness' is assessed by thoroughly and systematically evaluating the mobile opportunities across your online properties and determining the necessary steps required to establish or optimize your mobile product offering. Good measurement is the data-driven foundation of any strategy, as such, a deep audit and analysis of your existing measurement infrastructure will assist in determining overall product suitability for a mobile product offering. Looking at existing mobile users on your site and their level of activity and engagement across site sections and experiences will provide insight as to where you should (and should not) focus your mobile attention. Analysis should be conducted across various dimensions including mobile device type, mobile visits, and mobile engagement to identify key sites or experiences that would represent significant opportunities for mobile product development.

Don't get caught up with tactical questions like, "Which mobile platforms should we support?" or "Do we need an iPhone app and mobile web site, too?" in the beginning but instead focus on Strategy first. With a well defined Strategy, organizations can be better equipped to handle the myriad of challenges and avoid the tactical execution pitfalls that mobile presents.

Mobile + Social = an Olympic Sized Success

2010-03-26T15:43:27-04:00

While it has been some time since I pulled the NBC Olympics application off of my iPhone, I think the recent flurry of activity and commentary around the mobile social media space bears some relevance to the tremendous success NBC had with their Olympic endeavors. By all accounts the 2010 WInter Olympics were a huge success for NBC. It's great to see a big media brand delivering on the tenants of a dedicated mobile strategy. NBC leveraged Olympic mobile data and insights collected from the 2008 Beijing games to shape the mobile strategy for the 2010 Winter Games. It was obvious to NBC that mobile would be a huge opportunity for them, and they had put a formal mobile strategy in place, but I don't think they were ready for their Olympic sized success.

Some stats:

87 million mobile pageviews for NBC's Olympics coverage this time out versus 35 million pageviews for Beijing.
2 million mobile video streams versus 300,000 for Beijing.
7 out of 10 users who are now employing mobile to watch the Olympics didn’t use their phones to view the Beijing games.

While the growth in mobile traffic for NBC was not much of a surprise given the meteoric rise in mobile web browsing along with the proliferation of mobile applications, the social media engagement opportunity was. Statistics from comScore around mobile subscribers and their use of social media sites hadn't come out yet. I'm sure that NBC had a hunch - but I'd like to think it was a formal strategy decision on their part.

I firmly believe that in order to have success with a mobile application you need to offer the user continued engagement opportunities. Give them a reason to open the application again. Not just download and open once as so many applications are. Earlier this year Apple announced that over 3 billion applications had been downloaded and that there were well over 100,000 applications in the iTunes App Store - pretty impressive. But when you look deeper at the level of engagement with these apps the picture is not so rosy. There is a steep drop in engagement after the initial download with a majority of these applications. I mean really, how long can you press a fart button for or wave around a light saber before it simply becomes annoying? Only a small percentage of applications actually convert users into long term customers. You need to bring the social media aspect into your offering in order to engage users for the long haul. That's exactly what NBC did.

Although the app did not receive the best of praise among several pundits, and there were some definite mobile search issues, not one of the reviewers touted the social media aspects of the offering. The ability to tie into the athletes Facebook and Twitter streams was fun and highly engaging. Although some of the 'open social' commentary was a bit off color at times, this was segregated to its own tab so it didn't mix with the proper athlete's voice. I enjoyed checking in on the various athletes I was following (another feature in the app) and getting updates from Apollo Ohno, Lindsey Vonn, and Shaun White before and after their respective events. Their pre- and post- event ramblings were what kept me coming back - along with videos and medal counts.

Having a defined mobile strategy is a prerequisite, but engagement is the key to mobile application success - not an earth shattering revelation, but hardly the standard that most application developers are achieving.

A New Beginning

2010-01-11T07:00:00-05:00

Life is full of new beginnings - a new year, a new love, a new baby, a new job, a new challenge. I have been blessed to have experienced all of these at one time or another and now I stand on the brink of something great - an opportunity to make a new beginning with Semphonic.

Most of my career has been dedicated to making the complicated facile. Be it technology, science, gadgets, or gizmos I have been lucky enough to be in positions where comprehending and communicating the nuances of complex systems have been my primary focus. This is also true for my new role at Semphonic.

Gary Angel and the team at Semphonic have presented me with many exciting challenges for the coming new year. Primarily, I will establish, build, and grow a new Semphonic office in New York. No simple task, but one that I will embrace with the same fervor I did some fifteen years ago when I founded the New York operations of DataMap Technology, an Internet start-up providing environmental maps and data to the financial and consulting sectors. Picture a Google Maps interface, however, instead of hotels and restaurants, we populated the base map with distance and direction to hazardous waste sites and chemical spills – good times.

Secondly, I am joining the executive management team and will be called upon to provide an additional voice to the chorus of thought leadership Semphonic is so well known for. No small errand to be sure. I have been part of highly technical, motivated and effective management teams before and I hope to bring the same inspirational influence to the team at Semphonic. Having spent the last week out in Novato with the management team participating in the 2010 planning sessions, I am energized and impressed by the caliber of talent and level of thought leadership Gary has been able to amass at Semphonic.

Last, but certainly not least, I have been tasked with developing Semphonic’s mobile strategy and measurement product offering. A challenge I am truly enthused about and one that sits squarely within my area of expertise. Having spent the last two years at Nokia working through the measurement challenges inherent in the mobile space, while simultaneously developing an enterprise-wide measurement standard, I am looking forward to leading Semphonic’s clients through the murky waters of mobile measurement as well as providing insight as to the what, how, and why of their mobile strategy. Most of all, I feel confident that I can provide immediate value and significant advantage to all of Semphonic’s clients.

A full set of challenges to be sure.

A great new beginning – most definitely!