A different service fully and solely subsidized by PreEmptive Solutions will take its place. This service is materially different – please read the following notice carefully for information on how to continue to work with PreEmptive Solutions technology for Windows Phone.

Background

While Microsoft’s sponsorship expired on September 30, 2011, PreEmptive continued the service for an additional 60 days at our own expense while we explored a variety of options to continue our support for the Windows Phone development community. Microsoft’s sponsored service has ended, but our commitment and support for this community continues unabated.

PreEmptive’s challenge was to find an affordable means to support for the burgeoning WP7 development community without compromising the quality and capabilities unique to our protection and application analytics technologies; we believe the following provides both a valuable set of services at little and no cost for small WP7 development efforts with a smooth “on-ramp” for larger development projects and for organizations with more demanding service levels, governance or scalability requirements.

Summary

Obfuscation and Instrumentation continues at no cost through 12/31/2012.

Dotfuscator for Windows Phone, the post-compile tool that obfuscates and injects application instrumentation will continue to be offered to Windows Phone 7 developers at no cost through December 31, 2012.

Mobile analytics endpoint (wp7.runtimeintelligence.com) will be shut-off on 12/9/2011

The current analytics endpoint will be discontinued. However, developers have a number of options that they can consider;

· Subscribe to the PreEmptive Solutions commercial endpoint. This is a fee-based option that includes all of the features currently offered PLUS an advanced mobile portal, a RESTful API, and a higher service level (plus support beyond WP7). For more information, email sales@preemptive.com.

· License PreEmptive Analytics for TFS. This is also a fee-based option. This solution is an on-premises solution focused on exceptions rather than feature tracking. For more information, see Using Analytics for Windows Phone and Azure Exception Tracking - User Community Virtual Series and email sales@preemptive.com.

· Develop and host a homegrown endpoint. This is a no-fee option but development will be required. The CodePlex Runtime Intelligence Endpoint Starter Kit repository starter kit project may be of some help.

· Plan to migrate to the PreEmptive Analytics for TFS community edition to be included with Dev-11. This is a no-fee option but is NOT yet generally available from Microsoft. For more information, see the video A Lap Around PreEmptive Analytics for TFS with Justin Marks .

· Publish their app as a CodePlex project and utilize the CodePlex analytics endpoint (this is different than the option above). This is a no-fee option. For more information, see this tutorial (note that this assumes the developer is limited to the Community Edition of Dotfuscator – but the WP7 edition has full functionality).

The following feature summary table highlights the three principle options available to the Windows Phone 7 development community with a comparison to the discontinued Microsoft sponsored service. (click thumbnail to enlarge)

Click to enlarge

FAQ

Q: Will developers have to republish my WP7 app on or before December 9, 2011?

No

Q: Will users notice any difference in app behavior after December 9, 2011?

No

Q: Will I have to re-register my installation of Dotfuscator for Windows Phone 7?

No

Q: If I have only been using Dotfuscator for obfuscation, will I lose any functionality or will I have to do anything differently?

No

Q: Will developers have access to earlier runtime data generated by Runtime Intelligence for Windows Phone after December 9, 2011?

They will not.

Q: Where can developers ask additional questions regarding migration, upgrades or discontinuing use of PreEmptive Solutions technologies?

Post to the PreEmptive forum at http://www.preemptive.com/forum/index.php?f=26&sid=dfe90c2ba80de07692372dae962c58b2&rb_v=viewforum

PLEASE NOTE – this is NOT a moderated forum.

Q: Why would a development organization upgrade to a professional SKU of PreEmptive Analytics?

· Multi-platform (WP7, Android, JavaScript, all .NET and Java, native API…)

· Private endpoint (for large-scale enterprises with demanding scalability, governance, or other unique requirements).

· Analytics for TFS option (out-of-the-box integration with Microsoft Team Foundation Server).

· True application analytics such as custom data fields, development ownership of data, true SLA and support for developers, etc.

Conclusion

Of course we would have preferred that Microsoft had opted to extend their sponsorship for Runtime Intelligence for Windows Phone, but that was not the decision that they ultimately made. However, over the past 12 months, we had a front row seat watching a flood of innovative apps launch. We know that a good percentage of the WP7 development community relied upon PreEmptive Solutions for both analytics and protection (in a recent analysis of the marketplace, it was shown that 17% of all apps used either protection, analytics or both).

This experience combined with our confidence in the future of the Windows Phone platform has prompted us to extend free access to Dotfuscator for Windows Phone. We look forward to our continued support and participation in the growth and success of this exciting technology and marketplace.

First, let me start by stating categorically that we remain excited and committed to the Windows Phone platform.

Our approach to analytics has always been squarely focused on development organizations rather than on marketing; this is not a “spin,” rather, this approach informs and influences our product’s architecture and business model.

I’ll drill into this distinction in a moment, but first, I want to point out that it is this distinction coupled with our belief in WP7 as a platform and the value of our approach to application analytics in general that led us to invest in building out the WP7-specific service. While I cannot speak to Microsoft’s motivations (in any context, let alone this one), it is public knowledge that they funded this service so that it could be offered at no charge to the WP7 dev community (but now that’s no longer the case).

Why was this even necessary? Why couldn’t PreEmptive just give away our analytics and protection service like Google does?

“Marketing-centric” analytic services make their money off of your data – but first they have to cover their costs; the “back end” or analytics portal is where the majority of that “cost” sits. Today, Google or Flurry (or any other similar service) each rely upon web and other mobile platforms to generate the data volume that ultimately funds their backend service. At this early stage, WP7 app data can fund client API’s (perhaps) but is nowhere near the volume-levels required to pay for an entire service.

But there’s more – focusing exclusively on resalable content (and by extension, divesting from any requirements that do not) have profound feature and architecture implications.

Given a marketing-centric focus, you can readily understand why these solutions all share the following traits:

· Custom data fields and data types are limited in scope and volume(unique/heterogeneous data points cannot be easily rolled-up, sold or used to better target users)

· No built-in enforcement for opt-out policies – this is left entirely to the developer (if you do not provide user data to these services, you are of no value)

· No investment in analytics of software away from the presentation layer(this data has no relevance to advertising, profiling etc.)

· Software-specific data is only marginally supported – unhandled exceptions can be captured, caught and thrown exceptions are out of scope *same as above – does not contribute to monetization model.

· They own your data (privacy rules notwithstanding – the have full rights to monetize your data in any way they see fit). This is the entire reason they are in business – if they don’t own the data, how can they make money?

· They can provide their analytics software for free – since they have built a monetization strategy based on your data and do not invest in any development that does not directly contribute to that strategy, they can afford to give you the picks and shovels for free in exchange for the gold you produce for them (that’s big of them don’t you think?)

Conversely, focusing on application stakeholders as we do, we invest in features that do not contribute to a roll-up across companies and apps to monetize user preferences – in fact, some of these features can materially impede that objective – yet they are equally as important for developers who want to build better, faster and more effective software…

For example:

· Supporting custom data (app-specific fields and states) and data types (other than strings)

· Built-in exception tracking (unhandled, caught and thrown) to not only send you stack traces but provide insight into user experience and runtime environments (hostile or otherwise)

· Built-in opt-out policy enforcement (local, regional, cultural, legal and regulatory rules are so complex – serious development organizations need to centrally manage and reliably enforce compliance – could you sell a gun without a safety?!)

· Analytics away from the presentation layer (want to know how your Azure or other distributed services are performing and behaving?)

· AND LAST BUT NOT LEAST, you own your data. (given the heterogeneity and specificity of each application’s data – it’s value is exponentially higher to development stakeholders and proportionately lower in aggregate)

So what’s the takeaway?

With Microsoft, we invested in this mobile platform because we believed (and now we know) that it has significant value.

Given our focus on development rather than advertising, we have to innovate on both the technical and business ends.

We had hoped that our previous relationship with MSFT Windows Phone division had lasted a bit longer, but it is what it is.

We have a number of very interesting scenarios in mind that we think will prove to be both exciting technically and innovative from a business perspective – but we are not ready to discuss that yet (as soon as we can – we will).

If you don’t care about these added value features and don’t mind the strings that come with web/mobile analytics services like Google – then you should certainly use them. They are not flawed – they are just fundamentally different.

If you value the services we have been offering or have ideas on how we can improve them even further – PLEASE LET US KNOW…

Registered users will be getting a survey in the next 24-48 hours – please let us know what you think and what’s working (and not) for you.

As I’ve already said, we remain excited and committed to the Windows Phone platform.

(now i have to break to prepare my wp7 training materials; i’ve got a crew from my son’s high school writing WP7 apps for independent study credit)

Well, there’s really only one way to answer these questions …and that’s to ASK THEM.

I sent out an electronic survey starting on Monday of last week (September 19) and have received over 200 developer responses. Here is what they said…

Who is doing what?

• 32% indicated that they are only using Dotfuscator to protect their application
• 24.5% said they were only using Runtime Intelligence
• And 43.4% indicated that they were using both.

Analytics’ perceived value increases by 450% with developer experience

When looking at those developers that indicated that analytics and/or protection “set the WP7 development platform apart from all others,” analytics’ value actually increased by 450% (from 2% to 9%) as developers moved from no app, to less than four weeks to a ship date, to actually having an app in the marketplace (and getting analytics back). Interestingly, obfuscation (protection) peaked in value just prior to shipping.

So what’s the takeaway?

In my last post, we established the user of Runtime Intelligence were more successful than other WP7 developers. In this post, we see that these developers credit their success, to some material degree, to either Runtime Intelligence or Dotfuscator protection (or both).

The short answer appears to be yes – to be clear, I am NOT saying that merely turning Runtime Intelligence on increases an app’s success. What I think the following data does show is that the developers who chose to include Runtime Intelligence as a component of their development process are indeed more successful than those that did not.

I took a second look at the 26,469 apps that we downloaded from the marketplace and compared those apps that were instrumented with Runtime Intelligence with those that were not.

Specifically, apps with Runtime Intelligence (analytics) are:

Ranked 25% higher: RI-enabled apps averaged roughly 4 stars (8.02 out of 10). Non-RI apps came in at 3 ½ stars (7.6 out of 10). NB: apps with no ratings whatsoever were excluded from this calculation.

Have a 50% higher rate of engagement: 73% of the apps using RI had at least one rating while only 48% of non-RI apps had any rating whatsoever.

As I’ve already said, simply turning on RI will not get you a 1/2 star bump in your ratings - but clearly, the kinds of developers who are achieving higher user satisfaction are the kinds of developers who are choosing to use RI.

Coming next: Users of RI and Dotfuscator for WP7 make themselves heard!

You see, we don’t just obfuscate – we hide the fact that your app is obfuscated. We don’t just offer application instrumentation and monitoring, we inject that logic to simplify and streamline packaging and improve performance. …and therein lies the rub.

Something was in the air

While at Build soaking up the heady atmosphere of Windows 8 and all that’s coming with it, a few MVP’s and also the Lowdermilk brothers took a moment to ask me if I’d read this awesome blog post where a developer had downloaded and analyzed all of the XAPs in WP7 marketplace. Apparently the market insight was killer and covered everything from the most popular libraries to snagging cloned apps. No more guessing – the facts were all laid out. And they each mentioned to me how surprised and even a little worried they were by how few apps were being obfuscated. I didn’t think too much of it at the time as I am among the first to point out that not every app needs to be obfuscated – but I did make a mental note to be sure to check out the blog.

When I got back home, I finally had a chance to track this “all seeing blog” down – it was Justin Angel’s blog and the post was Windows Phone 7 Marketplace Statistics. And it really is a fascinating post with both initiative and insight; and then I got to the obfuscation section.

According to Justin’s analysis, only 3% of the apps in the market were obfuscated! And as I scanned down, there were comments to the effect of “gee, since no one else is doing it, perhaps I shouldn’t bother either.”

Even more surprising to me was the fact that our analytics (Runtime Intelligence) was not even listed in a very long list of third party tools – when I knew for a fact that we have nearly a thousand apps sending data.

This can’t be right! (and I was)

Given the nearly 6,000 downloads of Runtime Intelligence and Dotfuscator for WP7 and the activity that I had been seeing over the past year, these numbers just didn’t seem right. I wrote to Justin who was quick to share his detection logic (in fact he posted the source on his blog) and just as quick to invite any comments or refinements that I might have to offer.

To put a fine point on this, Justin was in no way defensive and was as interested in getting to the right answer as I was.

Hung by our own petard

As interesting as the static analysis of the WP7 marketplace is (and it is), static analysis only gives us a backwards facing snapshot of what’s already been deployed. We get no insight into:

• Best practices that we would want to replicate (which are different that common practices),
• Developer motivations behind their development choices
• Future trends especially when driven by new technology and market opportunities

• If the developers who built the 14.5% of “Dotfuscator processed” apps leaders or laggards?
• Do they have special requirements that set them apart?
• In short, do they have anything to teach the rest of us?

As always, we deeply value your feedback. You can directly reach the Runtime Intelligence portal dev team by leaving a comment on this blog entry, posting a message on the Commercial RI Portal forum, or tweeting us @PreEmptive.

Like anyone else who has an app inside the WP7 App Marketplace, I noticed that the App Hub was down most of yesterday with the promise of a functional upgrade in the works – and today I was very pleasantly surprised to see the result; a streamlined experience with expanded capabilities.

One of the first things that caught my attention was the exception reporting by app and by date; very useful indeed. Of course, MSFT is quick to point out that (and I quote) “Crash count alone isn’t a direct measure of app quality. Popular apps may have higher crash counts due to higher usage.”

Well that seems self-evident, but without usage metrics how can I evaluate the severity of my exception report counts? …. (and now, unless this is the first post of mine that you have ever read, you must know what’s coming).

To the cloud! (Sorry, I couldn’t resist). Using Runtime Intelligence for Windows Phone, I’m able to measure total sessions – by extracting these counts by day and mashing it up with exception counts from the marketplace – I can now supply the missing ingredient to make the exception count on the App Hub meaningful. (NOTE – I had to manually transcribe exception counts from the App Hub as there is no tabular option and the detailed download drops the daily count as it de-dupes the exceptions).

The App Hub is careful to point out that only apps running NODO (or Mango) can report exceptions, so I first had to remove the Runtime Intelligence session data coming from earlier versions of WP7 (an interesting statistic on its own).

Here is what I see… (and a warning here – the numbers aren’t pretty)

I took two apps of mine; Yoga-pedia and A Pose for That and looked at their respective usage on NODO+ phones via Runtime Intelligence and exception reports from the App Hub and then calculated the ratio of sessions to exceptions.

The time period I used for this test was the two weeks from June 12 to June 25. During that time, this is what I observed:

• 66% of A Pose for That sessions were run on NODO.
• 58% of Yoga-pedia sessions were run on NODO.

Here is the ratio of exceptions reported by MSFT and sessions from Runtime Intelligence… (click to enlarge)

Ratio of session counts and exception counts by day

Now there are three likely scenarios here.

1. Over this two week period, both apps were crashing every 1 in 10 times they were run (HORRIBLE). I don’t think this is the case because I have run these apps myself on multiple phones hundreds of times and they have NEVER crashed.
2. The App Hub is over-reporting exceptions (or somehow incorrectly associating exceptions with these apps). This is a beta feature on the App Hub – it’s certainly possible.
3. Runtime Intelligence is way under-reporting the total number of sessions in a given day. Certainly possible, but given the unit testing I have done, I don’t see this as being a major contributing factor to these ratios – but certainly a possibility.

Now, I had already put a “feature tick” on the default unhandled exception handler to count how many times it was invoked during this same period. The counts I have are well below the App Hub numbers (which might suggest number 2 above is the culprit – BUT NOT SO FAST). It is more than likely that certain exceptions (perhaps a majority) would interrupt the normal feature tracking transmission mechanism so I would expect that count from Runtime Intelligence to be artificially LOW.

As is often the case when managing an application “in the wild”, an unanticipated question has arisen and I find that I don’t have enough data. That’s why its ALWAYS so important to

• plan in advance what data is worth collecting to minimize the likelihood that you will end up in this situation and
• be sure that your analytic solution supports rapid and easy iterations and refinements to compensate for when your planning falls short.

So how am I going to determine if

1. my apps offer a LOUSY customer experience everywhere except for my personal phones or
2. one or both exception reporting counts and session tracking counts are flawed?

Easy - I’m going to post an update of my apps to the marketplace this weekend with Runtime Intelligence Exception reporting turned on. What?

Runtime Intelligence for Windows Phone includes its own exception tracking capabilities – it does require that the developer activate it (that’s why I don’t have that data now), but it offers a lot more data and it can be invoked for unhandled, handled, and thrown exceptions. Further, it can be configured to collect additional information (custom for the app), AND it can be extended to offer the user a dialogue to provide additional feedback if they like.

I will post my results over the next few weeks – meanwhile, if anyone has any suggestions or ideas – please let me know… I honestly have no idea how this little mystery will play itself out.

Before I sign off – here is one more tantalizing clue (although it may also be a red herring). When I look at the limited unhandled exception data currently being returned by Runtime Intelligence (I can see tower location, device manufacturer, OS, etc.), I see that well over 50% of the phones that had an exception were localized to a language OTHER THAN en-US – and that is way out of proportion to the actual usage trends that I have been tracking (and posted in earlier entries). Further, the localizations that had the greatest “disproportionate” number of unhandled exceptions were de-DE and de-AT. Coincidence? Conspiracy? We don’t need to guess – we will soon have the facts!

PS here are two links that may be of interest:

• A developer survey (PDF) we did last year on exception reporting and developer best practices…
• An article in this month’s (July) Visual Studio Magazine on application analytics in the development process.

Enjoy!

Title: Monetize Mobile Apps with Analytics
Registration: Thurs, Aug 4th at 11:30 EDT https://www3.gotomeeting.com/register/676857398
Registration: Wed, Aug 17th at 12:30 EDT https://www3.gotomeeting.com/register/433412582

Description: In this 60 minute webinar, we will take a live WP7 app and use real-world analytics to illustrate:

1. The impact of try/buy scenarios on paid apps
2. The relationship between free and paid versions of an app
3. Strategies for ad-driven app design that consider page location, first time, occasional, and power user patterns, cultural trends, and other demographics including carrier and model profiling.

Preparation: NONE required. However, attendees are likely to get more from the presentation if they have already:
• Installed and are familiar with the Microsoft Windows Phone 7 development tools
• Installed and have some familiarity with PreEmptive Solutions Runtime Intelligence
• Installed and navigated around the free SKU of the sample application that will be referenced in the presentation. The free app is Yoga-pedia.

Impressions by ISP provider (top 25)
To produce the next graph, I used an application to tell me who owned the IP addresses that my mobile clients are using (I used IP2Location – but there are many of them out there).

This is a nice way to see who my users favor in terms of their carrier. Here I only show the top 25.

Advantage: Understanding carrier popularity will help focus business development/marketing efforts and better manage potential risks associated with how your users may be negatively impacted by upgrade schedules (delays). Will your next app be dependent upon Mango?

Sessions per app page
In the raw CSV files that can be exported from the Runtime Intelligence portal, there is a column, ApplicationGroupId. The value in this column is unique for all signals (messages) that are sent from within a single app session. In other words, I can use this field to organize all user activity into the relative user sessions using this field. This is helpful for plotting specific user patterns.

The following graph simply counts the unique occurrences of ApplicationGroupId values by page name value (recall that this is the first parameter of the WhatPoseWhen method). This avoids counting multiple views of a single page within a single session and tells me how popular specific pages are across my user base. For this posting and for illustration, I’m only showing data for five specific pages.

FindAPoseDetail and BrowseSelectPose are central to the user experience (browsing for yoga poses and then drilling into a specific pose for detailed imagery and instruction). TellMeMore is the page where I describe what comes with the paid version of the app (nice to see that 10% of my users deliberately choose to investigate the upgrade possibility) and AppGuide and TopicList are essentially app documentation and I can see that these pages are not hit very often – and that’s not a bad thing – users should not need to use the documentation after their first use.

The graph below shows the average number of ads shown per page and maps them to where they rank in terms of how often the page is visited.

Happily, the two core pages of my app also get the most ads (and are also where my users are stopping to spend time). I can also see that users spend more time on detailed pose descriptions than they do browsing – even though the browse more often than they drill down (which makes perfect sense).

The ANID can be used to identify new and returning users. Dividing session count (ApplicationGroupId) by ANID gives the average number of sessions per user. The following bar chart takes the 10 ANIDs with the highest session counts and compares the resulting sessions per user value to the rest of the user base (whose count is roughly 500 other users).

What I see is that there is a core group of users that are heavily using my apps (YAY!). Now that I know who they are, I can zero in on their specific behaviors, how they relate to my ads, what features they use most heavily, etc.