Stone Temple Consulting Articles on SEO Topics

Yahoo's Dennis Mortensen Interviewed by Eric Enge

Tue, 16 Jun 2009 09:19:53 -0400

Published: June 15, 2009

Dennis R. Mortensen is a pioneer and expert in the Analytics industry. He is an accredited Associate Web Analytics Instructor at the University of British Columbia, the Author of data driven insights with Yahoo! Web Analytics, and a frequent speaker on the subject of analytics and online marketing. Mortensen is an Entrepreneur and was the COO of IndexTools until it was acquired by Yahoo! Inc., in May 2008. Today he is the Director of Data Insights at Yahoo! and sits on the Board of Directors at the Web Analytics Association, and he maintains the highly popular analytics blog, VisualRevenue.com/blog.

Interview Transcript

Eric Enge: Can you start with an overview of what has happened to IndexTools / Yahoo! Web Analytics is since the acquisition of IndexTools last year?

Dennis Mortensen: We were acquired on May 9th, 2008, and we are now one year into the integration project. We did a first simple update five months after the acquisition, which was renaming and re-branding the tool to Yahoo! Web Analytics plus a number of minor adjustments. This including a couple of new features that was supposed to come out in the upcoming IndexTools version that was on the verge of being launched anyway.

We then spent the last 7 months retooling this from the bottom up, meaning that we are now extremely close to being able to disconnect and shutdown the IndexTools data centers and move all legacy clients over to Yahoo. We are moving forward on having a completely different level of scalability, both from a vertical and horizontal point of view.

From a vertical point of view, we are now able to take on very large clients. From a horizontal point of view, we can now take on a lot of clients. Not just having 5,000 clients, but knowing how to handle 50,000 or 200,000 clients is what aspire to. At the same time, with Yahoo! Web Analytics version 9.5, which just came out on April 28, we managed to include a number of very interesting points on the front-end, including demographic dimensions.

This is not just demographic reports, but actually true dimensions within the analytics engine. This is also true for the new psychographic dimensions we included. We also introduced new visualization capabilities from a tool that we previously called Rubix, which IndexTools had been developing on. We've moved that into Yahoo! Web Analytics now. We also came out with new negative segmentation opportunities. For example, you can now look at people who did not buy, or people who bought but paid less than $100. This is the reverse attitude of the segmentation process that you usually do. Finally, we are rolling out the new version of the tracking code, which is version 5.

This version is now served off the Yahoo infrastructure, and of course it has a completely different level of stability. All in all, I think we have around 30 smaller updates. We also increased the number of actions (or goals as it is called in GA), to 50. We also included 38 custom fields for everybody, not just the selected enterprise segment, so now this is not a professional services request; it is something that's included into the tool. We also actually managed to re-brand it into purple. :-)

Eric Enge: What's the use of the tool if it's not in purple?

Dennis Mortensen: You and I know that it does not mean anything, but funnily enough this is something that people notice, and it's a very good indication of us truly moving from IndexTools to Yahoo! Web Analytics. Most people seem to believe (and fairly so) that it is accurately handed over to Yahoo now.

Eric Enge: I am sure that's a goal you personally have been pursuing with some passion.

Dennis Mortensen: Very much so. I think the finest task of entrepreneurs who have been working on a startup is to make sure that they actually end up handing over the keys to the acquirer so they can run the machine. I have personally been working very hard in getting us to the point where we could say we did the job and we did it very well.

Eric Enge: What are your thoughts on the impact this product will have on your competition in the analytics space?

Dennis Mortensen: Before I answer that directly, let me try to answer it a tad indirectly first. I believe it's almost naive to think that any media company can be in business without providing some sort of data back to its customers. You get reporting back if you buy a newspaper, TV or radio ad, so companies like Yahoo, Google and Microsoft need to do that as media companies as well. What happens at the same time of course is that you have a set of companies providing independent data collection and independent reporting; such as Coremetrics and Omniture.

We need to look at it from those two viewpoints. So, coming back to your question; in regards to the media companies, I don't really think that it means too much. I don't believe that people will choose Yahoo! Media over Google Media based upon the type of analytics that we provide. They will just expect us to provide an average or an above average level of reporting and/or analytics.

It's important that people understand that even though we collect and report honestly, you can present data in any number of ways, still being honest, but still favoring towards your own business. Look at a simple question such as attribution. If I spend $100,000 on search advertising and use Google Analytics, search will probably look more successful than it really is by the pure fact that they tend to report on last click attribution, and this is especially true if I've compared that to display campaigns.

If you look at Yahoo! Web Analytics, we provide multiple attribution models, including the original referrer. The original model is typically the one that comes in from display, which is media that Yahoo is very good at. I am painting a different picture of the same dataset, and I think you'll see the different media companies do that to some extent. So to conclude, that's one part, where you'll see us compete, namely on the type of insights that we provide to our customers.

On the other side, I think you will see independent vendors really have to expand on their products, because Yahoo! Web Analytics and Google Analytics are getting more feature rich. The technology is simply becoming better and better, but that said, I would like to confirm that we are not out to compete against or replace The Coremetrics' or the Omnitures. We are simply there to provide the best possible insight to our customers. That might happen to be somebody who is using Omniture and might choose to use our product instead, but that's not really the goal. We just want him to be enlightened.

What happens then, and the reason that they have to expand on their product portfolios, is that all of a sudden we will do things with our analytics products that they simply can't do with, such as Yahoo! demographic information. It is something that only we can do, and Omniture or Coremetrics can't do this, because they don't have access to the rich dataset of user behavior on Yahoo! web properties. They need to figure out other ways of being competitive, and I think that's their task.

Eric Enge: Let's dive into the demographics features in a little more detail.

Dennis Mortensen: Let's just provide a quick example so we know what we are talking about here. Let's say that I am a customer of Web Analytics provider X. The way that works is that I will call up my vendor and he will provide me with a JavaScript data collection tag. I'll put that on my website, and by doing that I will start collecting data. Then it essentially takes data that I have and sends it to my vendor, who, to put it simply, will then reformat it and send it back to me.

If I send him 10 data points, he will then send me 10 data points back. What media companies can do is collect the 10 data points, but then add 5 more data points and send 15 data points back. Thus, you also get insight on some information that you didn't have access to before. We've tested this out in the release that came out April 28 with a number of dimensions such as age, gender and interest groups. This way we get an opportunity for everybody who bought this specific product to tell us what they thought the best converting segment was, and you would be able to know that it is females age 25 to 34, interested in politics and so forth. (Editor's note: here is a sample screen shot:)

Eric Enge: How many different kinds of data points are you supplying that type of data on?

Dennis Mortensen: We came out with 4 new dimensions, the first of these are gender. Age is second, and as a FYI we provide 5 age groups. The reason for doing this is that we are not just doing this on Yahoo! specific traffic. You can buy an advertising campaign off the New York Times, and we can actually measure against the dimensions of the 10,000 visits that you'll get from that. That's why we have fewer groups, so we are actually able to come back with some decent information. Then we have 16 interest groups, including politics, sports and entertainment, among others. Just as a positive note; we are actually trying to expand that all the way up to 377 interest groups, but it's something we are still working on.

Finally, but perhaps not as interestingly, is that we still provide information on what type of Yahoo! properties people visit. You might look at a section on your site and you'll see that you have a large percentage that uses Yahoo! Finance, Yahoo! Sports, Hot Jobs or something like that. That's probably less sexy, but still an interesting marketing data point.

Eric Enge: How many of those do you have?

Dennis Mortensen: In the hundreds.

Eric Enge: What's the basis of how you are assembling the data? Are you doing that by seeing who has visited a website, and then matching them up with data you have because they have visited other Yahoo properties? What are the mechanics of all that?

Dennis Mortensen: Without getting too nerdy, it is based upon the fact that we have a huge sample of visitors that we have insight into and who come to one or more of our properties every month. I think the last public number was 550,000,000 unique visitors every month. That is a huge sample. If you just think about this, the guys over at Compete are using a sample of perhaps 2,000,000, and they provide some pretty good information actually.

Some of the visitors we will have more information on than others (all anonymously of course), and some of them we won't have any information. That means that I won't have information on all visits, but I will have it on some of them, and some of them are enough for me to provide statistical significant reporting back to you. Just to elaborate a bit on that, we actually provide a very honest way reporting on this, something that we call confidence level, which you can set yourself.

First of all, we are very eager to make sure that we keep our high level of privacy, so you can't report on anything with less than five visitors. With our tool you can actually filter down to a single visitor, but if that happens, I won't report on these new dimensions such as age, gender and interest because of privacy. Anything less than five, we don't report on.

You might look at 40 visits from within the last hour and I might not have information enough to provide you with the default confidence level. You can then choose to decrease that confidence level within the tool, and that will essentially provide new reporting back to you on the fly. The reason for that is that sometimes you would want to have a very high confidence level. That could be when you choose how to spend the amounts in a budget; because you want to be sure that you are spending it wisely. Other times it might just be a simple non significant redesign where you need to get an indicator of whether you want to go left or right, and you might be happy with the confidence level at 70%. So this is something you can setup in the tool, and you can use it for different purposes.

Eric Enge: How is this integrated into the tool itself?

Dennis Mortensen: The really cool thing about this is that there is nothing to install once you've been upgraded to the new tracking code. That means that all the clients that we signed up since the acquisition didn't have to do anything, because they were in a new tracking code to begin with. The legacy clients, as I said earlier, have to be moved over now, so within the next couple of weeks they will have the opportunity to move over to the new tracking code as well.

If you use that, these data points are attached, not collected, because this is information we have already. They are appended to your dataset without you having to do anything. We will give you a couple of reports right out of the box, including an age report where you simply get the distribution and a split on a gender report. You can also see this on the two dimensions at the same time in a matrix. We also give you an interest report because that's a good way of getting started so you can see what your segments are.

That said, the most powerful fact, in my opinion, however, is the opportunity to use these in filters. You might look at sales from yesterday for a specific product, a specific campaign or a specific section of your site. You can choose to filter by these dimensions. If you want to look at the sales for these specific products, you can filter it by gender and then you will get specific information for those products, which might be very different than for other products. So it's not just about giving an overall average for the whole site, it's an opportunity to use this wherever you see fit throughout the tool.

People might compare this to what they saw in the now discontinued Microsoft adCenter Analytics, but they provided a couple of reports out of the box. We were very eager to make sure that this was not just about a set of fixed reports, but actually a true opportunity to use those dimensions within the reporting system.

Eric Enge: As you've always said, even from the early IndexTools days, you were never interested in helping people do report surfing. The goal was to give them a tool where they could build their own queries and do their own thinking. You've been consistent with that here it sounds like, because the ability to track these metrics in the filters is very interesting. You can see how many sales you've made to people of a certain age over a specific period of time, and you can see if your sales become very centric on a certain age group over that time period, for example.

Dennis Mortensen: Exactly. Let's use something simple here. Let's say you have a 2-year old website, with 20,000 visits whose visitors are spilt 50%-50% between men and women. Then you look at your sales, however, and you might figure out that it is fine that you have an even gender split for traffic, but 90% of your sales are coming from females, and that is an insight you can actively use for changes. Not just that you have an even split, but that the sales are coming from one specific demographic. Then you can start to figure out how to deal with that problem specifically. You might even increase spending on some campaigns, and decrease it on some on the others.

Eric Enge: Or you can just change the focus of the landing pages.

Dennis Mortensen: Indeed!

Eric Enge: There are a number of things that you can do with that kind of data, and there are a couple of other things that you mentioned that peaked my interest as well. You talked about negative segmentation. Can you expand on that a little bit?

Dennis Mortensen: Usually when people think of segmentation, it's about including things such as people from New York or people who looked at a specific page on the site. That's how we tend to think of doing segmentation. What we've included now is what I call negative segmentation. Making sure that you do everything but this defined thing; I might for example like to see people who did not read about shipping details. I might want to see people who didn't come from New York, where we have an offline store. Negative segmentation is a powerful new way of trying to get insight on your visitors.

Eric Enge: I can imagine there are an awful lot of things you can do with that. You can certainly combine negative segmentation with positive segmentation, for example.

Dennis Mortensen: Exactly. You can choose to look at people who bought something, but didn't look at the shipping details.

Eric Enge: If you see that 70% of your abandon rate happens after people look at your shipping charges, you can decrease the price of shipping, but you can only do that by doing some of these comparisons.

Dennis Mortensen: Agree.

Eric Enge: Can you talk a little bit about the new path analysis details as well?

Dennis Mortensen: As you know, there is a huge debate in our little analytics community on the value of path analysis in general. Some people say that it's completely bogus, some people say it provide decent insight, and then we see some people simply use it to figure out how people navigate, where people drop off and essentially to get insight into how people go around their site. I am not advocating path analysis, but I'll tell you what the new edition includes.

What we had before was an opportunity to see two levels deep, and only ten steps on each level. What I am talking about here, when we say path analysis is for example, that when you can look at the last five steps in an e-commerce funnel, and before people move from checkout to payment, you have a drop off of 60%.

Since 60% obviously didn't go down the funnel, they clearly went somewhere else. You want to investigate where they went, and for that you can drill into the path. First of all, the new path analysis is a Flash Application. So you are thrown into a Flash app within our UI, which you can actually expand into full screen and really work the path analysis. You can now choose not just to see one or two steps, you can drill into endless steps.

Let me expand on this a little bit. Say I want to look at the 30 pages that my visitors go to on my website, I can actually now work the path analysis much more visually, because you can move items around. If you imagine an elastic spider web, that's kind of how it looks and that's how you can navigate around it.

Eric Enge: There is a lot of flexibility in how you route it.

Dennis Mortensen: Yes. You are not limited to two levels anymore. This is very flexible and unlimited levels now, plus you can expand it and work at full screen, so all your work canvas is the path analysis.

Eric Enge: Can you speak a bit about the changes in visualization techniques?

Dennis Mortensen: Most people will agree with me that most analytics applications are set up in essentially the same way. In the upper left corner you have a calendar, below that you have a menu with a number of reports that you could choose. On the right-hand side you have a chart, below that you have a set of row-based data. That's how you build up an analytics application, and that's all good and fine, but what we are trying to do with this new version is improve the chart itself.

Let's say you look at visits for the month of May, you'll have that as a bar chart, and you'll have visits on the y-axis and the dates on the x-axis. You'll also have a number of bars illustrating how many visits you have for every day. That's fine, and it used to be static and illustrated very nicely, but we've turned that into an active component now. That means you can expand on it and chose how you want to visualize it.

You can turn it into a trend chart, a bubble chart or into a bar chart and work it that way. That is not the powerful thing here though, the powerful thing is that you can choose to work with multiple metrics at the same time in the chart. I might customize the reports, so I can have visits, time spent on the site, bounce rate, average value per order, and a number of other metrics as well. What I can now do is choose how I want those to be visualized in the chart. I might want to visualize two or three of them at the same time, trying to spot a potential correlation.

I might want to use bar charts, but I want to color them based on the bounce rate. So if there is anything out of the ordinary, I will be able to tell from the chart itself. We even expanded it to the point where people are actually able to apply notes to elements on the chart. You can start writing notes on the chart, that's how flexible we've made it, and when you are done it works just like it did before.

I think we all agree that we can have data that might provide insight, but I can't visually see it because I have too many numbers to process in my head. I can keep drilling down until I get it, but if you have the right visualization opportunities, sometimes you can very quickly gauge where you have issues and then you can start working on them. Visualization is not all about coolness, I think it's actually a pretty essential element to have. Most people have charting of course, but I think we tried to take it just one step further. Not magic yet, but just one step further.

Eric Enge: Is the tool going to be available on an open basis to non-Yahoo advertisers?

Dennis Mortensen: I believe there is an understanding in the market that we are supposed to become a free for all tool like Google Analytics, and thus creating a head-on competition. There is nothing more fun than a competition, but that doesn't mean that it's the right thing to do however. At the current time we have no intentions of turning this into a free for all solution. Don't expect us to put up a webpage where you signup, you get a JavaScript and then you can go track your website. That's not really what we are shooting for.

We are looking for opportunities to connect with advertisers and publishers in the sophisticated end of the market. That means we'll probably not ever get to the million accounts, or however many accounts Google Analytics has, but that's not really our target anyway. If we can just capture the head- and torso end of the market, we've pretty much solved what we set out to solve. That said, it doesn't mean that you necessarily have to be a Yahoo! customer in the long run. Right now you need to be affiliated with Yahoo! in some way, shape or form in order to get access to an account, but this is something that we are expanding upon all the time.

Eric Enge: What will be coming down the pike next?

Dennis Mortensen: That is a very good question, but it's something I would have had an easier time responding to when we weren't Yahoo!. Now, we have legions of communication and legal folks that make it more difficult, but I think one thing that I can safely say is that you will see us keep appending the data with more and more information simply because of the power of combining a data company with a media company.

I think that's a safe bet. Another safe bet is the idea of combining post-click and pre-click data information, so you can see what people were doing before they arrived at your website. Just to give you an example if it doesn't make sense. Some people are actually searching for you or for your products and you will be in the SERP results, perhaps not on page-1, but you will be in the results. I'll be able to tell you information on their intent before they even visited your site, such as search phrases that they did not click on.

Eric Enge: Right. Like a search funnel.

Dennis Mortensen: That could be one thing, yes. Sometimes people search with seven different phrases. They end up with the last one, and that's the one they click on, but you have no insight into what happened before that. I think what is happening before that story is something we might want to move into.

Eric Enge: Thanks Dennis!

Dennis Mortensen: You are most welcome Eric!

Have comments or want to discuss? You can comment on the Dennis Mortensen interview here.

Other Recent Interviews

About the Author

Eric Enge is the President of Stone Temple Consulting. Eric is also a founder in Moving Traffic Incorporated, the publisher of Custom Search Guide, a directory of Google Custom Search Engines, and City Town Info, a site that provides information on 20,000 US Cities and Towns.

Stone Temple Consulting (STC) offers search engine optimization and search engine marketing services, and its web site can be found at: http://www.stonetemple.com.

For more information on Web Marketing Services, contact us at:

Stone Temple Consulting
(508) 485-7751 (phone)
(603) 676-0378 (fax)
info@stonetemple.com

Enquisite's Richard Zwicky Interviewed by Eric Enge

Mon, 8 Jun 2009 09:45:44 -0400

Published: June 8, 2009

Richard Zwicky has been involved in search marketing for 10 years, starting in the late 1990s. He started in the industry by managing the online campaigns for his own successful e-tail operation, which quickly led to developing Metamend, a leading search engine optimization firm which he co-founded in 2000. As CEO for Metamend, he managed and led the optimization campaigns for web properties ranging from SOHOs to Fortune 500 sites.

He split Metamend and Enquisite into separate companies in 2006, as Enquisite's services are designed for use by any SEO and SEM. Today, he is leading Enquisite, which recently released its first products. Richard's work is focused around helping search marketers manage campaigns more easily and with greater success. Richard believes in long-term successful campaigns that are built from the ground up, and are never caught flat-footed by shifts in search engine algorithms or by regional variances in search user behavior.

Interview Transcript

Eric Enge: Can you tell us a little bit about Enquisite Optimizer?

Richard Zwicky: Enquisite Optimizer is built from the ground up for search marketers. I used to run a lot of campaigns and it was always frustrating and time-consuming to get the right data out of existing web analytics. It was even impossible a lot of the times because, quite honestly, the focus in most analytics products isn't on the organic search marketing campaigns. They are focused a lot more on paid search. The legacy of analytics products goes back to the days when IT needed data about page load times and information like that, and they have continued to be built on that foundation.

We came at it from a completely different angle, and developed a new way of collecting, processing and reporting the data to help the search marketer understand that they needed to do the job more efficiently and deliver higher value to their customers from a variety of perspectives.

Our long tail analysis was one great example of that. People like Rand Fishkin were always looking for better data about the long tail of the campaign they are running. They want to know how to visualize it, understand it and understand its shape because a good site has a very standardized traffic shape and pattern for its tail's traffic, whereas tails from sites that are less well-optimized don't follow standard form.

But you can't see it unless you visualize it, you can't understand it or know how to deal with it. So we built that element of the report incorporating the long-tailed graph and you've got the choice of long tail or amounts of overlays, so now you can visualize where your traffic is coming from as well. But we built it using the logic behind how you would run a campaign. Of course, different businesses have different needs. For example, a retailer that sells only within the US will not care about search traffic coming from anywhere else. They want to understand what is coming from the US. You should be able to analyze that and turn that data into action. The application is built so that you can segment and break out your traffic by the logic with which you actually went into business. You can segment geographically down to the zip code level.

As another example, a standard analytics package will give you which phrases bring users to your site and which search engine sent those users. We make it possible for you to segment your visitors by which webpage they landed on, from what geography and from a variety of other parameters. So you really get what you want the way you want it. If you are am SEO firm, you might want to target the word organic and find out all the different the different ways that people are using organic to arrive at your website (long tail segmentation). Show me all the strings that include the word organic, show me what that tail looks like or show me just everything that includes the term SEO.

Eric Enge: So you can specify a stem like you can in Wordtracker or KeywordDiscovery fashion?

Richard Zwicky: Exactly, but it also provides analysis as well. Simply because you are getting traffic doesn't mean that it is good traffic. One of the other challenges that marketers have is the ability to see what the traffic really looks like and to understand what part of that traffic is actually meaningful and relevant. If you are a retailer, you would care about conversion; if you are a publisher, you would care about page views and time on site; Enquisite Optimizer discovers and reports on what is optimal on a site-by-site basis. It compares all of your referral traffic to identify optimal patterns in terms of the user behavior and which traffic has the highest potential and which one has the lowest. Just because you are getting a lot of traffic for a term, doesn't mean it is actually ever going to result in conversion.

There is a mathematical process where you can show what the normal traffic for conversions looks like and what other traffic matches up. You can actually target the right terms for the right pages and do better job of shaping your traffic. This saves you from trying do it yourself with trial and error, which might take you months. But with our system, it will show you that information and within a couple of days you will start seeing patterns of what is normal and what is optimal. Then all of the sudden it's helping you make those decision so you can get on with optimizing and building your campaign out as opposed to sitting there and trying to figure out what to do next.

Eric Enge: You mentioned a little earlier that the shape of the long tail curve behaves differently for sites that are less well-optimized. How is it different?

Richard Zwicky: It's actually quite interesting. Normally, there are a very few search terms that bring large amounts of traffic, and there are a much larger number of terms that bring in relatively smaller amounts of traffic (each). As it turns out, the cumulative value of all the low volume terms is about 70 to 75% of your total traffic. In other words, the number of smaller traffic terms is so large that they cumulatively deliver more traffic than you high volume terms.

But the tail in a poorly optimized site is constructed slightly differently, with much more of the search volume going to the high volume terms, and with a much smaller tail. If you target a term like blue shoes, you need to understand all the variations of the ways it may come up. You want to be able to capture things like blue tennis shoe, blue running shoes, blue canvas shoes, blue leather shoes, blue suede shoes.

That is part of your site's referral tail that a lot of times people don't optimize properly against, but as soon as you start recognizing these terms, you are not just getting those variations but you are getting blue canvas, blue canvas deck shoes, which starts building it out more and more. And that's what you see in a well-optimized campaign, you see that rich variation of terms also focused on certain themes, but all pointing back to the same core term that you want to capture.

As people start building, constructing longer and longer search queries, they are getting more and more definite about what they are looking for. And the reason they are doing that is because they are highly motivated, they are looking for what they want, and they want to get on with it, they want to purchase it, they want the information about it and they want to act on it. And when your tail is properly constructed, you are capturing all those variations through the optimization of your site and you are actually able to see it reflected in the tail.

Eric Enge: So in a well-optimized site, you might have 70% of your traffic coming from a long tail, but on a poorly-optimized one, it might be the opposite.

Richard Zwicky: Yes, that's a good way to put it.

Eric Enge: Being able to visualize your own long tail is huge because a lot depends on if the site is optimized properly or not.

Richard Zwicky: There are still opportunities to grow and improve on any site's long tail, even though if it is already well optimized.

Eric Enge: The way you collect data is through JavaScript on the publisher site?

Richard Zwicky: Yes, we provide every website operator unique JavaScript for their site. They put it everywhere in their site, not just on particular pages. And the reason for that is we provide the user behavior analyses to help give them more information. What is nice about it is two-fold. The JavaScript is actually served off with the Akamai network, so instead of having to log all the data to one central point we can use the nearest server that Akamai has, which makes it very responsive and very robust. This generally provides a load time of 12 milliseconds or less for anybody on a broadband connection anywhere on earth. And this means that we don't miss data even if people start loading a webpage and click the first link to move on, prior to the page fully loading. Because we have already captured the log information on their behaviors, we can report on it, and add value to our clients.

This also means that we don't have to go to sampling to fill in any gaps, which is a critical issue with some analytics. So we know that when we are reporting we know it's accurate and comprehensive data we saw and that the customer would actually receive as opposed to any hypothesizing or extrapolating to complete a picture. This also means that you don't have to worry about whether or not 10% of the data is missing. That 10% can be crucial. In this case, there is nothing missing. The only thing that would cause data to be missing is if the visiting user has disable Javascript in their browser. The other advantage we have is that we provide a single JavaScript tag for everywhere in your site. So when you want to analyze outcomes, conversions, actions or anything like that, you don't have to modify the script on a page-by-page basis. You just can figure it out once, and you can specify what events you want to track and then go backward and look through all the data.

You actually collect everything with that one JavaScript tag. If you have had the tag in place for a year, and then you realize you want to do a new analysis of the data over the past year, you can do that. This is not easily doable in many web analytics packages. Additionally, the JavaScript tracks sessions across multiple visits so you are able to understand attribution over time, not just attribute all your sales to the last click. If somebody came back twenty times and finally made a purchase, you can see how they first got there and when they came back the second, third and nineteenth times. This way you can actually understand how all of your online marketing efforts start fitting together, and that's incredibly valuable.

Eric Enge: Let's talk a little bit about Enquisite Campaign.

Richard Zwicky: Thanks! We are getting phenomenal feedback and response from the people who have been using Enquisite Campaign as beta testers in the lead up to launch, which occurred on May 19.

In the search marketing industry, by which I mean the paid side of search, the market has a very straightforward business model that everybody can understand. That is that you invest a certain amount in paid search and your agency gets compensated a percentage of that amount for managing your spend. The more successful they are, the more money you will spend in the future, and the agency succeeds as well.

On the organic side of the business, everybody is always negotiating for fixed-rate contracts. And this is fine, except for the fact that it's very hard for anybody to understand the true nature of the opportunity or forecast what the contract should look like, or how much effort is really required to succeed.

Also, a fixed-rate contract is disincentive to perform at a certain point, because a lot of SEOs can deliver a ton of tremendous positive value, but they don't get paid more for finding that other opportunity and driving all this new business into the customer. Having a fixed rate contract can be almost counter productive for them at the certain point, because they are not able to leverage that opportunity as intended or are limited by contract scope to how much value they can drive into a customer's business, thus also limiting how much of a profit they too can make for themselves.

Eric Enge: Right, the danger in consulting contracts that are fixed rate is if you have a consultant that's savvy enough to know when they've done enough to earn their fees, which is actually better than the consultants that aren't savvy enough to know that because they probably don't care about earning their fees, but then they have some other client that's barking at them and they stop paying attention to you.

Richard Zwicky: That's correct. And in the other model, you can still have a base fee for all the base work you are doing. But an incentive model may allow you to stretch your goals, go for that opportunity and discover where those other opportunities are. Then you have a model where all of a sudden you were rewarded for going that extra mile, because what it basically means is you are delivering added, unexpected and unforeseen value to the customer.

If the customer earns more sales, the consultant should win with the customer. Today, you might run under a fixed rate model, and you are able to get running on an existing campaign, but every time you come up for renewal there is a frustrating discussion about what value you delivered.

That discussion has become obsolete because of Enquisite Campaign. You can prove the value delivered, so it shouldn't be a question of if you your money's worth or not. It is more like "Wow, I can see that not only did I get my money's worth, I have got more than I ever expected. Definitely, I am renewing with you."

You need to be rewarded for the value of your work. There is no upside or incentive to go that extra mile in the present systems, where there are fixed-rates contracts and they don't really reward most SEOs. A lot of large agencies are now having to focus on the SEO business models. They struggling with the question of how to compensate or build the right pricing models to sell to their clients. Now the ecosystem can run much more efficiently so that some of the larger agencies are going to go out and contract more and more SEOs in a much more efficient manner and help everybody win together. It's a win all around.

The client wins because they are getting value, and the agency wins because they get compensated for delivering that value. I mean, what more could you ask?

Eric Enge: Can you describe a little bit about how you collect the performance data and do the value calculation?

Richard Zwicky: To collect the data we use the same JavaScript that we use in all of our products, but where the application actually begins is in helping people determine what an opportunity really is, so that you are able to determine if you are focusing on something that's worthwhile or not. Or, if a customer comes in and says I want to show up and get customers for a specific term, we are able to sit down with them and determine whether or not it is possible and worth the time, effort and investment.

Our system figures it out and runs a really intensive series of calculations to determine how many people will search for the term over to next 30 days. Let's say the term is Blackberry. How many people are going to search for Blackberry over the next 30 days? And if you are placed in the top four, how many referrals can you expect to receive for that term?

Eric Enge: Are you using that classic AOL data for people who click on number-one or number-two or you are using your own data?

Richard Zwicky: We use our own data. We've done a lot of analysis work, and one of the beautiful things about having such a large data sample internally, is that we are able to qualify, verify, validate and iterate the reference data as the marketplace changes. You are not always going to be number-one, but if you are placed in the top 4 and you are bouncing around in there, what's a reasonable expectation of the referral traffic you could acquire? Now, if you are number one all the time, you are going to exceed the numbers we're laying out as potentially available because we are doing the weighted average of how much traffic you will get if you are in the top four. We're also adding a slider so you can project "what if I only reach page 3?" type questions and answers.

That's the first part of the platform. The second part is that we make it possible for you to build a campaign based around conversions if you want. Then, customer can pay you as they make a sale. We can also do it based on a cost-per-click basis. Essentially, if you are only targeting Massachusetts for Blackberry phones, you would be able to build a very targeted SEO campaign for that, and our system will do the calculation to determine what the fair market price for organic clicks will be.

To establish a fair market organic price, we actually take into account the differences between informational and transactional queries, the difference in conversion rates between paid and organic in each area, the difference in user behavior within search results and the website for when they actually arrive there or how much of the traffic that you get from organic that is actually good.

This helps establish where the market really should be because you might want to be paid on a cost-per-click model or a customer may want to pay you on that basis. In an affiliate model they have to pay for every referral that comes through. But how do you define what the payment is? And to date, there hasn't been a good model for defining that in organic search. In Enquisite Campaign we have built it.

Eric Enge: Thanks Richard!

Richard Zwicky: Yes, thank you Eric!

Have comments or want to discuss? You can comment on the Richard Zwicky interview here.

Other Recent Interviews

Microsoft's Scott Prevost Interviewed by Eric Enge

Wed, 27 May 2009 09:46:47 -0400

Published: May 24, 2009

For over a decade, Dr. Scott Prevost has worked to bring natural language processing technology to the marketplace. As a graduate student at University of Pennsylvania, he developed theoretical models of prosody for synthetic speech, as well as technology to generate dialogue for autonomous agents. In post-doctoral research at the MIT Media Lab and FX Palo Alto Lab, he integrated gestures, facial expressions, and other interactional cues into his research, creating lifelike 3D characters with speech recognition, dialogue processing, and vision capabilities.

Dr. Prevost co-founded and served as CEO of Headpedal, Inc., a software company that specialized in creating virtual character interfaces for customer-facing applications on the web. Dr. Prevost also previously served as CEO of Animated Speech Corporation, which produces interactive, animated tutors for speech and language development. Dr. Prevost was General Manager and Director of Product at Powerset, where he was focused on developing the user experience for natural language search. Powerset was acquired by the Microsoft Live Search division in August 2008, where Dr. Prevost currently holds the position of Principal Development Manager.

Interview Transcript

Eric Enge: Can you provide a quick overview about yourself and Powerset?

Scott Prevost: I have been working on natural language systems with the goal of helping information retrieval in particular for quite a while now. Powerset was founded with the notion that we can improve search results by having a much better understanding of the meaning of the documents and of what people intend with their search queries.

The way that we do this is to apply very deep natural language processing technologies to the documents as we are creating an index. And we also apply that to the queries at runtime so we can do a better job of actually matching meaning to meaning as opposed to just finding the keywords.

Powerset was founded in 2006 and we launched our product in May of 2008, which was initially a Wikipedia search engine. Then we were acquired by Microsoft in the summer and closed the deal on August 1, 2008.

Eric Enge: Can you talk a little bit more about the goal of better understanding a searcher's intent and the mechanics that you use after doing that?

Scott Prevost: One of the key points that I want to make is that Powerset is not just about understanding intent in queries. That's part of the equation for getting better search results, but once you have that, you also have to have a much better understanding of what's in the documents as well. So, it's not enough to know that a user is looking for a certain kind of search result, you also need to be able to match that to what's actually in the document.

So, what we propose to do is very different from what most other search engine startups do. Most search engine startups are trying to take the existing keyword search model and add some bells and whistles to it or put a new front-end on it. What we did is completely reinvent how the index is built by applying technology that we licensed from PARC, which allows us to do very deep linguistic processing.

We essentially look at a document, break it into sentences and then we analyze each sentence using a very robust linguistic parser. We extract semantic representations out of that, and it actually has semantic representations that we store in our index.

We do a similar processing on queries at runtime, and then we look to match these semantic properties, the keyword properties and other document properties. What this means is that we can find sentences that may have the right meaning, but use slightly different words. If you type in "When did earthquakes hit Tokyo" in powerset.com, you will see answers that use words like strike instead of hit. Then you will see that we are actually able to highlight dates in the captions for those answers because we've done the linguistic analysis on the sentences, not merely matching keywords.

Eric Enge: So how is this different from Latent Semantic Analysis or Latent Semantic Indexing?

Scott Prevost: We are actually doing the semantic processing upfront, and we are doing all the hard work on the backend, so that's one big difference from all the other approaches that we've seen out there.

Eric Enge: So you are doing some preprocessing?

Scott Prevost: Yes. We are processing the documents as we index them. We are also trying to do some analysis at query time, because natural language technology is still quite expensive in terms of the compute power that's needed. So, the degree to which we can compile all that out in the index means we can produce a runtime that's on power with a keyword search runtime in terms of latency properties.

Eric Enge: So, when we talk about the problems with traditional search engines, one of the things that I saw you focus on was the fact that they required users to speak their language?

Scott Prevost: That's right, yes. Generally, we've all gone through the process of trying to find that document where we try to figure out what the right collection of words that will pull this document up is. That means that you have to start thinking like the author of the document, imagining how the thing that you are looking for might have been expressed.

We generally try our query a few times before we find what we are looking for. By adding the semantic analysis, we are allowing people to be a little more natural in the way they express themselves. You don't necessarily have to worry about the specific keyword, because we are likely to find a synonym.

You also don't have to worry about excluding stop words or which words are going to be matched with which words in the matching algorithm. We just want people to be able to write a natural phrase or even a question, and then let the search engine do the hard part; figuring out what the appropriate matches are.

Eric Enge: Right. In existing search engines it can be a disadvantage to have extra words that aren't actually necessary to the query. This is a result of using a more basic method for matching up the words in query with words on page.

Scott Prevost: That's right, yes. And of course it creates some interesting issues for us, because now we are trying to change user's behavior a little bit. They have grown very accustomed to thinking of a search engine as words and documents that include these words. So now that we are messing with that interaction model, our hope is that people's behavior will gradually change as they start to realize the power of the system that we are introducing.

One thing that we have been very careful with at Powerset is trying to maintain the old model as much as possible. So, if you just type keywords into Powerset, you will still get results that are just as good as those from Google, Live Search or Yahoo.

Eric Enge: So you have talked a little bit about stop words, can you expand upon that a little bit? Define what they are, how they are treated by regular search engines and why making use of them in Powerset is important?

Scott Prevost: Stop words are words that the search engine just disregards; prepositions or words like "what" and "where." It's a very salient limitation to implementation. Basically the idea is that if you try to match documents on those words, they tend to be less important in the query because they would match so many documents. But in reality they are the linguistic glue in the query and in language. They start to tell you how the other important words in the query link together, and that allows us to look for those links in the document when we are matching a query by processing them linguistically. Let's go back to the earthquake example. I am not specifically searching for the word "did," but that word is still part of the verb complex in that query. So the parser knows that "did" and "hit" go together. Basically, we are not matching for that specific word, but we are matching verbs together that semantically match. So instead of "did hit," we can use the word "strike."

Eric Enge: Right. So for example, you could accidentally get something like "did not hit?"

Scott Prevost: Yes. We are not currently processing negation in parser on a real detailed level because it is such a tricky problem. It would actually match queries that get the negation incorrect, but that is generally useful information for the user anyway because it is relevant to their query even if it isn't an exact answer.

Eric Enge: Right. So, that's an example of something that you would be working on in the future?

Scott Prevost: Oh, absolutely. That and things like sentiment analysis are all things that we will be working on in the future. For sentiment analysis, say you want to know what positive things a particular politician said about a particular topic. You would get a different set of results then if you just asked what they said about the particular topic.

Right now we are basically working on sentence level linguistic matching along with other broader document properties like keywords, anchor text and using all of these things to rank our results. But as the technology improves, we'll start to look at many more of these kinds of discourse level properties so we can really understand what the most important sentences in the document are and how they relate to each other. And as we can learn from these kinds of approaches, I think we'll see the relevance of search results improving with time.

Eric Enge: Right. For example, if someone types in "The Office," they probably don't just want to search the phrase "Office." They probably mean the TV show.

Scott Prevost: Yes. And in fact if you type that into Powerset, you will get a result that's tabbed at the very top, for The Office television show. There is also a tab for the UK television series by that name, one for the band and one for Microsoft Office. So that's a pretty ambiguous query, but chances are you probably meant the television show by phrasing it that way. That's the one that comes up first.

Eric Enge: Right. So let's get back to Latent Semantic Analysis. One of the things that you do is look at the entire set of documents, and determine relationships between words by proximity and frequency. This way you might discover that doctor and physician probably mean the same thing, or at least almost the same thing. What I am getting at here is the analysis of the corpus of documents to extract relationships.

Scott Prevost: We are not using what you are thinking about as Latent Semantic Analysis. We are actually using more of a symbolic approach to the linguistic processing. That's the first phase of what we are doing. We look at a document and break it into sentences, and then we actually parse the sentences using technology that we've licensed from PARC. What this does is it allows us to create fairly complex semantic representations of the meaning of those sentences.

And it also allows us to represent ambiguity in those interpretations as well. This way we can index he most likely reading of that sentence, and the other possible readings as well. What happens then is that these things become semantic features that get thrown into the mix with keyword and other document property features that are used by our retrieval system and ranking system.

We are not retrieving results just based on meaning matches and partial meaning matches. It throws that into the mix, and that retrieval and ranking system is a machine-learning based algorithm. In that sense we are starting to use statistical approaches, but we start with a very symbolic representation of the meaning in the document. Then that is used by a machine learning algorithm to retrieve and rank the documents.

We are not pulling the relationships based on things like frequency, we are actually uncovering the linguistic and semantic relationships through symbolic approaches.

We actually do have other projects going on within the company that are looking at more statistical approaches to these problems. But I would currently characterize that system as a hybrid.

Eric Enge: What exactly does it mean to say that it's a symbolic approach?

Scott Prevost: It means that it's rule-based semantic processing as opposed to just uncovering things from machine developed approaches. For example, if we have a rule in our system that says if you kill something it dies.

Eric Enge: What are some examples of search queries that highlight the power of this approach?

Scott Prevost: Let's start with something like Siddhartha. The first thing you will see is the summary of Wikipedia pages that are relevant and that you can tab through. You probably were looking for Siddhartha, the founder of Buddhism, when you typed it in, but there is also a film, a novel and an American rock band by that name as well. You can just click on the tabs to see those different snippets.

In the section below that, you will see something called facts from Wikipedia, and these are some of the semantic relations that we have automatically extracted using these linguistic techniques. In the second line you will see "Siddhartha renounced the world," and if you click on world, you will see sentences from which we extracted that fact. We extracted that from three different sentences on three different Wikipedia pages, and you will see that it's not the case that we are using proximity in the second one.

Siddhartha is actually pretty far away from the word renounced, but linguistically they are tightly tied together. It's just that there is another phrase intervening. So this starts to show you how we are taking data that's in Wikipedia and starting to structure it. If you click the More link at the bottom of that section, you'll see that there are a bunch of other relationships that we've pulled from.

Eric Enge: They are just a little less tightly matched.

Scott Prevost: Exactly. Now you can also get to this structured information pretty directly. So, if you type in "What did Siddhartha attain," you will see Enlightenment and Nirvana. So, in a sense, these subject-relation-object semantic triples are great for answering questions.

So, try something like "What was banned by the FDA." Now, if you are at the right part of the screen, you will see More. If you click that you will see up the longer list. And if you say click on something like "cyclamate" you will see the sentences from which we extracted that fact.

We are basically allowing a whole new type of interaction. I type a simple subject-relation-object question, and now I get a list of answers that are supported by the text that we've uncovered through this linguistic analysis. And you'll also note that we can start to make distinctions between a query, like "who defeated Hulk Hogan," and "who did Hulk Hogan defeat?"

If you search "who defeated Hulk Hogan," and you click on More you will see the whole list. And if you do the other query, "who did Hulk Hogan defeat," you will see that the lists are different because we are actually looking for these things in the correct relationship to each other in the text. We are not just looking for the keywords "Hulk," "Hogan," and "defeat."

That's an example of a pair of queries that would be very hard for a typical search engine to distinguish between, because the key phrases are the same and the word order is what defines the difference. So let's pick a query for the regular search results. Let's type in "how many nuclear reactors does Japan have?" Now, here is a query with a lot of stop words, right? But it's a query where I think it is pretty easy to tell what the user is looking for. In the very first caption we can see that Japan has 55 reactors.

We are basically interpreting the fact that you typed in "how many" as the fact that you are looking for the particular number of nuclear reactors. This is just something that you don't get when you use Google, Yahoo or Live Search, or any of the keyword search engines. Let's try "Who mocked Sarah Palin?" Now obviously, the other search engines do a pretty good job of finding relevant results for this. But what I want to show you are some of the captions in the blue link results. So we get things about impersonating Palin and parodies of Palin. It's not that we are necessarily just looking for the specific words Mock Sarah Palin, but we find synonyms that are semantically related to and can highlight those right in the answers.

The hope here is that we can help users better understand when one of these blue link results is actually truly relevant to them, and we can save the clickthroughs when they are not. Another thing that we can talk about is pulling data, or pulling search results from structured data. So, if you type "GM board of directors," we actually connect with Freebase in order to produce this result at the top.

Eric Enge: Along with the pictures of each of the members.

Scott Prevost: Right. If you type in "what movies did Heath Ledger star in," you will get the same results as if you typed in "films with Heath Ledger," because we are actually doing semantic analysis and you are essentially looking for the same thing whether you type in the first phrase or the second.

Eric Enge: The list of movies shown didn't change at all. There were just some subtle changes to the results below that. Those are interesting examples. Currently you are operating this on Wikipedia?

Scott Prevost: That's right.

Eric Enge: What was the reason why you chose Wikipedia in particular?

Scott Prevost: Well, there are few reasons. First of all, as we were developing the technology, Wikipedia was a great test bed because it covers just about every topic that there is to cover. We wanted to make it very clear that our technology was about linguistic processing, and that we didn't have to be within a specific, very narrow semantic domain for the technology to work. Some other natural language approaches have taken that very narrow approach, and that's not what we've done. So the fact that Wikipedia is so broad was very appealing to us.

The second reason is that Wikipedia is well written, so it parses pretty nicely. Although, our technology is designed so that when we can't parse something, we still index it as keywords. It has to be graceful degradation into the keyword world.

The final reason is that Wikipedia is prevalent in so many search results these days. It's almost hard to find a search query that doesn't have a Wikipedia result in the top ten. So we know it has a very valuable set of documents to index. When it came time to define a product to launch, we had some resource constraints. It takes a lot of hardware to spin an index that has as much information as the Powerset index.

So we had to find a smaller set of documents, and then it becomes a challenge to find a small set of documents that hangs together for the user in a meaningful way. So we decided initially to restrict ourselves to Wikipedia alone, rather than having Wikipedia and a few other smaller document sets that might not fit in.

But now we are currently expanding the index. We've been continually playing around with other kinds of documents. The technology is not particularly wedded to anything that's specific to Wikipedia, but it's such a valuable set of documents on the web that so many people use.

Eric Enge: So, if we think about this as runtime, if someone enters a query is there reason to believe that Powerset is more or less compute-intensive than regular search?

Scott Prevost: It's marginally more compute-intensive at runtime, but the reason that it is only marginally more compute-intensive at runtime is because we do the real compute-intensive things at index time.

Eric Enge: I assume that at that time it's probably significantly more compute-intensive.

Scott Prevost: Actually the only thing that's more intensive at runtime is the fact that we are parsing the query. Once we've parsed the query, then the actual retrieval it is very similar to keyword retrieval, except we are retrieving on semantic features as well as keyword features. But it's very similar apparatus.

Eric Enge: Right. But you probably have a higher level of investment to build the index, because, you are doing all that preprocessing?

Scott Prevost: That's right. We are doing very deep processing on the documents as opposed to just pulling out the words.

Eric Enge: Is there any insight you can give us at to how much more difficult it is.

Scott Prevost: It depends on the degree to which we do it. It's a very granular system and we can adjust a lot of knobs. It can be a anywhere from ten to one hundred times more expensive. I am sure we could make it a thousand times more expensive if we thought we would get the benefit from it. Our goal initially has been to improve relevance while disregarding cost in some sense. But obviously, we are pragmatic when push comes to shove. The goal was to find out which of these features are most important for improving relevance. Then as we learn more, we can simplify and skip some of the computation that's not giving us as much bang for the buck.

Eric Enge: Are there any components of Powerset that are integrated in the Live Search at this point?

Scott Prevost: We've integrated a few things. We've integrated some of our direct answers using Freebase, some improved captions and snippets under the blue links for Wikipedia. And we've also done some things with related searches. And of course we are working on a much more robust integration plan, although I don't have any plans to announce anything today. But some exciting stuff will be coming down the pipe for sure.

Eric Enge: Any closing comments?

Scott Prevost: We are excited at Powerset to be having the opportunity to take this technology to scale and to integrate it in a product like Live Search. We are really thrilled because it allows us to see our dream actually come to fruition. And I think that we have just a lot of exciting stuff coming down the road.

Eric Enge: Thanks Scott!

Scott Prevost: Yes, thank you Eric!

Have comments or want to discuss? You can comment on the Scott Prevost interview here.

Other Recent Interviews

YouTube's Product Management Team Interviewed by Eric Enge

Mon, 18 May 2009 09:19:17 -0400

Published: May 17, 2009

Our first interviewee, Tracy Chan is a Product Manager at YouTube. Prior to working at YouTube he was a Financial Analyst at Google. He has also worked as an Associate at Stockamp & Associates and a Corporate Strategy Intern at eBay. He got his degree at the University of California, San Diego.

Our second interviewee, Matthew Liu is the lead product manager on YouTube Sponsored Videos. In this role, he focuses on building an advertising platform that allows video creators -- from the everyday user to a Fortune 500 advertiser -- to reach people who are interested in their content, products, or services, with relevant videos. Previously, Matthew led numerous other projects at YouTube for advertising, content partnerships and rights management, and community.

Matthew has a MS in Management Science & Engineering and a BS in Electrical Engineering from Stanford University.

Interview Transcript

Eric Enge: Can you provide an overview of what Insight is, and why you created it?

Matthew Liu: There are millions of people watching hundreds and millions of videos every single day on YouTube. We started to hear from advertisers, content providers and everyday users that they wanted to understand YouTube's audience,. They were asking questions such as: "How do we really standout, how do we understand our ecosystem, and how do we know how the videos are performing"? They basically wanted to learn more about their audience in order to make better content.

As YouTube was growing, it started turning into the world's largest focus group. So basically what we did was build up a pretty powerful analytics tool that helps content providers, advertisers and users better understand their performance on YouTube.

This is a tool that's free to anyone who has ever uploaded a video. When I uploaded my first video, I got a hundred views in the first two days. And that was actually surprising to me, because it was a little animation video that was actually not that interesting. I was wondering if my mom was just watching it over and over or if other people around the world were watching my video.

With this in mind, we built up a product almost a year ago; which we launched on March 26 of last year. We started with basic functionality that could give you information on my views over a certain period of time, maybe a month. On a personal note, it helped me figure out that my video was watched 50 times by my mom in California, but it also got a lot of views in Spain and the UK. So it was really interesting, because you could finally see where your audience was coming from and what the lifecycle of your video looked like.

On top of that, we built a feature called Popularity, which analyzes how your video's performance compares to other videos. You can see how well your video performed on any given day relative to all the videos within YouTube or within specific geographic regions. And what is really interesting is that what we found our businesses were starting to use this basic data in very interesting ways.

The obvious value was in understanding the lifecycle of your video, and on what days of the week it was most popular. This can help content owners really start to own their program strategies on YouTube. If they get most of their views within the first three weeks, for example, serial content providers could start uploading their videos every three weeks. Then they could maximize the number of views that they get on YouTube.

Another interesting phenomenon we observed was that bands would put up concert footage or their new video clip, and they would have interesting pockets of audiences in different areas across the US. They'd actually start planning the touring schedules around them, because nothing is worse than scheduling a concert and having no one show up. But by having their content on YouTube, they could understand where the views were coming from and they could better plan their concert strategies.

Another really interesting use of the tool involves measuring ad effectiveness. This could save you money on promotional dollars within the YouTube ecosystem. You could really start to see the effect of specific advertising campaigns that you ran them and if you got the views and spikes you expected. Here is an interesting example: if you ran a homepage ad on YouTube, you would expect that the video that you ran the ad on would get a spike in views.

But, what we also saw was that all the other videos within that uploader's channel got spikes and views even if they put just one video on the homepage. So, you could really start to see the halo effects of advertising. Interestingly enough, you could also see the effectiveness of the different offline promotions that you were doing.

If you had a movie screening in Michigan, for example, you could see if that made people in Michigan start looking for your YouTube content, and then the halo effects of the surrounding states that potentially heard of it as well. So a lot of really interesting stuff is coming off of the first features we created for YouTube Insight, showing basic views trended over time and space.

A couple of weeks after we launched Insight, we added a discovery feature that allows publishers to understand how people get to their video. They can see if they found it through a search on YouTube or Google, or if it was an external link that they found somewhere on the web.

It may be an embedded video across the web or a part of the YouTube site that drove traffic back to your video. Now this is actually pretty obvious, and again there is an opportunity to devise optimization strategies around how people find your content. For example, if there were blogs that embedded your video, you could reach out to them and form business relationships.

One of the interesting stories that we heard involved the band Weezer. Weezer debuted one of their videos off of their latest album on YouTube, and what they found is they got almost 2,000,000 views within the first couple of days, which is a fantastic performance. When they looked in Insight, they found that a lot of those views were actually driven by tech blogs such as Valleywag and TechCrunch, which was a big surprise to them.

So what they did with this information was actually more interesting than the information itself. The single preceded the album release, so when they were promoting the album release and their tour, they actually spent a lot of their media money on tech blogs since they knew they were already established there.

Eric Enge: So they reached out directly to the tech blog, because clearly the tech blog had an interest in them at that point as well.

Matthew Liu: You can imagine all the types of relationships that you could form from that. Not only do we show you the sources of traffic, we allow you to drill down more specifically. So, for example, you can actually see the search terms that led people to your videos. We have a great promotional product called Promoted Videos which is basically Adwords for YouTube, that allows you to advertise against specific keywords. So you can have your search and your video results show up with organic search results on the site.

Again, Insight has proven to be a very powerful product, because now you can know which search terms are really effective and which terms were less effective. And the combination of the two really helps people start to find the audience that was looking for their content, whether they be advertisers or content providers.

Eric Enge: Right. So if you are a commercial entity that produced a neat video that you put on YouTube, you may want to buy advertising just to create visibility for your video. Then you could use the analytics functionality to see how that campaign performed.

Matthew Liu: Absolutely. Another really interesting thing about YouTube is that a lot of people just come to the site to be entertained. So for example, we get a lot of crazy, funny videos. You may find that the term "funny video" actually drives a lot of video views to a video such as Tea Partay. Because you now have access to this information, you can understand those general search terms that you may not have thought about before and really start to optimize. Insight is very real time. You can optimize in the middle of your campaigns, and it will really start to tell you what your strategies should be.

The next feature that we launched was the Demographics function, which basically shows you the makeup of your audience in terms of a sex breakdown and an age breakdown. This is pretty important to both advertisers and content providers, because they need to see if they are reaching their target demographic.

One of the things that we've realized about YouTube is that since it has such a massive audience, you can find any niche audience you want. An example we had was of a PBS producer who produced a show. He wanted to put the pilot up on YouTube, but the management at PBS wasn't really sure that YouTube was the right place, because they thought YouTube was geared towards a younger audience.

What they did was put the pilot up on YouTube and let it run for two weeks, and they found that actually 75% of their audience was over 35, which was their target demographic. So it really proves that there is an audience on YouTube for any type of content. We also found that people are starting to use the demographic information provided by YouTube Insight to close deals.

One of the most popular comedians on YouTube is a guy named Paul Telner. And he used the demographic information in Insight to show that he appealed to the right target audience and sign a deal with MuchMusic, which is Canada's #1 cable music network. Another example is Chris Bosh, who is NBA All-Star for the Toronto Raptors and also a member of the US Olympic team. Sharing information on his YouTube demographic helped him get a sponsorship deal with AOL Sports.

Eric Enge: You could view it from the opposite point of view, which is if you are a content provider who needs to decide who you want to target as a potential advertiser.

Matthew Liu: Absolutely. And we think people experiment with their content too. They put up multiple creatives to see what demographics these different creatives resonate with. It's using that focus group in a very, very controlled way, but it's very quick and free as well. And you have access to such a wide audience, so you can really see how things resonate within different groups.

The most recent feature that we've launched is Hot Spots. All the previous features focused around using aggregate geographic data, but Hot Spots starts to dig deep into specific views. It shows how your audience related to the video during playback.

What you basically see is a graph alongside your video, so you can actually play the video and see how your audience is responding second-by-second. If people dropped off, your graph would go downwards. If people rewound, you'd see spikes in attention. We also give you an overall attention score so you can understand how your video is performing relative to others.

We show this attention score and your Hot Spots graph relative to videos of a similar length. This is important because in a vacuum people drop off more and more as videos continue past certain lengths. It's an aggregate.

Eric Enge: Can you talk about exportable reports?

Matthew Liu: Basically we heard from our content providers, our power users, and our advertisers, loud and clear, that they want broad access to the data. So we have launched exportable reports. The premise behind it is that we want to give these power users the data how they want it, when they want and where they want it.

Exportable reports provides a lot more flexibility on top of the tools we already give you today. There are groupings of these videos that publishers want to look at that they are never going to tell YouTube. So, for example, if you had one marketing department focusing on a set of videos and another one focused on a different set of videos, there was no way to arbitrarily group those up, because YouTube had no way of knowing which individual works on each set of videos.

Now they can download analytics for the specific videos and then make those comparisons. Until today, we gave you discovery sources by geography and by time, but in order to see things over time you had to select different date ranges. So if you wanted to see the number of times a specific keyword was searched on a daily basis, you could do it in Insight prior to the release of this new feature, but it previously required some manual work, because you had to switch filters and things like that.

With exportable reports you can target the specific types of data that you want.

If it is a keyword term, you can select that, filter it in the list and then try and put it on a timeline. Or if you wanted to look at views from certain keywords versus having your video embedded on a certain blog, you can compare those sources side-by-side. There are a lot of interesting things that we have heard content providers and advertisers want to do with this data, such as plugging it into their own systems and comparing their advertising campaigns on YouTube versus those on the radio.

Now they are able to have that flexibility, and if they want to plug into a wider ecosystem, exports can take them a long way in getting there.

Eric Enge: Is the export a manual process?

Matthew Liu: Yes. It is a link and we provide it on a per-video and a per-channel basis. We are going to make improvements in terms of including more types of data and making it easier to access it, but we actually launched this feature very quickly from its conception. It was a 3-week cycle, so our goal was to launch it very fast, get users access to the features that we were promoting and then make improvements as we get feedback.

Eric Enge: Can you export any of the data in Insight or just specific things?

Matthew Liu: Right now, we basically have two reports. The first report gives you views, uniques, popularity information and engagement information. You can see comments, ratings and favorites on a daily basis by country and by video. And then the second report is referral data, so views by referral source are broken down by all the granularity that we have on a daily and country basis.

Eric Enge: That is some good stuff for people to pull out. They can combine it with their other analytics data as well.

Matthew Liu: Absolutely. We think that would be a great use of the exported data. We have heard some advertising agencies have their own internal reporting tools, and anytime that there is a reporting system that can plug-in, it makes them more efficient in terms of optimizing campaigns.

Eric Enge: Right, You can just export the CSV file out and then run their other tools.

Matthew Liu: Yes, absolutely. We are excited about this new feature, and we have received pretty good press from the blogosphere and from comments back on the YouTube blog where we can see people are finding it useful.

Eric Enge: Any comments you can make on plans to enhance the analytics further?

Matthew Liu: Insight had just two features when we launched a year ago, and now we have about six full-featured modules. So we are evolving very, very quickly. I can't speak specifically to features that we are going to be building up, but you can imagine there is a lot we can do with all the data that YouTube has. We display a lot of data such as engagement within the sites and how people are commenting on and rating the videos. You can imagine that expanding over a number of dimensions.

Eric Enge: And now YouTube has become the # 2 search engine on the web, so that really adds to the value of this data.

Matthew Liu: We are looking forward to helping people use the tool, because quite frankly we've been surprised about all the different use cases. Optimizing for search is a great way that people can enhance their experience on YouTube.

Eric Enge: Thanks for joining us today Tracy!

Matthew Liu: Thanks for having me!

Eric Enge: Hi Matt! Can you give us an overview of your role with YouTube?

Matthew Liu: Hi Eric, my name is Matthew Liu. I am a Product Manager, working alongside Tracy and others for YouTube advertising platforms. I am working on one of our newest launches, which happened at the end of last year, Promoted Videos. We think of it as the equivalent to Adwords on YouTube, as it is a paid Video Search product.

Eric Enge: From an optimization point of view, the first thing you have to do is produce content that is interesting to people who end up discovering it on YouTube, which sort of goes without saying.

Matthew Liu: Yes, absolutely. I think we've always had the philosophy at YouTube, whether it's talking to our users, content partners or advertisers, that whatever it is that you want to share should be good content. So when we speak to advertisers we ask them to try to make their advertisements videos that people would want to watch any ways.

By using our advertising products advertisers are able to help put a little bit of gasoline on the fire and allow it to spread more quickly and potentially become viral. Similarly, our content partners and everyday users trying to get viewership should really think about what the community is looking for in general at a specific moment. And they should really try to personalize their video for the YouTube community as opposed to simply just taking content that might otherwise have ran on television or some other medium.

Eric Enge: So I think there are a couple of key non-SEO things that people typically talk about. For example, advertising and allowing people to share your videos is a good thing to do. Also, making sure that the content in some way reinforces the brand rather than just being entertainment without purpose, so to speak. Allowing ratings and well selected thumbnails are also good promotional strategies as well, right?

Matthew Liu: Yes, absolutely. You touched on a couple of those things, such as ratings, comments and also on embedding. One of the larger paradigms is that a lot of people put content on YouTube and they allow themselves to engage in conversation with the community. Sometimes we see our larger content partners or advertisers shy away from that, because they are afraid of what comments and what ratings they are going to get.

Accepting comments and ratings may feel a bit more risky, but it definitely offers you very valuable instant feedback. So if we are able to get a couple thousand views and see what the ratings are and what people's comments are, it empowers you to make changes. And if you are getting positive feedback, not only is your video getting out there, but you are spurring positive conversation as well. So that's definitely one thing we recommend.

Eric Enge: I guess it gets back to the old social media lesson, the conversation is going to take place with or without you.

Matthew Liu: Yes, that's a perfect statement.

Eric Enge: The choice becomes very obvious once you think about it that way. So do you have any interesting case study examples of someone who used advertising as a way to really launch a successful video?

Matthew Liu: Yes. The first example involves OfficeMax, which is a large retail supplier of various office products. It is a traditional brand advertiser, with its own TV commercials in most cases, but they knew they wanted to do something a little bit edgier, with a potential to go viral.

They commissioned The Escape Pod to be their agency, because they wanted to do something much more creative. So they came up with interesting series of videos, the Penny Pranks videos, for their Back to School campaign. These involved a funny looking guy who would go to various places in New York City and try to pay for everything with pennies, and everyone would be outraged. He would try to buy a car with 200,000 pennies, or something similar to that.

They decided to use advertising to drive those initial views. They wanted to accelerate that and also as a byproduct increase the discoverability on organic search and on YouTube. So they worked with us using Promoted Videos and some other paid mediums.

What they found was they were able to get fairly efficient views, so they were very pleased with the price. They were able to get a ton of clicks, which drove a lot of traffic to their videos. And as a result, they started that viral loop. So over time, we saw that for many search query terms. On the organic side, for some of their target queries, their videos became the top search result.

OfficeMax actually was able to become so embraced by the community that our search engine deemed them to be the most relevant for that time period. And they also saw additional uplift on their other videos; not just the videos that they promoted from users watching and clicking on more from OfficeMax, but more views on the related videos as well.

They were very pleased, because they had a very successful campaign that they were able to conduct in a very efficient way. That's one major example where you can think of brand advertisers trying to efficiently drive traffic to their online videos, engage in positive conversation and even potentially engage in that viral spreading of video.

The second example that we can talk about regards a producer of consumer gadgets and products. During the launch of Promoted Videos they participated with us in producing a couple of videos that highlighted their iPhone 3G cases. The company is Zagg, and their product is called the Invisible Shield. It's an invisible, scratch-resistant film that goes on the iPhone. You could take a key or a knife to it and it will prevent your iPhone from being scratched.

So in the video they show two iPhones side-by-side, one with the cover and one without it, and they show the different results. When promoted against terms such as iPhone and iPod, it was not only able to drive traffic to that video, but ZAGG was able to convert the traffic into sales.

The amazing thing about it is that they were actually able to drive conversions at a cheaper value than they would have been able to do on Google and other competing search engines. One of the hypotheses we have is that for certain types of products where the user may not be as aware as to exactly what it is, being able to see it is far more compelling than just three lines of text.

Eric Enge: So what about the power of send to a friend, and other options for sharing?

Matthew Liu: There are a bunch of different sharing options, from sending to a friend, to embedding that video, to sharing on Facebook or MySpace, to even just copying and pasting the URL so you can go back to it later. So these all have various different positive benefits. I won't go into the details as to which ones we found most successful, but I think there is a reason why we encourage video distribution through different means beyond just YouTube, whether it's IM, Connections on YouTube or posting to third party sites. They definitely have a lot of positive values driving additional viewership and potentially even subscriptions. It just creates an overall deeper engagement.

Eric Enge: Let's get into more basic SEO kinds of things. Standard advice in the industry places a lot of emphasis on category selection, titles and descriptions, and the use of tags. Can you talk about that a little bit?

Matthew Liu: If you pull up a YouTube watch page, you'll see three main areas of tags that the user can input. We do have the title and the description tags just as you mentioned, but I think what a lot of people are missing when they use these three fields is comprehensiveness. A lot of times we see videos with very short titles, very short descriptions and somewhat erratic tags.

The first thing I would say is if your video has subtopics or a subtitle, include them in the original title, and include all the details in the description. We offer a lot of space where we usually type in all the details, and obviously we are indexing all those descriptions and tags, and they are going to be surfacing in both YouTube video search and Google video search. So it's important that you have comprehensive data.

Secondly, we would say be consistent. A lot of videos we see have a good title and a good description, but then totally random tags. So we actually do have measures that penalize this poor behavior. We recognize when videos are trying to spam, and that's actually something we penalize. So be consistent with your title description and tags. Make them clearly about that video and don't try putting unrelated keywords in any of those fields.

Another layer of video SEO is to make your video open. Allow it to be embedded and allow users to comment on it and rate it. We definitely do take user feedback as an additional ranking mechanism. This can hurt you if you end up getting a lot of negative ratings, but the positive benefit of getting higher ratings outweighs that risk.

Now let's talk a little bit more about engaging with that user. You mentioned the thumbnail which is probably one of the most basic things. Pick a thumbnail that is both representative of your video and engaging. Right now we will give you three thumbnails that we take from areas that we think are representative of your video, so any user that uploads a video should definitely take the time to find the best thumbnail.

There are some positive benefits to higher quality videos. Users may or may not care as much about the quality of the video itself, but because we are taking that thumbnail from the video, the higher quality of video will make the thumbnail a higher quality as well. And higher quality thumbnails are something that we definitely notice attract our users.

Eric Enge: Right. So you've got to care about the content and the quality of the thumbnail.

Matthew Liu: Absolutely. Then going further along with engagement, we've launched some features such as path annotations. These are becoming more and more powerful overtime, as they are an additional way for you to communicate with your users. We are able to put speech bubbles or links to your other YouTube videos.

Often times, savvy users do very interesting video tours where they link back to one another through different videos, or they even have games you can play by clicking on different annotations. It's interesting how you can create an extended cycle of viewership through annotations.

Then, rather than just interacting with their user base, they are also interacting with the rest of the YouTube community. So what we've seen is that a lot of successful people can cluster together. A lot of our top users have formed this community where they send video responses to each other, they comment on each other's videos and they subscribe to each other. So we definitely encourage people who are trying to get increased viewership to tag back.

We don't want to have people spamming or just randomly adding irrelevant videos as video responses, or comment spamming, and we definitely penalize videos that do these things, but when it is legitimate, posting video responses is a good way to network with other community members. Think of it almost as a message that you would get back on a social networking feed or a Twitter feed.

Just continue that dialogue with important members of the community. Often times if that original video does get traffic, then your video response may get additional traffic and help viewers discover you as a new source of quality videos.

Eric Enge: You get value by building relationships.

Matthew Liu: Yes, completely.

Eric Enge: Should people strive to avoid "stop words" in their titles. Similarly, should you include the word video in your title or description, so that if somebody searches on tech software video, for example, then you have a better chance of coming up. Do those things make sense as well?

Matthew Liu: Yes, they do. Especially in the context of discovery from Google, because Google also indexes YouTube videos. Another thought that I forgot to mention is if your video was shot at a particular location or on a particular day, then you should also include some of that information in the video's description.

Eric Enge: Another suggestion I've heard is to use adjectives such as happy or sad to pick up mood-based searches.

Matthew Liu: What I can tell you is that YouTube search and Google search are a bit different at times. It's not in all cases, but we have seen that some users tend to search in more generic terms,. So you'll see users searching for very specific pieces of content, such as "CBS video" or "NBA video". You will also see users searching for terms such as funny videos. What I would say is video owners should target both the very specific terms and they should also potentially broaden out a little bit so that there are more generic queries in the description and the tags.

Eric Enge: What's the best way to get a sense of the best keywords within the YouTube environment?

Matthew Liu: Great question. We don't have anything to announce for now, but we are working on various keyword tools. We have a couple of very basic keyword tools as part of Promoted Videos right now, which allow you to checkup similar keywords. The Insight tool that Tracy talked about also helps to understand keywords that are already driving traffic to your video.

We are working on a couple of other similar projects where we'll be able to have much more robust keyword suggestions in the near future. But in general, I would say use Insight and use the keyword tools that are already available in Promoted Videos, and those are probably going to be your best bet in the short term.

Eric Enge: I have also heard a suggestion that you go to the search tool when you start entering a query, and then the search suggestions that you can get there may be in volume order from largest to smallest?

Matthew Liu: I can't comment specifically about that. Those are suggested queries that we think users might be searching for as they start typing certain letters.

I will add a caution that publishers should avoid keyword stuffing because it's very easy for you to potentially broaden the scope for your video by adding a couple of keywords. But, it only takes one or two irrelevant keywords to trigger us to think that the video is trying to spam the system. Our penalties will outweigh the benefits you can get with keyword stuffing.

Eric Enge: But you did say earlier that it's important to be comprehensive, which means that you should include all the keywords that are in fact relevant (without putting too many total keywords), correct?

Matthew Liu: Yes, there is definitely a balance you have to find. It's actually more of an art than a science. Use keywords that are related, but don't type in every letter in the alphabet. Just come up with the most important relevant keywords and add all those words into your description and tags.

Eric Enge: It's got be highly relevant and something that people can search to discover your video, and then have a good chance of being happy when they get there. At a minimum, they get relevant content, even if it is not exactly what they are looking for.

Matthew Liu: Yes, absolutely.

Eric Enge: Thanks a lot Matt!

Matthew Liu: Yes, thank you Eric!

Have comments or want to discuss? You can comment on the YouTube interviews here.

Other Recent Interviews

Google's John Mueller Interviewed by Eric Enge

Thu, 14 May 2009 15:29:30 -0400

Published: May 11, 2009

John Mueller is currently a Webmaster Trends Analyst at Google Zurich. Prior to working at Google he became well known for his active participation in Google Groups and a variety of SEO forums.

Interview Transcript

Eric Enge: Can you provide me with your definition of cloaking?

John Mueller: The standard definition of cloaking is to show Googlebot something different than you would show your users. So, in a worst case situation, you would show Googlebot a nice family-friendly homepage, and when a user comes to visit that page, they would see something completely different.

Eric Enge: Like porn or casino ads or something of that nature?

John Mueller: Exactly. So if the user was searching for something and finds what he thinks is a good result, he clicks on it, and then there is nothing even related to what he was searching for on that page.

Eric Enge: Right. So that's clearly an extreme form of cloaking. There are many different levels of cloaking, and I'd like to explore some of those.

Some people, for example, may have a content management system that just insists on appending session IDs or superfluous parameters on the URLs. They may not be superfluous from the CMS' point of view because they are using the parameters to pull information from a database or something like that. And given the content management systems that they have, it's actually very difficult and very expensive to fix this problem at its core. So one solution would be to serve the same content to users and to Googlebot, but to modify the URL seen by Googlebot to remove the superfluous parameters and the session IDs.

John Mueller: That's something that we've seen a lot of in the past. We currently have a great new tool that can really help fix that problem without doing any redirects or without really changing much at all, and that's the rel="canonical link element. You can place it in the header of your pages and specify the canonical URL that you would like to have indexed. So you could take away all the session ID parameters or anything else that you don't need, and just specify the one URL that you want to have indexed.

Eric Enge: Right. And that's something that you announced with the other search engines just a few weeks ago, correct?

John Mueller: Yes, it's fairly new. It's something that not a lot of people have already implemented, and there are a lot of people who are already using it to clean up this problem. Crawling a website and finding many duplicate versions of the same content with different URL parameters such as session IDs can confuse search engines. Using this link-element helps to make it a bit clearer and can help to resolve this problem.

Eric Enge: So you basically implement the canonical tag on various pages and you tell people what the canonical URL is. If, for example, somebody has different sort orders for their products in the e-commerce catalogue (e.g. by price, brand, size, color, ...), you can basically point Googlebot back to the canonical version of the URL, it's supposed to behave much the same way the 301 redirect would, except for it does not actually take the user to the different URL specified? Is that a fair summary?

John Mueller: Yes. It's not a command that you would give a Googlebot, it's more like a hint that you would give us. One thing we've also seen is that people try to use it, but they use it incorrectly. For instance, they specify their homepage as a canonical for the whole site. And if we were to follow that as a 301 redirect, we might completely remove their website. So we have to take that information and determine if it is really a canonical for the other URL, or if the user may be doing something incorrect.

Eric Enge: And of course one way you could do that is by making sure the content on the two pages is identical.

John Mueller: Yes.

Eric Enge: So if you make a mistake and use canonical tag to send everyone to the home page of your site, presumably the content will differ from the other pages. And, as I understand it, the gold standard solution is to fix the problem at its core and not have to rely on the canonical tag.

John Mueller: If you can move to the cookie-based session tracking, then that would really help. But we know it's not always easy to change to a system like that. There might be a lot of money involved. So at least with this system there is fairly simple way to fix that problem.

Eric Enge: Right. So it's the backup plan that should be used if you can't fix it at its core or if it's just too expensive to fix it at its core?

John Mueller: Exactly.

Eric Enge: Yes, that makes sense. Now I imagine there are also people out there who served a different URL to Googlebot and its users before the canonical tag existed. Is that problematic?

John Mueller: I would suggest doing that for all new users who come to the site without cookies, instead of just for Googlebot. This way, if a user accesses an old URL that has a session ID, you can just redirect him to the proper canonical. That would treat users and search engines in the same way, and it would still help solve this problem.

Sites that are currently showing prettier URLs to Googlebot should not panic, as long as their intent is genuine and it is properly implemented. But I'd advise against this for sites that are in the process of a redesign or sites that are being newly created. Using rel="canonical" is the current best practice for tackling this problem.

Eric Enge: But if the system is relying on the session IDs, then it's there for a reason, right?

John Mueller: Yes, but usually most CMSs resort to session IDs if they can't access a cookie. So if you see that a user doesn't have a cookie, you can redirect them away from the session ID. And I think the important thing here is that you find a way that you can treat users and search engines the same.

Eric Enge: Right. You could use JavaScript to append your various tracking parameters to the URL upon the click. So that, in principle, is treating users and Googlebot the same.

John Mueller: Yes, but that really doesn't solve the problem, because there would be something that would happen within the site. But when the search engine crawls a site, they don't execute the JavaScript, so it would have to work with and without the JavaScript enabled.

Eric Enge: Right. So users that don't have JavaScript would of course be handled in an identical fashion to the search engine robots, and users who do have JavaScript would be able to benefit from whatever the tracking parameters are meant to give them.

John Mueller: Exactly. That's similar to using AJAX on a website. If you have a normal HTML website and you start adding AJAX components to that website, a user with a limited browser, maybe from a mobile phone or even search engine crawler, would still be able to navigate your site using standard HTML.But someone who has JavaScript enabled would be able to use all those fancy AJAX elements, and that would also usually generate slightly different URLs, so I think that's completely normal.

Eric Enge: Right. So, let's talk a bit about A/B or multivariate testing, which is something supported by Google's Website Optimizer product. It creates a scenario where users come to a page and some piece of code runs and decides what version of the page to show users, usually implemented in JavaScript. And of course the Googlebot will only see the one version, it won't see the alternate versions.

John Mueller: Exactly. So, the clue here is that the intent matters, as is generally the case with Google. If the intent is really that the webmaster wants to test the various versions of the same content, then that's no problem. And if the intent is there to show the user something completely different, then that would be on the border. You would have to look at that.

Eric Enge: I mean, you can always take any technique that was created with good intentions and find ways to abuse it. So let's say somebody is testing out four different versions of a key landing page on their site to see which performs the best for them. Maybe they are changing the logos and moving elements around, they might be changing the messaging a bit to see if one tagline is more effective than another, or they may be changing the call to action.John Mueller: If you are doing that with good intent to find the best solution for your users, and you are showing more or less the same content, then I wouldn't really worry about that.

Eric Enge: Say you have a graphic of some sort, an image file on your site that might be a menu link or a logo. And there are various techniques for showing the search engine's robot or any specific user agent's text instead of the graphic. What are your general thoughts in that area?

John Mueller: Generally speaking, if you do that with good intent and you more or less match the content up, then it's fine. So, for example, you could have a menu where you use JavaScript and graphics to create a really nice user experience with an alternate version that's in static HTML that might be behind the graphic menu then. If it matches up, that's fine. And if the home link has an alternate text tag, then that's fine too. But if you have a home link and alternate text that says, "click here to see our great cleaning products available in these 22 cities," then that's kind of sneaky, and not something that we would like to see.

Eric Enge: So, there are various grades of this, correct? One level is where the text matches up a hundred percent with what is in the image. And there is a notion of substantially similar, and then you could actually several more grades and have somewhat similar, and then completely different. And, I think you just highlighted an example that's completely different. So, an identical is an easy case, I think you already addressed that. What if something is substantially similar, but is not word-for-word identical?

John Mueller: I would say it depends on the case, but if you are not trying to deceive the search engine crawler or the user, then it's generally okay, but in general I would be cautious as soon as the content is not identical.. So if you have a link that goes to your homepage and it has a graphic of a house, then you wouldn't have to use house as an all-text. You could just say "go to homepage," or something like that, and it's fine.

Eric Enge: So again it gets back to the notion of intent that you've already raised?

John Mueller: Exactly.

Eric Enge: And, of course, one flavor of this is sIFR, which stands for Scalable Inman Flash Replacement. sIFR uses text input to render what is shown in Flash so it is guaranteed to be identical.

John Mueller: Exactly. Where we start to see problems is when a website has a completely Flash-based interface and a lot of different pages all on the same URL hidden behind it. Then it would be hard to include ten pages of HTML on a single page that match exactly what is written in the Flash file. So you have to find a solution for yourself there; how much really makes sense and how much you might have to cut back and just leave the basics in HTML and keep the bulk of your content in Flash.

Eric Enge: Right. And of course when you get to that scale, you are past what you do with sIFR, which is really intended for putting anti-aliased fonts on your page, which is a more limited technology. But I think once you get into the more complex situations, you can use SWFObject, correct?

John Mueller: Yes, it would be something like that.

Eric Enge: That technology doesn't guarantee that the alternate version shown in text is identical to what is in Flash.

John Mueller: Exactly.

Eric Enge: So it is open for potential abuse, but I would imagine that the policy again gets back to what you actually do and what your intent is in doing it.

John Mueller: Yes. And there are two other things that also play a role in that. The first factor is that we have started crawling and indexing Flash files. If you have a lot of content in your Flash file, we will try to at least get to that and include it in our search results.

The second is that there are still a lot of devices out there that can't use Flash. So if you have a website that relies on Flash and you suddenly notice that there are a bunch of mobile users who are trying to use their iPod, iPhone or Android Phone to access your website, then you would start seeing problems because they wouldn't see the Flash content at all. , And if the HTML content doesn't match up with what you are trying to bring across to the user, they will simply leave the site.

Eric Enge: One grade of this problem occurs when you try to implement something in Flash, but you are not going to be doing it with the intent of rendering the same thing that you can easily render in HTML. You are probably using it because you want to create a highly graphical type experience. It is not always the case of course, but certainly one of the things that's appealing about Flash is that you can create a really attractive visual experience. Say you have a man driving a fast car on the German autobahn, the Flash isn't going to narrate the course of the drive.

But in your text rendering of what is in the Flash, you would want to describe what is happening. For example, "it's a nice day and a man gets into his expensive car and heads out onto the Autobahn". So you are actually implementing text that isn't in Flash, but the content essentially is.

John Mueller: Yes, that's generally fine. If the intent is okay and it matches up so you can see that there is a car and a man driving on the autobahn, then that would be fine.

Eric Enge: So again, it is about making sure that you are pretty much rendering the same information so that there isn't anything confusing in the user experience? Like if you flip from one mode to another, Flash, JavaScript or AJAX enabled or disabled, so to speak.

John Mueller: Yes. If you can think about it from a user-experience point of view; if the user sees the HTML content in the search results and clicks on that page, does that match up what he would be expecting?

Eric Enge: So what about serving different content based on an IP address to address things like language and national or even regional issues? Just to think of a regional issue, the products that your customer base in Florida buys could be quite different than the products your customer base in Minnesota buys. So you want to serve up the Florida user one set of offerings and the Minnesota user a different set of offerings. John Mueller: That is something that I see a lot as a European user, because in Switzerland we have four different official languages, and as soon as you start using a web site, it automatically tries to pick a language that they think is right. They are wrong most of the time, and it is something that really bothers me a lot. So I guess I might be a little bit emotional about that.

One thing that I have noticed that you have differentiate between whether or not your content is really limited to a specific language or geographic location. For example, you have a casino website that you can show to users in Germany and in France, but you can't show it to users in the US. That's kind of an extreme situation, but in a situation like that you would still have to treat Googlebot like any other user that would come from that same location.

So if we crawl your website from the US, and your website recognizes us as an American visitor, then you should show us exactly the content that an American visitor would see. And it would be a little bit problematic if the website started blocking all American users because of legal reasons. So what you would do then is make a public website that everyone can access and then just link to your private website that has been limited to users in a specific region.

So, for example, you would have a general homepage that tells what your website does, gives some information and provides something that search engines can crawl and index. Then when users get to the right location they can click through to your actual content.

Eric Enge: So are you suggesting that if a user accesses that website from Germany, they come to some initial page and then they have to click further to get through to page they are actually looking for?

John Mueller: Exactly.

Eric Enge: So it is not acceptable to just simply serve them?

John Mueller: Yes, that might cause problems when Googlebot visits. The other problem there is that IP location and language detection is often incorrect. Even at Google, we run into situations where we think, an IP address is from Germany so we would show German content. But in reality, the user maybe based in France, and it is really hard to get that right. So if you try to do that automatically for the user, you are almost guaranteed to do something wrong at some point.

That leads to leads to the other version of this problem, where users in the wrong location can still access your website. And in a case like that, we would be able to crawl and index the website normally, but I recommend that you include elements on your website that help the user find the version of the website that they really want to use.

The important thing there is that you use different URLs for the different locations or different languages so that we would be able to crawl all of the specific content. So when I go to Amazon.com from Germany, for example, I have a little banner on top that says "Hey, don't you want to go Amazon Germany? We are much closer; we have free shipping". And that way, the search engine would still be able to see all the content, but users would still find their way to the right website.

Eric Enge: So this of course is a little bit different than the scenario where you implement a website at casino.co.de, or .co.uk, or .com, or .co.us, where you really are creating versions that are meant to be indexed in the local version of the search engines?

John Mueller: Exactly, yes.

Eric Enge: So that's a different scenario that someone could use if they wanted to.

John Mueller: I think the key point is whether or not users are allowed to access the wrong version of the website, or if there is a legal reason why it is blocked completely.

Eric Enge: So if the legal reason isn't there and it is just that you want the default language that a German user sees, and you are willing to accept the fact that you are right about 90% of the time and you are wrong about 10% of the time, they can click the French link if they are really from France?

John Mueller: Yes. I think that the important part, especially with languages, is that you really provide separate URLs so that Google can crawl all language versions. And this way you also don't have to do language detection on your site. The user will search for something using a German or French-speaking Google, and we will show the French or German-speaking pages appropriately.

Eric Enge: So they end up in the right place through that mechanism? John Mueller: Yes. And you don't even have to do anything on your side. Maybe if you have a homepage you could show a little drop-down and let the user choose. Or you could have it pre-populated with the determined location by default, but you are still giving the user a choice between the different language versions. You give the search engine a choice and we will try to send the users directly to the right version.

Eric Enge: What are your thoughts on serving up different content based on cookies, such as explicit or inferred user preferences.

John Mueller: I think the general idea is also to make sure that you are not trying to do anything deceptive with that. Say, for example, you have a website where you just have general information. If a normal unregistered user comes there and you show that same general information to Googlebot, that is fine, because even a logged in user finds more information when he accesses the same URL. So if you make sure that it matches up with what a user would see, then that's generally not a problem.

Eric Enge: And since we are talking about cookies, presumably we are talking about a user who has been at the site before. So if they come back, their expectations may be for somewhat of an enhanced experience based on their interactions.

John Mueller: Exactly. So if you have it setup in a way that logged in users or users who have preferences get to see more detailed content, then that's fine in general. But if you have it in a way that users who were logged in see less content or see completely different content, then that would be problematic.

Eric Enge: Right. Can you give us an overview of First Click Free and what its purpose is?

John Mueller: We started First Click Free for Google News so that publishers could provide a way to bring premium content to their users. For example, if you have a subscription based model for your website, you could still include those articles in the Google News search results and a user who goes to those articles would still be able to see them and read that article normally. But as soon as they are trying to access more on your website, they would see a registration banner, for example.

Now, we have extended that to all websites, because we know not everyone can be accepted into Google News; it is kind of a special community. So if you have some kind of subscription or premium content, you can show that to Googlebot and to users who come in through search results. But as soon as something else is accessed on that site, you are free to show a registration banner so that users who are really interested in this content have a way to signup and actually see it.

Eric Enge: So the idea here is you have subscription-based content and Google wants to make its users aware that that content is there and it exists.

John Mueller: Exactly.

Eric Enge: So the user goes to Google, they see the article, they decide to go read it, the site implementing First Click Free checks the referrer and makes sure it is from Google, in which case they show the full article including all pages of a multi-page article, not just the first page?

John Mueller: Yes.

Eric Enge: And then the user potentially gets the registration banner when they go on to or a subscription box on a different article?

John Mueller: Exactly.

Eric Enge: Now, can a user just go back to Google and search on something and try to find that same article somewhere else in the search results?

John Mueller: Theoretically, yes. That would be possible, but we found that most users don't do that. It is more work that way, and if it is content they are really interested in, they will figure out a way to access it normally. When you like the content, you might say a subscription and say "Okay, this is a good website. I want to come back and read more of this content. It is fine if I just pay a small amount for it".

Eric Enge: I would imagine that for most subscription-based sites that it is an effective program to expose their content and increase their subscriptions.

John Mueller: Yes, absolutely.

Eric Enge: Exposure is really good. To do this, you basically bypass the login screen and give it access to all the content that you do want to index when Googlebot comes to the site.

John Mueller: Exactly, yes. I would expect that you could probably do the same for other search engines. You might want to check with them, but I think that is generally acceptable if the user sees the same content as the search engine crawler would see.

One thing that I have noticed when I talk to people about this is that they are kind of unsure how they would actually implement it and if it would really make a difference in their subscription numbers. It is generally fine to run a test and take a thousand articles and make them available for First Click Free, make them available for Googlebot to crawl and make them available for users to click on.

You can leave the rest of your articles blocked completely from Googlebot and from users. Feel free to just run a test and see if it is going to make a difference or not. If you notice it is helping your subscriptions after a month or so, then you can consider adding more and more content to your First Click Free content.

Eric Enge: Right. You can take it in stages. Are there other questions on these topics that you hear from people at conferences or out on the boards?

John Mueller: Another thing about cloaking is that we sometimes run into situations where a website is accidentally cloaking to Googlebot. That happens, for example, with some websites that throw an error when they see a Googlebot user agent. It is something that can happen to Microsoft IIS websites, for example, and that would technically also be cloaking. But in a case like that, you are really shooting yourself on the foot because Googlebot keeps seeing all these errors and it can't index your content.

One thing that you can do to see if that is happening, is to access your website without JavaScript, with the Googlebot user agent, and see what happens there. If you can still access your website, then that's generally okay. Another problem that sometimes comes up with language detection is that a website will use the same URLs for all languages and just change the content based on user or browser preferences.

The problem here is that Googlebot will only find one language, and we will just crawl the whole website in that one language. So, for example, we have seen cases where Googlebot was accidentally recognized as a German-based user, and we re-crawl the whole website in German and suddenly all the search results were only showing up to German users.

Eric Enge: So people in the UK couldn't see the UK-English version of the site, because, the Googlebot wasn't aware the content was there?

John Mueller: Users in the UK would be able to see that content, but since the Googlebot was recognized as a German user, it was seeing the content in German only. In this case, the old pages would be re-indexed in German, so if someone was searching for an English term, they wouldn't even find that site anymore.

The lesson here is to really make sure you have separate URLs for your content in different languages and locations.

Eric Enge: Right, for purposes of this example, we are assuming that the content is identical but translated.

John Mueller: Exactly.

Eric Enge: And, you want to have separate pages for the different language versions of the content.

John Mueller: Exactly.

Eric Enge: Excellent, thanks John!!

John Mueller: Excellent, thank you!

Have comments or want to discuss? You can comment on the John Mueller interview here.

Other Google Interviews

SEOmoz's Sarah Bird Interviewed by Eric Enge

Mon, 4 May 2009 12:57:23 -0400

Published: May 4, 2009

Sarah serves as COO of SEOmoz.org, Inc. and is also a law and technology blawger (don't be as dense as I was, "blogger" + "law") extraordinaire.

Prior to Sarah's tenure at SEOmoz, She worked as a litigator for a small firm in Washington State where she managed diverse and complex cases.

Sarah attended Simon Fraser University in Vancouver, Canada; Koç University in Istanbul, Turkey; University of Washington School of Law in Seattle, Washington; and East China University School of Law and Politics in Shanghai, China.

Interview Transcript

Eric Enge: Let's start by talking a little bit about a performance-based SEO agreement and how both sides of the deal should be thinking about it.

Sarah Bird: A performance-based SEO agreement can work out really well if the parties have a lot of trust and openness, and if there is a good contract in place. But before you get a good contract, there needs to be a lot of good communication. You've got to talk about all the details and all the expectations.

Obviously it's attractive for the merchant, because they don't want to pay for something if they are not going to see results. And they feel like they'll get better results if the person who is doing the SEO work has incentive to perform well. So, from a merchant's perspective, they are only paying for what they actually get.

From the SEO side there is a feeling that if you have a product or business that you really believe in, then you can participate in the upside driven by that. If you know you can do a good job and take them to a whole new level --reaching their customers and selling their product-go ahead and do it. The potential reward can be really, really tempting and really powerful.

So, there are some obvious reasons why these kinds of contracts seem like a great idea; and in fact they can be. I am a pretty conservative person myself, so sometimes I talk about performance-based agreements with a negative slant, but I do believe they can work. It's just there are so many things you've got to do first.

Eric Enge: We have actually done very well with them when we've done them. As an SEO consultant, the way I approach it is to determine what level we'll be working on and how much we'll be spending. For example, if we are working at a 10K per month level, we might take half of what we are spending and keep it as a fixed retainer fee, and then take the other half and put it into the performance bucket. And if we are successful, we'll look to double our money on what the base fee would have been.

So, if we take $30K in retainer, and we put $30K at risk based on performance, I want an opportunity to make $60K on that risk piece so we make a total of $90K. And, that's kind of where I think it really creates the big gain on the other side.

Sarah Bird: Exactly. Clearly it can work. What it sounds like you've been doing well is having a lot of those difficult conversations upfront about what both your and their expectations are. Another important part of communicating ahead of time is talking about when it will end; at what point do you no longer get commissions for the work you do? And I think a lot of people leave that part out of the conversation, and then it becomes difficult.

Many months will go by and the merchant will not want to keep paying commission forever to the people they have hired. If they are having success, they will want to keep some of that success for themselves. On the other side, SEO is thinking is that, "Hey great, I am just really starting to kick it into gear and just starting to reap the rewards that I knew I could get with all the time and effort I have invested".

That is an important conversation to have. I am curious--if you are comfortable talking about your strategies --how long do you typically do your performance-based contracts?

Eric Enge: We look for something like a 6-month time interval. One of the things that's very complicated about SEO. You never know when you are going to see the return. Sometimes you can get returns really quickly when all the chips fall the right way. But other times it takes six months before you have significant changes in trafficking. It really depends on where the site is currently, what the smartest strategy for them is and what it takes to execute that strategy so search engines will pick up on it. So, it gets complicated, but six months is usually pretty safe time interval.

Sarah Bird: I think that's a nice time interval. Like you said, you are sure to see some results by that point in time. And, the merchant is probably pretty comfortable with that time period, because it doesn't seem like a lifetime for them. I think that's really within the typical range of about four to ten months commitment level on any kind of paper performance agreement. Another thing I recommend is being very clear upfront about who owns the intellectual property that is being created.

That can even include your expertise on what should go in the title tags and what articles should be included. Sometimes you are even writing the skeleton of the article, if or actually fully creating the content. And it needs to be clear who owns the domain. Those are important questions.

Sometimes SEOs even go in and set up the whole domain. They setup the analytics programs, they do everything, and then two months later the merchant tries to boot them out. The SEO has put in all this hard work and gotten the chance to get rewarded for it. I think that's an important discussion to have for clarity ahead of time.

Eric Enge: But my main concern from that perspective in contracts is that there aren't any restrictions on any SEO techniques and methods that are inferred by the contract. If we use a novel new technique to help the client out, we certainly would agree to not reveal any details of their specifics. But, as a concept the piece of SEO expertise we have to own, because that's what we do for a living.

Sarah Bird: Yes, exactly. And I think the more you can set forth what you are counting as your SEO expertise ahead of time in conversation, and hopefully in a contract, the better both parties will be. If their expectations are set, everyone is going to be happier in the end. And if you do have to unwind early, or even if you are just unwinding on time, there are no surprises about what you are taking with you.

People who are just beginning to do SEO on this performance-based level tend to overlook this. It is really critical to have that conversation. I really like that you guys are doing a base monthly retainer, because then there is no complete loss to you for the investment and the work you do. I think a lot of the SEOs have that experience where they give advice, they have great ideas, but they are not in-charge of being able to implement it for whatever reason.

You can give all the advice in the world to a client, but if they don't implement it, or if they implement it the wrong way, then you may not get any good results. It's not something the SEO always has control over, so I think that idea where you have a base monthly retainer, that's at least going to cover a certain amount of your cost and time. It's a great way to mitigate the risk of loss compared to a solely performance-based SEO agreement.

Eric Enge: If you make a set of wonderful SEO recommendations, and it takes them four months to implement it,

where does that leave you? That's certainly one reason why we always take some of the money in hard cash rather than put it all at risk. The other way to handle it is to provide the web development resources in the contract. Then specify that you have the right to make the changes you recommend, within reason, of course. This way you start to have more control, which matches up with risk in a very nice way.

Sarah Bird: I think that is exactly right. And I think any SEO company would ideally like to have the manpower to also be able to complete the implementation. And whether or not that's an option, or the degree to which it's an option, will really depend on the clients and what kind of business structure they have. So yes, I think that works really great, especially for smaller businesses or for businesses that are just starting out.

They are much more likely to want an expert to takeover all of there web development resources. But if we are trying to go to some huge site that already has tens of thousands of pages, there is no way you could do the web development for a site that size.

I also want to point out that you have to have a lot of trust in whoever you are working with, especially if you are new to the industry. But trust is not a replacement for a contract and people shouldn't be shy about requesting a contract. If you are asking for a contract, it doesn't mean that you don't trust them, it means you want to make sure everyone is on the same page and stays on the same page for the time period that you'll be working together.

Because you are working with a business, they could always have people coming in and out of roles. New people may not have the same institutional knowledge because they weren't there when the agreement started. I've seen agreements, performance-based agreements, that begin well, but then go downhill when someone new comes in.

If you don't have that kind of contract written down for everyone to rely on, you can get into trouble even when you wouldn't anticipate that in the beginning of the relationship.

Eric Enge: I totally agree. When you are trying to build rich information resources to help establish the website as a leader or an expert in its field, you may go off and create lots of different valuable content. So when you do that, one of the things you do, especially if you are working on large websites you start trying to create lots of valuable information, and you end up referring to other things on the web.

You may start referring to information that's been published by others on pages that maintain a copyright notice. So, there is a concept called "fair use" that is applied in situations like this. Can you outline what it is, how it works and how they should interpret it in terms of not taking risks with the way their content is produced?

Sarah Bird: I think this is a great topic, and it's a legally hairy one because there will never be a crisp definition of what's fair use is. On one side of the coin, there is this feeling that, if you created the content, you should be able to own it. I think everyone agrees that we want people who make good content to have a good livelihood. But I also think everyone agrees that this product-=information--is good for society--its dialogue, its ideas, its speech and in how human beings can advance through it.

There is this natural tension between our core beliefs. We say people should be compensated and deserve to preserve the value they create, but then, we all want the information to be spread around freely and openly. So fair use is how we resolve this problem, and it basically says you can use other people's' content as long as it's fair. You can't steal that content, you can just use a certain amount of it in certain context.

We are going to call that fair and legal. It's okay to do it a little bit, because we want to share the idea. We just don't want to totally rob or steal from the original content owner. So there is no clear definition for fair use. It merely evaluates the factors of context, the purpose and character of the use, the nature of the content, the amount of the work used in relation to the whole, and the impact on the potential market.

The first factor is the purpose and character of the use. If you are using someone else's content to make money off it, that looks a little worse and feels a little less fair than if you are an academic professor and you are just trying to discuss an idea in a classroom. That's a noncommercial use of ideas, but the situation isn't always that clear and the decisions are generally not made that easily. You can talk about it, put it in practice, but somewhere along the line it can always get blurry.

The other factor is the nature of the content. If it is scientific, biographical, historical or factual, then the public has a greater interest in accessing the information. And because the goal of their use is to encourage progression of knowledge, the more academic or scientific the content is, the more likely the court is to say that it' is fair use.

Eric Enge: Isn't there also this notion that the things that are purely factual and available from multiple sources are fair game? Information that can be used very broadly, like the fact that Seattle is in Washington State.

Sarah Bird: Exactly.

Eric Enge: But, you still have the notion that if somebody renders the data in a specific way, then their specific rendition of the data can still be protected.

Sarah Bird: That's where it starts to get blurry. Everyone can agree that the fact that Seattle is in Washington should be fair use because we want to spread knowledge. But on the other side of that tension, you always have to be aware of the content factor. Whether it's a song, some sort of graphic an artist has created, scenes from a movie or a product review, these things are not as factually based and not actually based in science or biography.

They contain opinion and creativity that's more highly protected. One of the questions you should ask yourself, among many, is if it is just a fact or if it is somebody's creative work? If it's a fact you are probably more in the clear than if it's somebody's creative work. That can become a little blurry sometimes, but overall I think we hit the nail on the head with your Seattle example. No one owns that, so you will be fine using it.

Another factor is the amount of work used in relation to the whole. For example, when they launch a new movie, they want you to come see this movie. In the advertisements they often take extracts from critics who've written reviews, if it's "two thumbs up," or if it's "fantastic," or whatever. But you'll notice they only take a couple words. They don't take a whole article, they only take a few things and that is considered fair use because you are not taking the critic's whole work. You are just taking bits and pieces.

If you are online and you are reading a great Search Engine Journal piece on the Google algorithm, we can look at that article and we can take a sentence here and there and then talk about it on our blog, and then expand upon it with our own ideas. So we are interacting with the ideas on Search Engine Journal. Now, we can't take whole paragraphs or the whole articles and republish it on our blog, because that would be too much and that would be unfair.

But, you can have a dialogue and you can extract little blurbs from it. How much is too much will depend on how large the article is overall, and how important the piece of information is that you took.

Eric Enge: The related question is the role of citation. How does referencing the original work affect the equation?

Sarah Bird: Fair use technically has no laws requiring you to provide any citation. For example, when you are making a mash up of various things you've found on the web, you don't need to cite every single one of them. You can create a new product out of bits and pieces about their products with no problem. But if you want to be a participant in a dialogue and you want to help create that impression that you are using this content to engage in a dialogue and to spread ideas, that's going to help your fair use in a sense.

You are basically saying "I am participating in the communication of ideas. I am not trying to steal someone else's glory or content for myself and I am not trying to ignore them. As a matter of etiquette and as a matter of participating in a community, you can give them credit somehow. But citing it or giving attribution is not required by law.

A judge isn't going to care whether you linked to them or just cited their name. They are not going to ask you that, but as a good net citizen it's something you should consider. You can just pat the other person on the back and tell them you really appreciate the article they wrote.

As SEOs I think we're more aware that it's vital for the other person to get that link back. But I would be misleading people if I told them that, the law says you have to link back when it doesn't actually say that.

Eric Enge: The law doesn't care basically.

Sarah Bird: No, the law doesn't care about that, but it is certainly one of the elements of being a good net citizen.

Eric Enge: Right. So, you are putting yourself in a position where they are less likely to get angry with you.

Sarah Bird: Yes, absolutely. It can help make it clear that you don't have bad intention, but you really are just trying to engage an idea. There is one final factor we should discuss, which is in some ways the most important factor because it sums up the others. That last factor is whether or not the content you are borrowing from someone else has an impact on the potential market for that content. For example, I can't go and republish someone's book in it's entirety on my website, because that would mean I've just stolen part of the market from the original author.

It has an impact on his market, as people can get it from me so they're not going to buy it from him. I think that sort of sums up what we are balancing on all the factors. It's important to think about whether or not taking someone's song, graphic or part of an article, impacts their market and their ability to get value out of the product they created.

Eric Enge: Sometimes it goes further and people actually steal your content.

Sarah Bird: Yes, it does happen. It happens all the time, where someone takes your content, they take your whole blog post and republish it on their site, and that's obviously not fair use. But there are several things you can do, especially if you are in the US or in some jurisdiction that's covered by the DMCA. The first, and I think best, suggestion I make is to at least try to contact the website owner and say "hey, you are stealing my content, take it down." Start by having a professional conversation with them.

Some people roll their eyes when I say some of that stuff, because there are websites who know they are stealing content. They don't really care that they are stealing content and they are not going to care who contacts them, right? But other times it is a genuine misunderstanding of how the online system works. There are people out there who think that they are not stealing as long as they link back, but that's not true.

Those are people who think giving attribution is enough, but it's not the same thing. The first thing I think is to try to contact the website owner. Usually they will have contact information on the website, but if it's just a scraper site, it's unlikely they are going to have any contact information on their website.

You can also try to get their who is information, and hopefully that will; be accurate. If it is accurate then they are in the US, and they are more likely to respond to an email. But if you can get an email address for them, that's a great sign already that the request will be effective.

The important thing about the DMCA is that it does allow you a quick way to get the content that someone's stolen from you out of the search engines. The whole idea is to protect your content, and obviously you don't want someone to go to the stolen content, you want him or her to go to your content. So maybe you can get the person who actually runs the website to take it down, but if that doesn't work you can get Google to take it out of their search engine results so that other people can't find it. And, that's sort of the idea behind a DMCA request.

If I am a copyright owner and I have a problem with someone stealing my content, I can send this letter to Google and let them know that. If you do that, Google contacts that website to let them know that they are going to take the content out of their index. So if you are doing work for a client and they get one of these messages from Google by mistake, you can actually respond to Google and tell them that you got this letter saying you were stealing content, but it's not true.

Then Google, who has to be the middleman, will put it back up. It's a really quick way for copyright owners to try to address this online stealing of content, but it's not a full trial or anything, it's just an incentive for search engines like Google to take content down quickly. And if you've been accused of stealing content, they'll also give you this really quick and easy way to get your site back up. Again, without a trial, without a jury, you just have to contact Google and tell them they have been mistaken, and swear it is your content.

Now if there is still a disagreement, the content owner is now responsible for basically filing a lawsuit, because you've got to protect your rights in court. So, the DMCA is the first step, and it works for most people, but, you may have to file a lawsuit if the other website claims it is their content.

Eric Enge: Any experience with how quickly the DMCA requests are responded to by Google?

Sarah Bird: I've had good luck with those actually. I think they have fourteen days by statute to respond to DMCA takedown notices, and I had great luck getting into that window. We are talking in matter of days, but I have heard other people say otherwise. I don't know what makes the difference, and I am just guessing, but maybe they didn't send the takedown notice to the right place or they didn't sign it.

There are few things that have to be done for the process to work smoothly. You've got to sign the request, you've got to swear that you are the owner and you have to be acting in good faith. So it could be that people who are not having success maybe are missing a step. But I am not really sure.

Eric Enge: Right. So, another thing that we usually recommend as an interim step is to contact their hosting company if you've already tried and failed to contact the site owner.

Sarah Bird: Yes, exactly. Hosting companies are also required to respect DMCA requests. You can do the same kind of process with asking them to take it down as we just discussed with regard to Google. So yes, I think that's another great option.

Eric Enge: And if the hosting company doesn't respond, then they have some liability in this situation?

Sarah Bird: That's exactly right. That's how the law gives incentive to the search engines and hosting providers to act quickly. If they act within a certain period of time, then they can't be held liable for any of the possible infringements taking place. So, it's a great incentive. It's not a perfect system, but I think it's a pretty good one.

Eric Enge: Well, and if you work the system effectively, then you can protect your rights reasonably effectively.

Sarah Bird: Definitely. And I would say there are only two caveats. One is if you are not from the US, but you are trying to use the DMCA to takedown a site or get a takedown revoked, you should probably think twice about doing that. Because foreign companies may not be subject to the laws of the US, but if you use the DMCA process, you are agreeing to be subject to the US jurisdiction that covers our copyright views.

It's a technicality, but foreign companies and people may not realize that they are agreeing to US jurisdiction if they use this process. It's not just an administrative thing, it's an actual legal process. The second thing I wanted to highlight, which probably goes without saying, is to definitely make sure it really is your content when you send a takedown notice. Don't ever use the DMCA process abusively, because there are really severe financial penalties for abusing the takedown process. So if you send one because you are just trying to get a competitor's site taken out or whatever, you can get in some serious trouble. Make sure that you are not just guessing that there are some copyright infringements, definitely make sure you are acting in good faith and it's your content.

Eric Enge: Right, The penalties are actually pretty stiff.

Sarah Bird: Yes, they really are.

Eric Enge: Another interesting thing that happened in the industry recently is that the Federal Trade Commission made some changes in the rules about self-advertising.

Sarah Bird: Yes. It is debatable whether or not you would actually call them changes, but they are being perceived as changes within the SEO and online marketing world. There have been rules for print advertisers and for television advertisers about stealth marketing techniques and about substantiation in place for a long time.

You have to be able to substantiate any claims you make about your products; you can't just say "Ours is better than Jesse's," or "This will make your kids pay better attention in school." You can't just make those kinds of claims, you have to be able to substantiate them. For a long time it wasn't clear whether or not those same standards of disclosure about substantiation applied to the online world as well. So finally, the FTC has come up with some guidelines that say they indeed to apply to the online world.

This should just remind people who make a living on the web to know what's okay and what's not okay in advertising. Let's say you are an affiliate marketer and you are doing some advertising campaign work for some sort of a vitamin company. And say they tell you to put in the advertisement that the vitamin will make people lose 30 pounds. You can't just to put that in there without being sure it's true, because you have to have some ownership and responsibility for the claims you are making.

You actually have to go to the manufacturers and merchants and ask for proof for these claims about the product. And if they can't give you any proof, then you shouldn't run that in the ad because you will be on a hook for those claims.

Whether or not the claim is true is not the issue, the FTC is more concerned about whether or not you did your homework and got proof. Advertisers have actually made claims about products, and then the FTC has come in and say "Hey, where is your proof about these claims?" Then the advertiser would say " we don't have a proof right now, but we'll get you some". And then the advertiser does some research, and it turns out their claims were in fact correct.

That is not good enough for the FTC, you can't just make claims and hope they prove to be true once you finally got around to substantiating them. You need the substantiation ahead of time. I think that's an important element. You can't just go with your gut feeling, you have to actually substantiate it in advance of making the claim. And if you are the marketing agency, online or not, you have to make the effort to see that proof before putting the advertisements out on the market.

Eric Enge: Right. So, let's ask a question about a specific scenario. Say you are an agency and you go back to your client ask them if they can substantiate that 74% of people in the blind taste test picked their cola product over the other cola product. And then they come back to you and send you a nice email that says, "Yes, we did a blind taste test." But they choose to omit the fact that the blind taste test was done directly outside their headquarters' building. But they ran this test and you as the agency receive their email saying that they had in fact ran the test. If you have this kind of documentation have you done your due diligence at that point?

Sarah Bird: Well, you are very close at that point. The confirmation that just said, "Yes, we did substantiate", is not quite enough. You have to ask them to send you the documentation next. They should have that substantiation on hand, it shouldn't be news to them that they actually have to prepare some kind of document saying that they did substantiate their claim.

If they tell you that it's not really something they are prepared to show anyone, that should be a red flag to you. Verbal confirmation that claim is substantiated is not enough; you want to see some sort of evidence that they did it.

Eric Enge: Sure. There are certainly a lot of businesses out there that have tiny internal staffing that have grown by being very aggressive about their marketing. But, if your visibility goes up, your exposure to these kinds of issues goes up.

Sarah Bird: Absolutely.

Eric Enge: For example, you have all these social media networks that tend to grow very quickly. I don't know if they do that much advertising, but if they did, and they made claims, they may not have had time to do any substantiating.

Sarah Bird: Right, and that's just risky according to what the law and the guidelines say. Now of course every business owner has to decide his or her own risk tolerance level. Some people are going to say that the FTC gets around to validating so few companies' claims because they don't have the resources to check everyone, so chances are pretty good that they won't check me.

That is true in a way. The FTC is only one agency and they've got a lot of stuff to do, so they are not going to check everyone. So some businesses are going to be willing to take that risk. I think the risk tolerance level is highly correlated to how big of business it is and the kind of presence it has with consumers. Big businesses just need to be more attuned to this risk. There are just all kinds of factors to consider in deciding what level of risk you are okay with. To summarize, the FTC definitely made it clear that online advertisers are also held to substantiation rules.

Another thing they made clear is that online advertisers are also covered by rules about so called stealth marketing. This basically means that if it's not obvious that the advertiser is being paid to do this advertisement, there needs to be a disclaimer saying he or she is in fact being financially compensated.

Like Tiger Woods advertising golf balls, we all know he is being paid to do that, so they don't have to put a little disclaimer up there saying "Tiger Woods received money to do this". However, there are more subtle things, like if you are going through a chat room and you leave a comment on a blog about a product and link to it, it is not necessarily obvious that you have a financial association from the merchant that would require disclosure. It's considered to be material to the consumer's decision, because they never really know how much to trust this person and what they are saying about the product.

The consumer's perception is going to change if they realize that that person is being paid by the company that makes that product. So if you are an affiliate for someone and you write a nice post talking about how great of a product it is, you should disclose somewhere on your site that you make money every time someone buys the product you are talking about. I think that has people really scared.

In the online marketing world right now, there are people who think that that's really not a fair law. I personally think it maybe makes marketing much more difficult, and I feel like it hampers peoples' creativity, but it's probably a good thing for consumers to know if someone is being paid to talk about a product. So it's uncomfortable, but it's probably overall good for e-business. It will increase trust in the marketplace, which I think is always a good thing.

Eric Enge: Right. And when you look at that from Google's perspective and their effort to address paid links, what's being said here is not only is it Google's position, but it's the FTC's position as well.

Sarah Bird: That's exactly right. And you'll notice that that's why Google had to move to adding the words "sponsored-links" on its SERPs page through that same idea. We can't just say that these all happened to get to the top of the page because they were the best matches to your search, because people actually pay Google to put these here.

Google is kind of ahead in the game in that regard. The EU had a law come down specifically addressing stealth marketing and word-of-mouth marketing last summer. They had a very similar disclosure requirement about the financial relationship between marketers and products. So the FTC is actually little behind on this, which I think is causing a backlash now. People have gotten used to marketing in certain ways, and they are kind of getting a rude awakening now. You've got to disclose who you are to the consumer, make the relationship clear.

Eric Enge: Thanks Sarah!

Sarah Bird: Thank you Eric!

Have comments or want to discuss? You can comment on the Sarah Bird interview here.

Other Recent Interviews

Omniture's Bill Mungovan Interviewed by Eric Enge

Tue, 7 Apr 2009 09:38:54 -0400

Published: April 6, 2009

Bill Mungovan is currently the director of product marketing at Omniture responsible for their SearchCenter product. Previously, he helped build the search engine marketing practice at Carat, working directly with agency clients in exceeding their ROI and branding goals through search.

Mungovan brings a broad range of skills in search advertising optimization, account management, search directory development and search content production to his role. He previously served as Director, Client Relations at LookSmart where he was overseeing the day-to-day operations of the Account Management and Customer Service teams.

Prior to joining LookSmart, Mungovan worked for Snap/NBCi in the Search and Directory space, and also worked at CNET in San Francisco.

Mungovan has shared his search expertise as an invited panelist at several industry events, including Search Engine Strategies in New York, San Jose, Chicago and Dallas, PubCon and OMMA West. He holds an MBA from the University of San Francisco and a B.A. from the University of New Hampshire.

Interview Transcript

Eric Enge: Let's start by talking about a basic overview of Omniture, and then move into an overview of Omniture SearchCenter.

Bill Mungovan: Omniture is a general value proposition to the market. Our Online Marketing Suite includes our web analytics tool, SiteCatalyst. It also includes 9 other products, such as Genesis, an integration tool that pulls in data from other sources and Test&Target, a landing page optimization and a multivariate testing tool, and of course SearchCenter.

Eric Enge: Is Test&Target based off the acquisition of Offermatica?

Bill Mungovan: Yes, Test&Target is basically based off of the Offermatica technology. It is a dynamic Landing Page Optimization with Multivariate Testing on its landing pages.

SearchCenter was basically built in the context of that marketing suite. SearchCenter is what we call a search management tool, in that it accesses each of the major search engines from a single location and provides automated bid management and portfolio optimization. You can access all sorts of different reporting functionalities through the SiteCatalyst integration.

We think about SearchCenter as a tool for search marketers, but given the fact that we have Genesis, we can pull data in from other sources, like an email provider, an ad-server, a CRM system like SalesForce.com or a client's custom, internal database.

Eric Enge: Right. Pulling in data from other sources is one of the big challenges with bid or campaign management. People are so used to treating everything like they are direct response marketers. But a company that has physical locations, and a web site, is likely going to have interactions with people going to the web site and buying offline, and vice versa. So, being able to pull in data from other sources allows you to credit those campaigns in a meaningful way so that you can more effectively manage your bidding strategy.

Bill Mungovan: Yes. That's the heart and soul of the way we think about search, which is obviously the hot topic. We use SiteCatalyst to collect all of that data. So, in your example, you could pull in point-of-sale data or data from a call center. There is really no shortage of examples there. Then we can generate bid rules and bid strategies based on that data.

That's how Omniture thinks about the world, given the fact that we have SiteCatalyst as an underlying platform. We can pull data in from all these different sources, and then use that data not just for attribution, but also to improve bid strategies.

We have clients whose web sites generate more sales over the phone than they do on the actual site itself. Say they sell complicated items that people want to talk through over the phone. We need to be able to tie back exactly which keywords led to sales over the phone, and how much those sales were worth. So, it's not just attributing a sale to the correct channel, it's actually determining bidding based on that data.

Eric Enge: You made reference to portfolio management, and there is an aspect of that that I'd like to dig into a little bit, which is the notion that if you are bidding on a very high volume keyword it's really easy to get enough data to make decisions about whether that keyword is profitable or not.

But, we have the long tail, where the data is scarcer. It's maybe only a few clicks a day, or maybe it's a large pay-per-click account that has hundreds of thousands of keywords that get a few clicks every a week. So, by portfolio management, do you mean a strategy for looking at those keywords in a more holistic group fashion?

Bill Mungovan: Yes, that's exactly what it is. It's just an option for us to have two types of bid management in the system. One is the bid rules, which are basically just if-then statements. So, if you are getting this much revenue from a keyword, then you should increase the CPC by a little bit. But we also have portfolio optimization on the other side, which is just another option for marketers.

We found that having both presents more options to our advertisers. Now, with respect to the question of not having enough data to actually understand what's happening on a keyword by keyword basis for long tail keywords can happen. That's the biggest fundamental problem with portfolios of keywords.

I think the portfolio optimization approach does not have enough data, and our tool projects it out based on what limited data we have and what we think may happen in the future. If there is no data, there is only so much we can go on. After a certain point, we just assume that that keyword is just not going to generate any clicks. But, that's one of the problems that we see. We do mathematical projections for the future based on the limited data that we have.

What I am getting at is that our approach to search is the opposite of complicated mathematical Black Box formulas. We also have that built into the tool, we just don't believe fundamentally within Omniture that you can click a button and your entire search marketing program will be quickly taken care of. That's a Black Box approach that we feel has run its course in the market.

We just don't believe that there is any single approach to bid management or search marketing that's going to work for many different clients. It speaks to the broader vision in which we view search, which is that we are not an agency, but we have agency services within Omniture.

Our goal is really to be as transparent as possible to our customers. Transparency is a key issue for us, as we have many clients who have us manage their search program for a very short period of time, about three to six months. Then we coach them along the way on how and what we are doing.

We get them up to a certain level of performance, and then give that over to them in-house. So, not relying on service revenue the way an agency might works to our advantage, because we can give full transparency to our clients. And that model has been working pretty well for us.

So, to get back to the portfolio question, the idea of us being able to take care of the whole thing for you is just impossible in our mind.

Eric Enge: Right. Can you give me a set of things that you are managing? There may be ten keywords that are producing great volume, another fifty that are producing marginal volume and then some that produce less than 10% of the volume of the high-volume keywords. You still want to be able to manage those less than 10% keywords at some level, correct?

Bill Mungovan: Yes, and that's an area where you would apply different rules to the different types of keywords. One thing we tell clients a lot is there is no faster way to lose money in search marketing than to set up the wrong portfolio or to really have poor performing keywords dragging down the average of some of your best performing keywords.

Similarly, you wouldn't necessarily want your highest volume keywords in the same portfolio as your lowest volume keywords. You may, depending on what the keywords actually are. But you may not, and we want our clients to be fairly careful about how they set up the rules in their portfolios if they are, in fact, using that particular feature. You may actually give more credit to certain keywords because those at higher volumes are doing all the heavy lifting.

Eric Enge: Let's dig a little bit into the announcement you had recently with Scotts MiracleGro.

Bill Mungovan: In general, we are doing more and more deals within Omniture, both in the SearchCenter business and in other pieces of our business that involve multiple products. And Scotts is a good example of that, because essentially there is only so much you can do if you just think about search engine marketing as a silo. So, by bringing in data from other sources and using it effectively, we opened up a lot of different options for search campaigns. That's what Scotts is trying to do.

We don't have results for this particular example, just because it's a new announcement for us. But, in general they were having a hard time understanding exactly how email marketing campaigns could be used to remarket. And they also want to know how email marketing may have impacted or not impacted what happened on their site and what happened in their search marketing program.

Scotts was trying to take a more holistic view of their online business optimization efforts. They made the choice to stop thinking about email as one silo and search as a separate silo. And so, by using SiteCatalyst as their platform, they used Omniture Genesis to pull in the ExactTarget data, and SearchCenter for their search data, and measure it all in one place.

Eric Enge: Right. So interactions can be more easily understood.

Bill Mungovan: Correct. And, on a related note, they had problems with what they called Post-Click Behavior, which is basically visitor engagement and what happens on their site once they attract a customer. They've stopped thinking about email and search as just visitor acquisition tools, and started to think about the whole thing holistically.

They can see what happens when somebody clicks on a keyword and comes to their site, including where go, how much time do they spend on each page and what are they engaging with on the site. And they can do the same thing with email as well. Once someone opens the email and clicks through to their site, what they do and what is most important to them can be determined.

By using all those products in one place, and using SiteCatalyst underneath it all, Scotts was able to gain that level of insight. These are relationships that we are pulling together these days, because people want to start to look at online marketing more holistically.

Eric Enge: I believe ExactTarget is the email platform that Scotts is using, and there is an integration of data between email and the search campaign. What are some of the other data sources that can be pulled in and integrated in a fashion like this?

Bill Mungovan: Omniture Genesis is the name of the product that is designed to pull in data from third-party sources. So, ExactTarget is one of many, many email providers we know of.. There is also ad-serving data, which allows display data to also be pulled in.

Another very big category for us is CRM data. By tying actual backend CRM data to upfront advertising, or search engine marketing in particular, you can start to learn a whole lot more about what people do after the lead has been generated. You can also include call center data.

That can be anything that people do with an SAP or an Oracle database, any of those enterprise-level systems which may be point-of-sale data, such as data from a system of kiosks. There are really two types of data: marketing data including online data, such as email and display advertising, and offline marketing data. So, data from the television marketing or any kind of offline media can also be pulled in depending on how it's structured in whatever system it's currently in. That's the one side of the advertising data. The other side would be backend sales data, which is the CRM, Kiosk call center data and the other enterprise systems that may live in an SAP or an Oracle database.

Eric Enge: Right. And there has to be some pretty interesting things going on there to pull in CRM or call center data, which clearly can be massive in size.

Bill Mungovan: Yes, and we have a product called Discover OnPremise for when it does get too big. It's something we got from Omniture's acquisition of Visual Sciences. For example, we have a rental car customer who is trying to figure out exactly how many people book online. Then they'll go to each individual location around the country and observe how many people actually show up to pick up the car they reserved online versus people who don't. Then they see what people actually buy, how far they drive and all other sorts of data like that. As you can imagine, it just gets absolutely massive at that point.

So Discover OnPremise is a much more powerful and robust tool for when integrations get well beyond the needs of a standard advertiser. But, we do have advertisers who have millions of keywords in SearchCenter and are tying some of those actions back to the systems that don't have anything to do with what happens on their actual site. So, it starts to get pretty interesting at that point.

Eric Enge: Are there ways to create ties into TV advertising, print advertising and radio advertising?

Bill Mungovan: Well that's really the million dollar question that every advertising agency in the world is trying to figure out; exactly how does offline data impact online behavior and visa versa? And what we propose to people is to pull that data into SiteCatalyst, start to figure out your own correlations and, if possible, figure out the causality between different marketing programs.

For us, we just provide the repository for the data, and then we allow agencies and advertisers to actually start to figure out what is occurring on a campaign-by-campaign basis. But yes, you can pull that data into SiteCatalyst.

Eric Enge: What are some of the strategies for how you provide the data to SiteCatalyst? And what kind of data is it that you are providing in some of those more difficult scenarios?

Bill Mungovan: I believe CRM data is the right place to focus the discussion because it's just a little bit more tangible and measurable. For example, Omniture uses SearchCenter for our own marketing efforts in order to get more Omniture customers. A really common scenario for us would be to run an online advertising program and then generate a lead on a web site. But what actually happens to that lead, at least in our case, is that it then goes to a sales force.

The sales force follows it up, and some percentage of those leads actually turn into customers. We track it all the way down to how much we spent to acquire that customer, both online and through our sales team, and then we figure out what we've got in return for that. In our case the CRM system we use is Salesforce.com. But there are any number of CRM systems from which we can pull the data.

For us, Cost-Per-Lead is a pathway to one very small piece of the full picture, which will actually help us figure out Profit-Per-Click. So, if you are able to figure out how much you spent on all operating costs, you can pull that data in through CRM integration, and then actually bid on the keywords that lead to the highest profitability for your business.

Those are some of trickiest, but most interesting and most progressive features.

Eric Enge: Let's talk a little bit about some specific tactics. For example, you know your paid search campaign results in phone call orders. And one tactic you can implement to make tracking much more effective is to give everyone that comes to your search from a paid search ad a custom 800 number. This way you can know the results of your paid search campaign just based on what number they call into. That's a tactic that is designed to give you much more accurate data.

Bill Mungovan: Yes that makes sense. And another tactic that one of our clients is doing is automatically generating codes on the site itself. This way the customer can actually see that code, so each customer who visits the site from a given campaign will be identified. And we can actually get it all the way down to the keyword level. We know what keywords they came from that led to a call.

Customers see a certain code on the site and then make a phone call and either make purchase or not. Then we have the call center actually take that code in from the customer, so we can record where that customer came from and what they did on the site. Then we can pull that data back into SiteCatalyst and make bidding decisions based on what happened.

Eric Enge: You can also give customers that walk into a physical store a rebate as a part of some promotion that the store is holding. Then they collect the rebate by going online, filling out a form and plugging in the rebate number. Then the web site can check cookies to see if the person came in from a search campaign of some sort.

Bill Mungovan: Yes. But we wouldn't necessarily be able to tell what specific keyword they came from. It is still a good example, but we've actually seen the opposite happen as well. When people come online from a specific keyword they come through to a page and have to print out a coupon that contains a bar code with information in it such as the keyword they searched on, bring it into the store and then redeem it in the store.

What we can see there are two things; how many people print out the coupon and do not go to the store, and how many people print it out and actually redeem it. So that's just another way of understanding what's driving people to make offline transactions.

Eric Enge: Exactly. Try to discover every aspect of the interaction that you can.

Bill Mungovan: That's something really cool that we've seen with a retail client. It's pretty complicated, but it's very interesting at the same time. The client is able to look through SiteCatalyst as they are running geo-targeted campaigns. Again, these are big box retailers with stores in many different locations, and they are running different ad campaigns for the different geo-locations. And they can see what people are purchasing online and, more importantly, what products people are bundling together in a given geo.

So, there might be a video game and CD on sale in the upper Midwest, and that particular bundling may be very, very different from what people are bundling in Los Angeles. So what they've done is taken all the online data and figured out what products people are bundling online. Then they rearrange the actual placements in the store based on what's happening in that geo. So, when you walk into a store, you would see two products next to each other on the shelf based on what people are doing online in that geography.

Eric Enge: So you basically isolate the best way to put together bundling based on how people are behaving in different areas?

Bill Mungovan: Yes. We figure out what they are buying based on the digital shelf and apply that knowledge to the actual store and rearrange products accordingly.

Eric Enge: What would you recommend to someone running a TV campaign?

Bill Mungovan: It is absolutely critical to pull your TV data into the same system where your online advertising data is running. You should at the very least make sure that you are actually measuring apples-to-apples in one place. So, it's not an easy question to answer in terms of what TV campaign yields the highest possible return online. That's a very complicated thing, and it will be different for every customer.

But our advice to the market on that is to pull the data into the same place and then start to run reports on correlations between media in a given geo and what's happening online.

Eric Enge: You can also try things like Vanity URLs, but things like that are very uncertain.

Bill Mungovan: We have seen a lot of studies that tell us that very few people actually remember your URL address from the end of your television or radio ad, and even fewer go on their computers and actually type it in. For us it's more interesting to just let the campaigns run separately. So, you have your search campaign online, a display campaign, and then a TV campaign. Then pull the data into one place and use analytics to figure out the correlations.

Say you saw a bump on the 21st of January, you can find out exactly what media was running in which geo, and then you can start to make correlations between the two. So, I think that those tricks of Vanity URLs and things like that don't work in every case.

Eric Enge: Right. Well, I would think that there is a risk of actually lowering the actual return in return for trying to figure out how to measure it.

Bill Mungovan: That's right.

Eric Enge: Can you outline how the pricing model works for SearchCenter?

Bill Mungovan: We typically charge as a percent of ad spent so the more you spend, the lower the percentage. We have customers in all shapes and sizes. We've had clients take it in-house and then they just felt like they really couldn't handle it for a while, and they then requested the additional help of our services group.

So we manage it for them for a while, or on an ongoing basis, for an additional percent of ad spend fee. Then after a while we can give it back again when they are ready. We have some flexibility as part of that offering.

Eric Enge: Can you say anything about some other well-known customers that you have using the service, and the total spend you have under management?

Bill Mungovan: Sure. We have 600,000,000 in spend under management. One example of a large customer that I am allowed to disclose is Delta Airlines. They are using an agency called . We have both agencies and direct clients using the tool. And we have other retailers, like Backcountry.com, using the tool as well.

Eric Enge: Thank you Bill!

Bill Mungovan: It was good talking to you, thanks a lot!

Have comments or want to discuss? You can comment on the Bill Mungovan interview here.

Other Recent Interviews

Google's Rajat Mukherjee Interviewed by Eric Enge

Tue, 31 Mar 2009 12:42:56 -0400

Published: March 31, 2009

The following is the transcript of an interview of Rajat Mukherjee, product lead for Google's Custom Search Engines project.

Dr. Rajat Mukherjee is a group product manager on the search team at Google, Inc., working on several products, including Google Custom Search.

Prior to joining Google, Rajat was senior director of product management, Yahoo! Search Technology, and was responsible for managing the product platform for Yahoo! Web Search, ultimately providing innovative search products for consumers to find, use, share and expand content across the Web.

Prior to joining Yahoo!, Rajat was the director of software engineering at Verity, Inc., where he was responsible for managing and creating new applications including Verity Ultraseek (formerly Inktomi Enterprise Search), Verity Enterprise Web Search, Verity Response and Verity Federator. Before Verity, Rajat was a research staff member at IBM's Almaden Research Center, conducting research on Web technology and content management.

Rajat began his career as a research staff member at IBM´s Thomas J. Watson Research Center in Hawthorne, New York. There he conducted research on parallel and scalable Internet servers, high availability in clustered computing and scalable transaction processing. His experience included contributing to the design of infrastructure used during the 1996 Atlanta Olympics and the Deep Blue-Kasparov chess match.

Dr. Rajat Mukherjee holds a Ph.D. and a Master of Science in Electrical and Computer Engineering from Rice University, as well as a Bachelor of Technology from the Indian Institute of Technology in Madras, India.

Interview Transcript

Eric Enge: Tell me what's been going on with custom search engines?

Rajat Mukherjee: Over the last couple of years many things have happened. It's been very exciting to be part of a team that's spearheading a new hosted search paradigm. And it's been a lot of fun developing the product features with the help of an outstanding development team and watching the product grow.

Eric Enge: Can you give us a top-to-bottom overview of the major initiatives you have and the goals you are trying to accomplish?

Rajat Mukherjee: When we spoke two years ago, we had over 100,000 registered search engines. Today, we have more than one million registered custom search engines. Since I joined the project in January 2007, we have had about one hundred times growth in traffic.

So, the program has been doing really well, and we have pretty significant traffic. I think this effectively validates the need for high quality contextual search, and it proves that there is a need for this kind of product in the business world. Our goal is to satisfy this need with the best product that we can build.

Eric Enge: Absolutely. You have also focused a lot on site search, correct?

Rajat Mukherjee: We have a range in terms of product capabilities and deployment of technology. Site search is one requirement, but we also have a range of different types of search experiences.

For example, we launched a Blogger widget on Blogger's experimental site . With this, people can actually create a search experience that's not just searching their own blog, but also pages they've linked and their blogrolls. It's like a blog neighborhood, if you will.

Many personal search experiences have been created, e.g., around bookmarks, and many developers are doing similar things around their communities. So, that's one category of search engines.

Google Site Search targets businesses. There, you have a whole bunch of organizations that are developing search across a single site, or collections of sites. So, if you have a publisher with many publications, for example, you can search across all those different magazines or brands.

Then, we have interesting community-based implementations. Adobe has implemented Community Help, which allows people to search both Adobe.com as well as content from their developer communities. Thousands of sites are encompassed within Community Help, and Adobe has taken the further step of integrating this search experience into Creative Suite 4. That was very exciting. Basically, users and developers have created this range of different search experiences based on contextual needs.

Eric Enge: Right. This notion of searching comments from the community is an interesting one.

Rajat Mukherjee: Yes, I think the community aspect is something with the potential to develop and we hope to add interesting new features as we go forward. I think that's going to be an exciting area down the road.

Eric Enge: Right. Have you seen anybody just simply investing the time and building high quality search experiences by editorially selecting a set of sites to use?

Rajat Mukherjee: Yes, absolutely. If you look at About.com, they have a very interesting implementation. They have multiple sections, and each of their sections is handled by an editor or guide. They use Custom Search both within and across sections. So, they've used advanced capabilities like Linked Custom Search to create dynamic search engines that morph over time, and that's a very interesting implementation.

Eric Enge: Yes, that is interesting. How long have they had that out there?

Rajat Mukherjee: It has been around for over a year now, if I am not mistaken.

Eric Enge: That must be doing quite well for them.

Rajat Mukherjee: Correct. They've also adopted interesting advertising changes. With Custom Search, you can tune your ads to be more topical by using keywords. We use these keywords to further refine the ads that we display. Partners have seen revenue improvements with this feature.

Eric Enge: You could really do some very interesting tuning there. It's one of those things where you might pick the few obvious phrases, and find that they don't do as well as when you pick some less obvious phrases.

Rajat Mukherjee: Right. We are interested in looking into this area in greater detail and developing better technology for it, because monetization is one of the key aspects of Custom Search. This has also led to its success, because we've had monetization in place since our initial launch. Publishers are obviously benefiting from that, both in terms of getting the right ads in front of their users and monetizing search. Any improvements in this space can help our webmasters and publishers right away.

Eric Enge: Another thing I have noted is the programmability of custom search engines has increased dramatically.

Rajat Mukherjee: Yes. As I mentioned, Linked Custom Search provides users the ability to create dynamic search engines that change over time. And we update these engines automatically by reading the appropriate definitions that developers have made available to us.

In addition, we've also added a programmatic interface for management and creation of Custom Search engines. That is very interesting, because we've integrated Custom Search with several web hosting providers who've provided their users the ability to add search to their websites from the hoster's control panel. We announced this a while back with a number of hosters like BizLand, IPOWER, StartLogic and FatCow. They have implemented this provisioning API, allowing webmasters to create search engines and place search boxes on their websites without even coming to Google.

Eric Enge: So, why don't we talk a little bit about some of the other new things that have happened, such as On-Demand indexing?

Rajat Mukherjee: We've made a lot of improvements in our indexing pipeline over the last year in order to go deeper into the sites that we are searching across. We introduced On-Demand Indexing, which allows you to index specific content within 24 hours. That's very, very important. For example, when you are launching a new product on your own site, you want it to be indexed and searchable, so you can use On-Demand indexing to achieve that.

As part of these indexing improvements, we've added support for Sitemaps and integrated more tightly with Webmaster Tools. It helps to have many of these services integrated.

We just launched Google Services for Websites, a program for partners that offers integrated services from Google that are relevant to webmasters. This enhances our offerings to hosters so that their users can quickly sign up for Google services, such as Webmaster Tools, Custom Search and AdSense.

Eric Enge: Right. It's a way of helping get it out there, and an expansion of what you did with the hosting companies already.

Rajat Mukherjee: Yes, exactly. We are looking to make it easier for hosting companies to use our APIs and services and offer them to their users at the hosting control panel

Eric Enge: What aspects of the product are you focusing on?

Rajat Mukherjee: Over the last couple of years we've done a lot of different things. I'll put them into three categories.

The first one is product improvement; features that we've added to improve the product. I think we are starting to see some really interesting implementations of dynamic custom search. We've had great feedback on our experimental Blogger gadget and we hope to improve it and make it available to all Blogger users. We're also excited about our ads improvements.

The second category is the offerings we provide targeting different classes of users. Site Search offers businesses optional ads, branding controls and greater presentation control via XML results. Business customers also have greater needs for On-Demand Indexing. AdSense for Search is for publishers who wish to monetize, and developers can take advantage of Custom Search and AJAX Search APIs. These offerings address the needs of different market segments.

The third area is how Google itself is using the technology. We use Custom Search in many of our products and services. We eat our own dog food.

In addition, Custom Search is available globally. More than half of our traffic comes from outside the US and in languages other than English, so that's exciting. Custom Search was one of the first Google products that was internationalized into forty languages in one shot. It's really pushing this global approach to the product from day-one.

Eric Enge: Right. What country outside of the US do you receive the most volume from?

Rajat Mukherjee: There are quite a few. We, of course, get a lot of results from the UK. We also have countries like Germany, France, Brazil, Turkey and China. It changes on a weekly basis. There are many languages that are showing tremendous growth in traffic, e.g., Spanish.

Eric Enge: What improvements are you making on ads?

Rajat Mukherjee: We have the ability to tune your ads through keywords that you can provide for your search engine. We are looking to expand this as we move forward in terms of being able to automatically figure out what ads we should show for a given topic. We hope to improve user experience and publisher revenues with more topical ads. And we also talked about integrating promotions. I think we talked about subscribed links last time -- we've tightly integrated that so that people can provide promotions right on top of the results for specific queries.

Eric Enge: For the benefit of people reading this interview, what exactly are subscribed links?

Rajat Mukherjee: With subscribed links, a webmaster who is providing search on his website can decide that he wants specific information to show up right on top for specific keywords. Say you are a travel website like Orbitz, you can show a special promotion that you are offering for a trip to Hawaii integrated right into the top of the search results for users who are searching for queries like "hawaii", or "maui".

We've made it easy for people to configure that. And we'll be looking to make further improvements for promotions. The idea here is that you can promote a given set of results that you know your users want to see right to the top.

Eric Enge: Tell us a little more about how Google uses Custom Search internally.

Rajat Mukherjee: We learn a lot as we use the product ourselves. Many Google sites are powered by Custom Search, for example, Google Analytics and our Help Centers. We also customized the search experiences for Google's landing pages for the 2008 Olympic Games and the 2008 elections. That's where the contextual nature of search shows right through. So, for example, if you do a search for "nuclear power" on Google, you get very good, but generic, results about nuclear power. But, if you performed the same search on the elections landing page, you would get information around the policies of both candidates during the elections. So, that was a very tightly focused search experience, and that's where custom search really does a great job.

Eric Enge: How does someone go about creating something like this politically-oriented version? What is the process?

Rajat Mukherjee: This was actually quite simple. When the election team was putting together this landing page, they decided that they wanted to have a very tightly-coupled search experience. Effectively, they decided on a number of high-quality sites, including the Convention sites, candidate sites and top political blogs. There were obviously many active blogs at that point in time.

That was done by a set of Googlers, and they used the collaborative features of custom search over a couple of days. Custom Search allows users to invite others to contribute to the search engine.

Eric Enge: Right. So, there is some up front work to decide the family of sites that you want to have included, but if you are knowledgeable about your space, you can probably do that fairly quickly, rather than trying to invent something you don't know so well.

Rajat Mukherjee: Exactly. I mean, this really confirms that people who are experts in the field can collaborate for a better user experience.

When Google celebrated its 10th anniversary, we created a timeline and we used Site Search to search the timeline. So, we've been getting a lot of feedback from internal teams in terms of features or enhancements they'd like to see in the product. So, with internal requirements and feedback from webmasters, our users have helped enhance the product significantly over the last couple of years.

Eric Enge: Can you talk a little bit about how the timeline was put together?

Rajat Mukherjee: We were trying to highlight product launches and events that have occurred over the last ten years in the development of Google as a company. An interactive timeline application was built - you should check it out. Site Search was used to search the timeline. The search user experience was very tightly coupled with the timeline. So, when you click on a search result it brings up the interactive timeline, which is a very nice AJAX-driven dynamic browse experience.

Eric Enge: It really sounds like a lot of cool things are going on as programmability is allowing different people do more aggressive things, and the outreach through distribution partners like the hosting companies.

Rajat Mukherjee: We have worked pretty hard on outreach to developers, and we have a strong developer community as well. On that front I've actually had the opportunity to present to webmasters, because Google has organized a set of SearchMasters conferences -- three over the last year - Mexico City, Sao Paulo, Brazil and, most recently, Bangalore, India. At SearchMasters we have been able to reach out to more than a thousand webmasters, talking not just about products and APIs, but also about integration with Webmaster Tools and other Google services. So, that's been really exciting.

Next month, Nick Weininger, our Tech Lead, will be participating at the World Wide Web Conference in Spain.

We also have a Custom Search session at the upcoming Google I/O developer conference in May in San Francisco.

So, we are constantly looking to reach out to our developer community. We get a lot of feedback from them and on the Custom Search user group as to what users want to see in the product.

Eric Enge: Yes, I imagine you do. Can you talk more generally about the need for such technology and where the industry is currently?

Rajat Mukherjee: I think the good news is that our effort to create a contextual search offering that's relevant and useful for business has been validated. I think over the last couple of years the growth of the product and the feedback we've gotten have been tremendously positive. So, I think we are doing really well in terms of addressing the market need.

One point I'd like to make is that we customize not just the product, but have also customized our offerings for different customer classes - webmasters, publishers, businesses, partners and developers. I think that puts us in a very good position in the industry. We cover everything, from addressing small developers' needs for personal search engines, to helping large publishers who have significant configurability and scalability needs. Our traffic is growing and we're getting positive feedback from our customers.

Eric Enge: Right. So, our good friend from the election, Joe the Plumber, could create a very simple custom search engine for his website, while a massive company like Boeing could create an entirely different kind of custom search engine.

Rajat Mukherjee: Correct. And they have different options. Those who want to monetize have the ability to do so, and they can get the product for free. Those who want to have more control can use Site Search, which is a licensed offering. So, we've made sure to address the needs of these different markets, not just focus on technology.

In terms of product, we have a lot of unique features, e.g., Linked Custom Search for dynamic behavior, the support for On-Demand Indexing and deeper coverage of sites, and topical ads. The ability to do promotions easily and drill-down into results using labels and refinements are all unique features.

The other important point is that monetization has been built into the product right from day one. It's not something that we had to figure out as we went along -- it was part of the core value proposition of the product. Now, with the APIs that we have for distribution and the features we have in the pipeline, I am really looking forward to entering the next phase of product growth. So we've created a very compelling platform offering.

Eric Enge: Right. Well, ten to one growth in the past two years is a hard act to follow.

Rajat Mukherjee: Yes. 100 times in terms of traffic! It's true and it's really exciting. And I would love to see where it is in a couple of years. In terms of future features, there are lots of different things that we are looking at adding to the product. Improving monetization will always be useful for everyone involved; it's a practical need in the market. And, if people can monetize better, especially in these times, I think that's great. A lot of people are looking to those solutions. We have been working on features around improved presentation and customization. Obviously, when you say custom search, people want to customize their search results in various ways. Today we have the ultimate end customization via XML results - you can do whatever you want - but, we want to make it easier for people to do this even without major development resources. So, simple product configurability is something we are always trying to enhance. We are looking to improve support around more structured content. Because our product is global, we are looking at better linguistic support, and there are lots of tools that Google has already developed that we can use in custom search, e.g., transliteration.

There are many other features that we'd like to add. For example, we want to add query suggestions contextual to the topic of the search engine. That's not trivial to implement.

SearchWiki on Google.com means one thing, but what does it mean in Custom Search? Who your community is, right? So, we need to look into how we deploy some of these advanced features in the context of Custom Search.

Customers always want improved analytics, and that's something that we would like to improve. I am personally a big believer in the mobile experience. So, as Smartphones become more common, what would it mean for us to provide a mobile search experience? So, these are some of the ideas that we talk about. And, of course, some of these get baked into our roadmap as we move forward.

Eric Enge: Mobile search is a very interesting animal just because of its very nature, a very challenging environment with small keyboards and more awkward navigation

Rajat Mukherjee: I agree. And I think that's where features like suggestions make a lot of sense, in terms of being able to click on something as opposed to typing it in. If you are searching in a language that's difficult to type, automatic suggestions and transliteration really help.

Eric Enge: Yes, of course. And that will play in some international environments. Well great, there's a lot going on; and it's obviously a very successful product line. Where can readers follow the latest developments?

Rajat Mukherjee: All our developments are announced on the Google Custom Search Blog.

Eric Enge: Thanks Rajat!

Rajat Mukherjee: Thank you Eric!

Have comments or want to discuss? You can comment on the Rajat Mukherjee interview here.

Other Recent Interviews

Palo Alto Software's Sabrina Parsons Interviewed by Eric Enge

Tue, 24 Mar 2009 09:17:42 -0400

Published: March 16, 2009

This is a little bit off our normal path, as it is not an SEO or analytics centric interview. However, Palo Alto Software has aneat product that small businesses can use to manage their operations, known as Email Center Pro. In addition, it offers some nice analytics tools that can be used to help imporve business operations and results from email inquries from the web.

Sabrina Parsons is currently CEO of Palo Alto Software. See is also an avid blogger on her blog, Mommy CEO. She was a principal co-founder of Lighting Out Consulting, which helped Internet companies realize their online marketing potential.

Prior to co-founding Lighting Out, Sabrina was Sr. Producer at Epinions.com. As Epinions.com's first employee, she conceived, designed and implemented Epinions.com's affiliate/partner program, leading an engineering, web design, and marketing team to the successful implementation of one of the most powerful partner programs on the web today.

Before joining Epinions.com, Sabrina was the Director of Online Marketing for Commtouch Software (CTCH). She launched, product managed, and developed online strategies for Zapzone Network, a global email service provider for webmasters and the SOHO market. Prior to working at Commtouch Software she was the Online Marketing manager for enCommerce, now a part of Entrust Securities (ENTU). At enCommerce Sabrina oversaw two web site re-designs and managed all online marketing activities.

She, along with her husband Noah, funded and successfully grew the UK Palo Alto Software office in London before selling it to the parent company and returning to Eugene Oregon where she joined the Marketing Department as Director of Marketing and Communications.

Interview Transcript

Eric Enge: Can you give us an overview of Palo Alto Software?

Sabrina Parsons: Palo Alto Software is a company that's been around for a long time in the hi-tech world. We've been around since 1983, and our primary focus is to help people succeed in business. Our target market is small business and entrepreneurs. We have provided planning and implementation tools to help people better manage their small businesses for 25 years now. Our flagship product is Business Plan Pro, which people can purchase at Best Buy, Office Depot and all those places, as well as online.

But, we also have a marketing planning product, and recently we've re-released that product in conjunction with John Jantsch and Duct Tape Marketing. So, that's been really exciting, but our most exciting new venture is Email Center Pro, which came out in May in Version 1.0.

Since then we've had nine feature releases, so it's a very different product than it was last May. It is a software-as-a-service product (SAAS), which is also very new thing for Palo Alto Software. We've traditionally been software developers developing Windows' Software, but we see SAAS as the way of the future. So, we'll probably be doing a lot more in the SAAS world in the future.

I've been the CEO of Palo Alto Software for just about two years now. Prior to that I was dealing with all the business development, strategic alliances and marketing communications for Palo Alto Software.

Eric Enge: Can you talk a little bit about Email Center Pro from an overview perspective, and go a little more deeply into what it does and how it works.

Sabrina Parsons: Email Center Pro is a little bit of a departure from what Palo Alto Software usually does. We've been very entrenched in business and marketing planning, and then that's been our expertise. We originally created the product about eight years ago to help us internally manage our incoming emails and customer service.

About two years ago, we really looked at the marketplace and looked at this build versus buy concept. Our customer base had gotten bigger and the small little internal tool that we had needed to be rebuilt. It just wasn't robust enough for our needs. And at that time, we just said "well, this is crazy, this isn't what we do. Let's go out and buy a solution." So we went out and just looked in the marketplace for a solution. We are not a large company, as we only employ forty-five people, so we didn't want to pay thousands of dollars for something that maybe was a sledgehammer approach to our very defined email problem.

So, the more we looked in the marketplace, the more we realized there really isn't a great email management tool in the customer service space that helps small businesses streamline and improve their workflow and become more efficient. There are lots of tools out for larger companies with a larger budget and an IT infrastructure. But, if you are a small business that maybe doesn't even employ someone in IT and just uses a consultant, you don't have $50,000 to drop on that kind of communication software.

So, you really were just making your email work and doing a lot of duct tape and paperclip type of solutions to deal with more and more incoming emails from your customers, partners and vendors. So in May we released the consumer version of what we had internally been using for a long time, which we called Email Center. That's why we called the commercial product Email Center Pro, it really is just focused on helping small businesses become more efficient dealing with their customers, partners and vendors, and helping them provide a better customer service experience. Having something that is easy to use and implement where no one has to worry about upgrading is very valuable. We really feel like SAAS is the way to go, especially for small businesses.

Eric Enge: So, let's just take a little deeper dive into some of the specific problems that a typical small business might have. Say a small business has a dozen people working for it and fairly robust customer service communication happening through a small number of email addresses. How does this fit into that picture?

Sabrina Parsons: As email addresses are posted on websites or forums are used, a lot of small businesses deal with it through the triage methodology. Those emails come in through aliases and they get sent to various people at once. Sometimes they are dealt with through a LISTSERVE, and that's the same sort of thing. So this 10 to 12-person organization has people wearing a lot of different hats and dealing with a lot of different types of customer interactions.

At the top of that organization, you have the business owner who probably has his hands in all these email addresses. You have a problem that happens when an email comes into an info@yourdomain.com email address. That creates this workflow issue where you've got multiple people receiving the same email into their inbox. Sometimes the business owners jumps in and answers one of the emails that look interesting - but often times neglects to let other proplr know. So customers may get multiple responses from the company - one from the business owner and one or more from other team mebers who receive the same email.

Inevitably this workflow leads to a lot of effort making sure emails are answered accurately and only one time. And everybody is dealing with it in their individual email, so there is no way to track history and there is no visibility or accountability.

Say there are four emails in this organization and they all get sent into Email Center Pro, they would all be organized by mailbox. So you can have an info mailbox, a sales mailbox, a partner's mailbox and a help mailbox, etc. And as those emails start coming in, they are automatically put in one place and everyone in that organization can have access to them. They can log in and see what's going on, and the moment someone starts reading an email and decides to reply to it, everybody else can still see that email, but nobody else can take an action on it. It will tell you, "Joe is replying to this email."

We've also then given people internal communications tools. I can place a note on that email and let Joe who is in the middle of responding to it, something that I think might be relevant. If I am a business owner, I can go in and cherry-pick the emails I want to respond to, but everybody else can see my responses and how I've chosen to deal with that email. Or, I can come in and look through some emails and maybe just add some notes, and then assign them to somebody else for follow up.

The system also has all kinds of checks and balances in place. If somebody gets assigned an email, you can have an alert setup so you are alerted when you get assigned to an email. That way you know someone has assigned you an email, so then you can go in Email Center Pro, deal with the email and answer it. Anytime an email is replied to, it automatically gets archived and moved out of the inbox. And that's one of those workflow issues.

People like to see clear inboxes, because it makes them feel like they are accomplishing something. So, we've made that part of the workflow at Email Center Pro. An email comes in, it gets replied to and you don't have to file it, delete it or move it, because it automatically gets archived into sent mail. You can archive any email that comes in that doesn't need a response.

From a customer point of view, I will automatically get assigned the response if I answer an email so that I can continue the conversation with that person. And, that's what we are really trying to help companies do, treat emails like conversations with real people, because that's what they are. These are people that are actually sending requests for information to buy things and get proposals, and they just get forgotten or ignored all too often. You see all these helpdesk ticketing systems that people are using now. You send an email, and then all of a sudden you get a response and a number. But the problem is that you don't want a number, you want an answer to your question. So those are some of the things that we are trying to deal with, with Email Center Pro.

Additionally because we are a company that is very fanatical about data analytics, and we feel like you can get a lot of information to help you run your business better if you have access to the right information, we built a bunch of different analytics into Email Center Pro to help that business owner understand what's going on. Just to give you a practical example of how it's being used, we have a client company called Herbal Buddy, a top Herbalife distributor.

It's vitamin company, which distributes Herbalife products on HerbalBuddy.com. It has an e-commerce site and it does very well. They signed up for Email Center Pro, started using our analytics and realized some patterns and some trends, because they were all of a sudden able to see email traffic in the same way that they look at website traffic.

They started realizing that they actually had trends in terms of when customers or potential customers contacted them. Herbal Buddy matched those trends to their Google Adwords account, and actually spent more money during the hours when they noticed more email volume. In the first month they did that, they increased their sales though Adwords by 30%.

We are trying to provide a lot of analytics about what happens in email so people start to understand email communication as an integral part of their outwardly facing marketing effort. If you send out an email marketing campaign to a bunch of customers, you have to respond and be willing and able to deal with people coming back to you, or you are potentially losing customers and sales.

From our perspective, Email Center Pro is a tool to help you manage your incoming emails and deal with the workflow to and make things easier. But it's also intended to give business owners some of that visibility that they can get from Google Analytics to allow them to track and manage their income and expenses and to give them that extra information. Now more than ever, in this economy, it's so important for businesses to understand that keeping a customer is a lot cheaper than finding a new one.

Eric Enge: Can you outline the various specific types of analytic data you can get?

Sabrina Parsons: Sure. You can look at your mailbox traffic over all your mailboxes or a specific mailbox over any time period you desire. You can see mailbox traffic over the last week, the last month or the last quarter. And you can also drill down to your sales mailbox, your business development mailbox or you can look at your entire email traffic in cumulative.

Another analytic that you can look at is actual mailbox breakdown, which is a pie-chart that shows you the breakdown of all of the messages in the system by a mailbox, whether they are assigned or unassigned. You can find out if your team is dealing with them or not.

We also have a mailbox traffic distribution analytics, which helps you see the distribution of emails across your different mailboxes. This allows you to see the trends of when you are receiving the most emails and where they are going. We also have a mailbox response-time analytics which allows you to see how fast you are responding to emails, either by mailbox or globally over the whole account. And that's one of the analytics that I track on the Palo Alto account, because we use Email Center Pro internally for all of our customer service. I like to see what our average response time is globally over all of our email and specifically for instance in just our sales@ inbox.

I track the response-time for the sales inbox because I want to see how fast our customer service people get back to people who have questions that come into sales@paloalto.com. I can adjust the average response time for business hours. Since we are open Monday through Friday, 7 a.m. to 5 p.m, the adjusted average response time for the sales inbox currently today, is one hour and three minutes.

That means that, on average, every email that comes into sales@paloalto.com gets answered within an hour and three minutes. . These analytics help me to quickly understand how we are getting back to our customers, and how my team is dealing with emails. So, as an business owner, I love that feature.

You can also see user activities, analytics which are used for more accountability and visibility. I track these analytics for for sales and for our customer service team that deals with all of our sales inquiries. I can see how many messages are assigned to each person, how many messages each person actually answers and what their individual average response time is. For example, Jay, who is one of our stars on our customer service team, has been assigned 550 emails in the last month, but he has actually answered over a 1,000. So that gives me a sense of his productivity. He is answering more emails than he is getting assigned, and his average response time is 47 minutes right now.

Eric Enge: Right, which is better than average.

Sabrina Parsons: Exactly. So, that gives me some really good visibility over what's happening with my business and it allows me to see if my customers are getting responses in a timely fashion.

We also have a mailbox traffic timeline, which gives a detailed interactive timeline of emails sent and received for a particular mailbox for a date range. So, it's not just overall mail volume, but it's actually separating sent and received emailsThere are more analytics, but I have mentioned that main ones.

We have other widgets also available for more visibility into your business' email communications, but they are not all pure analytics. You can have a saved search that constantly searches different types of communications and conversations that your team is interested inIn our account, we have saved searches for all our eBay customers, or for anybody who wants a Mac version of Business Plan Pro.We don't have a Mac version right now, but we want to track what the interest is for a Mac version.

We also have a recent actions widget, where you can monitor everything that's happening on a global level or on a subset of it if you are looking at one mailbox. You can see who is doing what to emails in Email Center Pro and track certain mailboxes and keep an eye on who is answering emails, what the responses are, and how quickly they are responding.

I think kind of takes you through pretty much all of our informational and analytic widgets. We also have something called the Conversation Tracker, which works with our conversation history. For every email that comes in, which we call a conversation, we can track its history.

You can use a Conversation Tracker to specifically track certain conversations you are interested in. Next to each email conversation you will see a display of exactly when the email was received, if it was assigned to anyone, who assigned it, who it was assigned to, if it was replied to, what time it went out, if ,the customer replied back and what time that response came back in.

You get a full picture of what's actually happening with that conversation. And again, the idea there is that these are people that you are dealing with and they want to be treated respectfully and intelligently. But, on the business side, that tracking needs to be efficient and quick, and something that actually saves you time and money as opposed to something that adds more workload to what is usually a pretty barebones team in a small busines.

Eric Enge: Right. So, the ability to look into that history is clearly one of the important things you can do here. If you are a busy customer service person, and you've just had a conversation with someone a week ago, you're not always going to remember everything that went on.

Sabrina Parsons: Exactly.

Eric Enge: So let's say the customer service person who originally answered someone's emails is on vacation. Do you have some sort of way to flag that so they get assigned to someone else?

Sabrina Parsons: Yes. If Jay has been dealing with the customer, and he is not there, it comes into the general inbox. So you'll see that Jay is not even signed in, and presumably teams will know that somebody is on vacation. That conversation can just be assigned to somebody else. We typically assign one "owner" to every mailbox, and that person monitors the mailbox for things like that.

So say Jay is not around and we assign the conversation to Shawnee for the time being. If Jay is curious about what was going on with that customer when he was gone, he could go back and track that conversation. Then he can jump in and say to Shawnee, "hey the next time that person responds, I am going to grab that because of the long history we have together."

That's one of things that I think Email Center Pro does very, very well. But Email center Pro is not just useful for businesses that have customer service teams. We have a PR of company that's just ten people that uses Email Center Pro. The reason they use it is, because they had an email failBefore this PR company started using Email center Pro, they had anoffice manager, who got all their info@ emails into her outlooks.. She would then triage each info@ email and send it to the right people. If she didn't know who to send it to, she'd send it to the CEO, and he'd decide. She was a really consistent employee and she'd been at the company for a couple of years. Things seemed to be fine with this workflow until the one week she got very, very sick.

She wasn't there, and unlike other times when she'd gone on vacation and arranged for someone else to check the inbox, and everybody forgot that the info@ email came into her. So, a week later when she came back, she had a request proposal from a very big company in her email, just sitting there, unresponded to for over 5 business daysBasically this PR company missed this opportunity because nobody checked that email. And this happens so often in small organizations, where it's just one of those things that people forget.

And, that's precisely one of the reasons why Email Center Pro is great, even if you are not dealing with hundreds of incoming customer service emails. And, that's why we wanted to price it the way we did. We wanted it to be an affordable option for small businesses which don't necessarily need a huge back office infrastructure, but they still struggle with this email problem.

Eric Enge: Thanks Sabrina!

Sabrina Parsons: Thank you Eric!

Have comments or want to discuss? You can comment on the Sabrina Parsons interview here.

Other Recent Interviews

InfoUSA's Pankaj Mathur Interviewed by Eric Enge

Tue, 17 Mar 2009 15:13:52 -0400

Published: March 16, 2009

Pankaj Mathur is the Vice President Sales for the infoUSA Licensing Division. He has been with infoUSA since 2005 and is currently managing the POI data licensing relationships with local search engines, navigation partners and LBS players. He works with the product team at infoUSA and with customers on new technologies and products.

Pankaj has an MBA in finance from the Carlson School of Management at the University of Minnesota. He brings tremendous experience to the table as he has worked in accounting, as a financial analyst and in project management roles in the past.

He attended the prestigious IIT (Indian Institute of Technology) for his Undergraduate Degree. He majored in Naval Architecture and supervised construction of merchant ships in China, South Korea and Thailand before deciding to pursue a Masters Degree in finance.

Interview Transcript

Eric Enge: Can you provide us with some background on yourself?

Pankaj Mathur: I graduated from the Indian Institute of Technology, and came to US for MBA (Finance) at University of Minnesota in 2003. My undergrad major is in Naval Architecture, an engineering discipline focused on design and construction of ships! I worked as a Naval Arch for little over 10 years and supervised design and construction of over 55 merchant ships. Right around 2001, I started thinking that I want to do more and something different with rest of my professional life and here I am. Although on hindsight, I wish if someone had explained to me how cold Minnesota can be!

Eric Enge: And how long have you been with the company?

Pankaj Mathur: I have been with the company since July 2005, and my focus is on LBS clients, particularly local search engines, Internet Yellow Pages, navigation and Telematics (this is the science of science of receiving, sending, and storing information via telecommunication devices.).

Eric Enge: Right. So, the navigation clients mentioned are companies like TomTom?

Pankaj Mathur: Yes. It also includes clients like TCS TeleCommunication Systems, a company based in Oakland. TCS offers a server-based solution, because on the mobile handsets and connected navigation units there is no need for storing all the information (maps, POIs etc) on the device. The information can be retrieved real-time from server as needed. While it may sounds like a small difference in terms of storage of data but the difference in terms of customer experience is quite drastic.

Think of the difference between an email and a fax. In a printed format the message from email and fax may not look much different. But fax is an image (pixel based) while email is text (character based).

Eric Enge: Tell me a little bit about the company?

Pankaj Mathur: We have been in business since 1972 and public (Ticker: IUSA) since 1992. In 2008, we did approximately $750 Million in revenue. Historically, we have been in business of mailing and marketing list. So, if you are a local restaurant in Charleston, South Carolina, and you want to send fliers out to all the businesses in your area advertising your lunch buffet, you come to us.

Back in 1972, our founder Vin Gupta realized that most companies (including Fortune 500) do not really know where their customers and prospects are located. So he started collecting phonebooks in his garage, and started building lists.

Over the years, we evolved into a very sophisticated large scale information compilation company. Currently we maintain a database of about 15M businesses in the US and Canada, and a database of about 250 Million consumers in North America.

The company started with compilation of publicly available phone books. Overtime we started getting into more marketing specific attributes like employee size, sales volume and corporate hierarchy etc that are not available in phone books. Another drawback of phone book is that not every business advertises in the phone book. Even some of the popular consumer centric categories like gas stations, car washes, small churches, Post offices, DMV etc have scant coverage in phone books. You also have to do multiple books in geography as each book might have some incremental coverage, this creates a challenge of duplication of records and I will talk in more details about it.

To improve coverage, we also compile sources such as government filings, country clerk data, utilities connects and disconnects, bankruptcy filings, tourism guides. With each additional source, there are always new compilation challenges. For example, legal filings or utilities data we may come across a listing for John Doe LLC that is a franchisee of Taco Bell, and also owns a gas station with a convenience store. The legal entity (John Doe LLC in this case) may not even be incorporated at the physical location of these businesses.

So, to deal with these types of problems and removing duplicates we started calling businesses in 1992 and since then we have been calling EVERY business in North America at least once a year. In 2008, we completed 30 million phone interviews and dialed over 45 million numbers for the same.

Eric Enge: Right, validation.

Pankaj Mathur: Validation. It was primarily done to increase deliverability of mailing lists back in 1991. There were no commercial applications like Google or navigation devices back then. The phone validation helps us with few key aspects of compilation such as:

Collect consumer facing information as opposed to legal entity information
Completion of name, address, categories
Removal of duplicates (KFC or Kentucky Fried Chicken)
Standardization of common terms in names and addresses
Removal of out-of-business listings (phone books are published only once a year)

The example below highlights all of the above points.

Don't get me wrong, I am not trying to demean the value of yellow pages; they've done an amazing job of establishing value in local communities and the industry has been around for several decades. But in Location Based Services ("LBS") environment people have started using data for turn-by-turn directions & people have started using data for more specific relevancy-based search. The yellow page data is not even close to fulfilling these tasks.

Eric Enge: Yes, absolutely.

Pankaj Mathur: So this is one of the reasons why the phone verification helped us. I mean KFC and Kentucky Fried Chicken is easy for us to de-dupe. But if there is a Joe's Pizza and a Joseph's Italian Pizzeria, which is non-franchised in some unknown small town, it is extremely difficult to identify them as same unique.

In this perspective, definition of business becomes important. Here is a live example, see below for D&B data with a PO Box Number and phone number that is connected to business owner residence. Compare that to InfoUSA data, which capture all four locations of the Mexican restaurant owned by Annas. In a LBS environment, what do you think people would be looking for?

Also note that even a PO Box number can be geo-coded (to centre of zip) but the turn-by-turn navigation experience will not be near as good and not to mention that users will not get any food served!

Let's look at another example:

The problem here is that it shows COSTCO with a suite number. This is probably a management office but when people do local search for COSTCO, they are looking for places where they can shop. I hope you are getting the challenge of "definition of business" in LBS environment.

Let's take a look at the problem of duplicate results:

Even for a popular category like Pizza, you can see in example above the duplicates as a result of variations in names and addresses. Interestingly none of the four listings gets the combination right.

Eric Enge: Yes. They either don't have it or they have the 108th in O Street.

Pankaj Mathur: Yes. Let's look at why this is happening:

These are two actual captions for Valentino's pizza delivery. Valentino's is advertising in two different sections of the phonebook, maybe restaurants, and pizza delivery. So, this helps explain some of the errors and duplication in the records, and maybe there are more. I just didn't go deep enough.

Here is another example for gas station in Foster City, CA

Compare this to four gas stations that InfoUSA has captured by phone calling:

All four gas stations above have been in business for over 10+ years. The reason for lack of these listings in yellow pages is simple; people do not look up yellow pages for gas stations. Hence these gas stations are not advertising in phone books.

Eric Enge: Right. So, this is the kind of thing that validation does for you, right?

Pankaj Mathur: Exactly. Some businesses may even discontinue their listing once they have established themselves in a local community. We keep calling these businesses year after year, even after they have dropped out of phone books.

We are seeing a trend that the coverage of phonebooks is actually dropping. So, back in 2001 we could get almost 98% of our listings from phonebooks, but as of today, that coverage has dropped to the low 70s.

Switching gears, I always get asked about specialty content providers like Zagat and user generated content. From what I have seen, end users are very passionate about few selected categories like eating, drinking, personal services and shopping. Thus people may feel excited about a restaurant, a new IKEA store opening or a hair stylist. But InfoUSA compiles over 10,000 categories and a large chunk of these ignored by users for lack of personal interest.

Even across the popular categories where people have personal interest, the users often end up creating duplicates (just like phone books) on account of variations in names and addresses. There is also a problem of ownership of reliably tracking out-of-business listings.

The restaurant below has ceased operations in Dec 2007 but is still being showed in search results as late as February 2009!

Eric Enge: Let's look at your InfoUSA verses enhanced content example. Who wrote this description under products and services?

Pankaj Mathur: Actually, no one. And, that's exactly the point I am trying to make. A lot of this so-called enhanced data that you find in the marketplace has been modeled. I am sure you can write at least 10 or 20 attributes about McDonald's just from being there once.

In the example above, the In-N-Out Burger actual address is 260 Washington Street, Daly City, CA, 94015. The above discrepancy is happening because low cost data vendors try to back fill information such as city name, state and zip (these are not published in phone books) to complete addresses. Thus in this case, the data provider ended up creating a valid address, which is 260 Washington St, San Francisco, CA, 94111 as Washington St runs right through San Francisco and Daly City. The only problem is that there is no restaurant at the above published address.

Here is another example of modeled information on COSTCO that does not really exist!

Unfortunately in the example of In-N-Out Burger and COSTCO, it is always the end user of local search application that discovers the inaccuracy of data the hard way. Can you imagine the impact such user experience on your brand name? I would probably not use the application ever again.

One important point for all the example shown so far, every data set will have inaccuracies and having one or two records wrong does not statistically amount to much of an interpretation. But the examples are good to enough to measure or think about reliability of data compilation process. How hard it is to identify a COSTCO that does not exist? How difficult it is to remove duplicates of a high visibility franchise like In-N-Out? Given the present reliability level based on these examples, what is the reliability of data source for delivering accurate information on non-franchise and small local businesses?

Eric Enge: So, how often do you revalidate the data?

Pankaj Mathur: We validate every listing at least once a year. But there are some event-based validations that may trigger before 12 month cycle. If there is a change of address at USPS or phone disconnect we will call the business ASAP to validate.

Eric Enge: So do you use an automated call system to validate the information, or how does that work?

Pankaj Mathur: We have a compilation team of about 800 people based in Omaha, NE for compilation of data. Of these about 300 are responsible for phone validation. We use a smart-dialer but the information is collected in a very human form. This is also the reason for InfoUSA phone validation being such top notch as operators are able to alter the tone, questions based on responses given. The cost of having people make calls is very high but given the lift in quality and accuracy we feel that it is worth it. All of our customers are aware of quality differences between InfoUSA and low cost alternatives and are willing to pay more for quality.

Eric Enge: So, it's 14,000,000 businesses?

Pankaj Mathur: Yes.

We focus on three key aspects for compilation of business information

Coverage – to some extent total number of listings is an indicator of coverage but there are other sources D&B, Acxiom, Localeze etc that claims over 17-18 million records. From our perspective, these numbers often include legal entities, duplicates, out of business listings and even COSTCO stores and In-N-Out Burgers that never existed!
Accuracy – we manually key in over 5,000 phone books every year, this helps removes duplications like KFC and Kentucky Fried Chicken. Phone validation helps complete information and helps identify more duplicates and redundant listings.
Relevancy – this is the hardest metric to grasp and LBS industry is currently still grappling with it. To give a simple example, a listing for "California Pizza Kitchen" inside San Francisco airport may be relevant if I am inside the airport and need to eat. The same listing becomes completely irrelevant if I am driving around looking for a place to eat. Another example, a teenager may not consider "PF Chang's" as a valid result for Chinese food. By the same token, a business traveler may not consider "Panda Express" as a relevant result for the same keyword. Thus relevancy is often dictated by user's intent, environment, demographics, time of the day etc. Frankly speaking LBS industry has done a very sub-standard job of it so far.

The partial list below shows information compiled by InfoUSA to help with relevancy

Storefront Photos & Door Step Lat/Long - Currently over 3 Million and collecting 150,000 new photos every month
Brands
Franchise & Chains
Specialties- for categories such as Attorneys and Physicians
Make- over 100 makes tracked for categories such as Car Dealers and Tire Dealer
Type- over 300 types tracked for categories such as Radio Station, Movie Theaters, Malls, Schools, Libraries etc
Work-At-Home Flag-over 1.3 Million records flagged across categories like caterers, contractors, realtors, photographers etc.
Landmark Addresses- over 1.4 Million records for additional location information such as airports, malls, corporate campus or another business
Shopping Malls- classified over 6,500 malls into categories such as factory outlets, lifestyle malls, power malls etc.
Emergency Flag for Hospital - over 4,300 flagged for true emergency locations
Date of last phone contact
Hours of Operations
Credit cards accepted

Eric Enge: Which are the companies currently licensing InfoUSA data?

Pankaj Mathur: Currently we are licensing data to all top five search engines (Google, Yahoo!, MSN, AOL, Ask etc), we are powering Points of interest data on over 90% of in-car navigation units (Toyota, Lexus, Honda, GM, Ford, Nissan etc). InfoUSA also licenses data to KGB (fka InfoNXX) , mobile applications like Telenav, Tellme and Telematics services like OnStar.

One advantage of having such diversified list of customers is that quality is driven by the most critical applications. Thus we have to compile gas stations with accuracy that Lexus can use for turn-by-turn directions and compile hospital with 24/7 flag that OnStar can use. But in the end it sets the quality bar very high at our end.

Eric Enge: These are the people who are purchasing local business data from you?

Pankaj Mathur: Yes, either directly or indirectly. Sprint & AT&T has Telenav installed on handsets and their customers will use InfoUSA data through Telenav application.

So you may ask if it is this easy – key in the phone books or call every business then why aren't our competitors or anyone else doing it?

Eric Enge: Well, I can answer that question; it's a problem of scale.

Pankaj Mathur: Exactly. The marketing side of the business, which still accounts for over 80% of corporate revenue, helps subsidize the cost of compilation. It is impossible for any company to phone validate information solely based on LBS clients as the market value is not high enough to justify incurring such a high cost.

Eric Enge: Right. Let's go back to enhanced content a little bit. Do you ever try to capture any data like hours of operation or things like that?

Pankaj Mathur: Oh, yes. In spite of everything that I said about accuracy and phone validation, there is still a lot of other data that I would want to know as an end user. For example, what are the store hours? For a restaurant, I want to know soup of the day, price range or what other users have to say about it?

Thus there is still room left for information like ratings and reviews. But also note that all of this information is either too subjective or too volatile that it is not possible for InfoUSA to capture it using phone validation (even ignoring cost for the time being).

Thus there is no way InfoUSA can ever tell like Citysearch that this restaurant is four out of five, or that this pizza place sucks, because it's a very subjective opinion.

There are companies that are collecting such data and given below is a partial list.

Restaurants (CitySearch, BooRah, Zagat, Open Table, wCities)
Gas Prices (Gas Price Watch, OPIS, Gas Buddy)
Users Reviews (City Search, Yahoo!, BooRah)
News & Events (Topix, Zevents etc)
Classifieds (LiveDeal, eBay, Craiglists)
Editorials and Profiles (Merchant Circle, City Search)
Contractors (Service Magic)
Taxonomy, coupons, parking, golf courses, live inventory and several more

Lots of this data has consumer appeal and LBS players have to mix data from multiple sources, which brings another key topic to forefront i.e. Data Aggregation.

Look at examples below to understand the challenge a LBS application faces when trying to match data from multiple sources.

To alleviate this problem and help customers leverage power of phone validation InfoUSA performs. We appended our record identifiers (InfoUSA IDs) to selected partners (see below), this extended the reliability of validation provided by InfoUSA to enhanced data partner and also solved the problem of data matching (using InfoUSA 9-digit IDs). Thus City Search can provide real-time feeds on ratings, reviews, prices etc to any of our customer, who can then match the record to a unique InfoUSA record without compromising accuracy of search results and spending any time or resources in matching.

Another important aspect of our 9-digit identifiers is that it is very persistent and Mike Dobson has blogged in detail about the value of having a persistent identifier. You can read the blog at http://blog.telemapics.com/?p=92.

Eric Enge: So, this relates to another question I had which is, that when I was going through the process of finding how to add records to a search engine, Google and Yahoo were fairly straightforward, I got to Microsoft and where I landed was actually a page on the InfoUSA site. So, it's huge that Microsoft is using you nearly as a sole source.

Pankaj Mathur: Even in cases when a customer of InfoUSA accepts your listing on their own branded page, many times the listing is delivered to us for validation. I cannot disclose the names of clients as many of them have advertised this as their own service. InfoUSA makes about 4,000-7,000 net corrections every month as a result of such feedback. But the reasons why these customers ask us to validate these listings corrections are two folds

Objective validation of listing submissions - you cannot always assume that merchant submission is always 100% honest. We find that about 50% of listing correction submissions are genuine or warrants some kind of corrective actions. For the remaining 50%, there are cases when a taxi cab may try to list themselves under airport or someone playing mischievous and asking us to change a pizza delivery number to a residential number. There are also cases of a competitor calling and asking that a bar be also classified as an escort club!
Costs and Data Hygiene - our customers will have to incur the cost of validation and over time this creates a problem of data hygiene. For example, if one of our customers added a listings (after verification) for "BJ Brewery at Mariners Island and Fashion Blvd in San Mateo, CA". InfoUSA might have captured the listings from another source as "BJ's Breweries at 2206 Bridgepointe Pkwy, San Mateo, CA" Thus you have a duplicate and you are also stuck with ownership and hygiene of second listing in perpetuity.

Eric Enge: When a local portal of some sort wants to license data from InfoUSA, how does that work? Let's say I am SuperPages.com and I want a very large amount of data, because I am trying to be complete with what I am doing. How is that structured?

Pankaj Mathur: The deal structures vary according to the needs of our partners. Because they are all in local search space, there are some commonalities. For instance, all them would want physical location address for the merchant as opposed to direct mail clients who may want PO Box addresses for higher deliverability.

But even within LBS customers we see several customizations which are application specific such as in-car navigation customers may want to drop categories like contractors or listings for lawyers. While mobile application clients may focus on a very narrow band of categories like eating, drinking, shopping, travel and personal services and discard the rest.

Eric Enge: Can you give us a general sense as to what pricing looks like?

Pankaj Mathur: We have a usage based pricing, because as you can see we are continuously investing more and more into the content. Also InfoUSA is investing heavily in improving content for driving better customer experience. Currently we have projects running across categories like ATMs, Amtrak stations, fuel grades (electric cars), towing services, taxis, locksmiths etc. So when it comes to pricing, we want some incentives to be able to continuously invest in improvements. We follow different kinds of usage pricing models based on applications such as CPM (Ad supported & web based), transaction (411), per subscriber (mobile) etc. The usage volumes are also drastic different, Internet players can probably deliver billion of pages views but same cannot be expected from in-car navigation.

Eric Enge: Thanks Pankaj!

Pankaj Mathur: Thank you Eric!

Have comments or want to discuss? You can comment on the Pankaj Mathur interview here.

Other Recent Interviews

Eric Enge and Michael Gray Discuss Twitter

Mon, 9 Mar 2009 13:12:30 -0400

Podcast Date: February 26, 2009

The following is a written transcript of the February 26, 2009 podcast with Michael Gray of Altas Web Service and Eric Enge:

Eric Enge: Hi, this is Eric Enge with Stone Temple Consulting, you can see our website as www.stonetemple.com. I am pleased to be here today with Michael Gray of the Atlas Web Service; you can see his website at www.atlaswebservice.com, where he offers a variety of consulting services, and you could also see his blog which is rather well-known at wolf-howl.com. How are you doing today, Michael?

Note: Michael Gray tweets as @graywolf and Eric Enge tweets as @stonetemple.

Michael Gray: I am good, how are you?

Eric Enge: Let's discuss the basic uses of Twitter and the different ways that people can use it.

Michael Gray: There are lots of different ways to use Twitter. I wouldn't say that there is really a wrong way, but some ways are definitely better than others. The first level is people who just use it to follow their friends. Maybe they have five, ten, fifteen people that they follow and that they just use it so they can send status message back and forth.

They may just be having normal, everyday conversations, but this level is usually used by people who have been separated by geography. It's actually a big way for all of them to share information with each other, but it's a very small little thing. It is typically not a big amount of information that is being shared or anything that's really important. It's mostly a personal thing.

The next level is people who are just interested in communicating with other people online. Maybe they have friends that they have met through other forums or newsgroups or who share the same interest. Maybe they play the same online games or they are the part of the same community. And again, those people are almost always separated by geography.

The next level is people who are trying to engage a larger part of the community. Maybe there are people who are really interested in iPhones, and they like talking and sharing other apps, doing little hacks and finding out the best things to jailbreak, that sort of thing. They are just on talking with people in a larger part of the community who have a specific interest. As it moves up, we get people who are just interested in talking with other people online.

Maybe these are people who work from home and they don't have a lot of interpersonal interaction in their lives, and they are just looking for people that they can talk to and discover interesting things from and share links with throughout the day. At some point, we start to cross into the commercial territory, but this is kind of fuzzy because there is not always a hard line here.

People who have an online business or a website, sort of foot and bolt camps where they have personal and business related interaction. Maybe they are dropping links to their blog, maybe they are talking about some of the services they are selling, maybe they sell photography and they are posting their pictures or maybe they make videos and they are trying to get people to watch them online. . And again, it's sort of a mix somewhere between business and personal.

That hybrid model is probably where you'll find the most activity, where most people are. They will talk about their personal lives and then they will mention things that they are doing with their business websites, or something like that. There is sort of a cross promotion there.

Next, we get into the level of people who are strictly business. These are people who are on Twitter on a company account. It's either one person or multiple people running an account, and they are using it for some level of business. And there is a lot of variation within this particular group.

They could be using it as a customer service avenue where they are looking for a people who happen to have problems relating with their service, so they are scanning for specific keywords and trying to solve those problems. Maybe they are selling products and they are looking to help people solve their issues. For example, someone who sells spyware software may be scanning Twitter looking for people who are constantly complaining about spyware. Then they drop little hits to them about how to fix a particular problem. That sort of thing.

Probably the farthest away from that is the strictly commercial level, where people adjust them Twittering out sales promotion, different things that they want to get to get a conversion on there, maybe it's products that are on discount, maybe it's like liquidate, they just run a really cheaper discount or liquidation things like that. Something like Woot is a very good example of that, where they are just Tweeting out products, and that's it.

Eric Enge: So, when someone is that commercial do they get much of a response?

Michael Gray: It depends, being a commercial account is pretty tricky. You've got to setup that expectation when you start the account. If you start out being an extremely personal blogger and you switch to strictly commercial, you are probably going to get some resistance. If you are a thought leader in the space and you mix in some commercial links, sometimes you do get some backlash.

An example of this is a Guy Kawasaki's feed. A lot of people go there and are expecting just to hear Guy give information about businesses and creative stuff like that. But Guy also has a couple of websites, Truemors and Alltop, where he will go through and Tweet out anywhere from four to 12 links to any of those things every day. Some people get pretty upset because they are expecting to hear Guy talking about business, not Guy pimping himself. So, he will get some backlash, but he just tells them, “hey, it's me, I am doing these, so you've got to kind of manage that expectation.” If you are just doing sales it really helps if you are a very well known brand so people will care.

Woot is well-known online and they are online just to sell their product. You are going to start out from day one and you are probably going to have a lot of resistance. The way to overcome that is to give out good, time-sensitive deals. If it's a unique deal that you can get that no one else can, you would do better with that, depending on how good the deal is.

The more unique and more valuable the ad is, the better off you will be with a commercial account. Most people don't have the ability to offer those deals, so they have to find a balance between being helpful, giving out information and mixing in a little commercial material.

Eric Enge: So, where are you in this spectrum?

Michael Gray: Most of my accounts are mixed accounts. My main account, which everybody probably knows me for, is for experimenting more than anything else. I will try a lot of different things, and sometimes people probably wonder what the heck I am doing. I talk with people on the account and that's fine, people can ask me questions and that sort of thing, but there is a lot of experimentation going on there as well.

Sometimes, you will think I am hitting the bottle or something, but there is lot of experimentation going on, so take some of the things you see with a grain of salt. The other accounts that I run are all commercial accounts, but I don't have anything that's strong enough to support a commercial brand on its own. And, I just don't think that that's necessarily where most people want to be. You get a lot more interaction if you are willing to engage people at some level.

I think one of the most important things that people can do is always answer back. If someone asks you a question that has to be answered, you should really answer it. It's not only common courtesy but it's just a really good way to interact with your community. And again, I think it's really an outreach program and it's a great way to build a profile that's useful to people.

One example is an account called BBGeeks, and it is all about BlackBerries. Basically what they are doing is scanning for people who are having problems with their BlackBerries. And if they have an answer to their problem in their blog, they will Tweet the person back give them a link to find the solution. Sometimes they will send them to a competitor's website or an informational website. They have a series of questions and answers that they have links for and that's just a really easy way to help someone.

Eric Enge: How much time does it take to get somewhere with an account? I mean, you have spend a lot of time on it, correct?

Michael Gray: Sure. Getting an account of the ground is probably the hardest part because there are lot of people who are looking at Twitter for a short term gain, and basically they will go in and they are just looking for spam. So, you can't go in and make thousands of friends and expect people to take your accounts seriously. You've got to grow it gradually and you have to learn how to work the tools better over time.

There are a lot of tools out there that let you Tweet at specific intervals. Say you know you are not going to be in front of you computer tomorrow, but you want to make your account still look like it's alive. There are programs that you can use that actually will help you do that. The easiest way to do that is to use something like Twitter mail and LetterMeLater. Basically, what it does is set out a secret Twitter account and a secret email account that you use to send an email to your Tweeter account at a scheduled time. Another service that I have been using is Easy Tweets. It's probably a lot more integrated than a lot of the stuff out there, but the downside is that it's is a paid service. With this you really have to decide how many accounts you are doing, how much time it will save you and whether or not you want to pay for it. If you are running more than one, it's probably worth looking at a commercial solution, but if you are only running one account, you might be able to get by with some of the free tools.

Using searches is another key feature. You can go in and setup things like a Tweet-beep to monitor for particular searches, and it will send you an email once it hits a certain threshold. You can also use an RSS feed for search. So, if you want to go through and look for everybody who is selling accounting software and everybody who is selling accounting software that they hate, you can go through and setup an RSS feed for that.

All you have to do then is scan the RSS feed once a day. This way you don't have to be sitting in front of the computer all day monitoring that kind of stuff. If you use tools to make it, you can check-in two or three times a day and respond as needed. Again, as the account grows and gets more active, and you get more followers you may have to devote a little bit more time to it. But once you've got the background down on what keywords you are looking for, you really can do it in probably an hour a day.

Eric Enge: That isn't too bad an investment, depending of course on the return you get out of the whole exercise. What about strategies for building followers?

You talked about good content, but you can go beyond that, right? You've got to seek people out and communicate with them proactively at some level too?

Michael Gray: Sure. Now, when you are an SEO, probably one of the best things that people do is recommend what to start with. They say you are going to need link, and where you should start looking for them. So, you mine the backlinks of your competition. You should just take that same theory and apply it basically to Twitter.

If you are running in a particular account and it happens to be a travel account, you can go out and find the big travel accounts on Twitter like Jet Blue and Southwest, and you start mining their backlinks and following the people who are actually there. There is a debate as to whether you should follow all of your followers or not. Some people say it's right, and some people say it's wrong, but that decision is up to you. The thing that they will say is, if you are running a commercial account, the object is to get people to follow you.

Following people back is probably just the smart thing to do. Now, are you going to do that manually? That's up to you, they are getting in a lot of programs that help you do that automatically so that you don't have to. It just depends on how much time you have to invest, how many accounts you are running and whether or you want to use it or not. But, mining the backlinks of your competition is probably the smartest place to start. And then, I would almost always follow anybody who asks you a direct question. That's just because in the end, it's an easy way to get somebody to follow you.

Eric Enge: Right, it makes sense. Now, what about this notion that someone follows you and later they drop it. Do you go back and proactively stop following them?

Michael Gray: Here is the way the followers work: Twitter tried to combat spam because they had a bunch of people who would go in, follow a million people and several people would follow them back. Then, if you've followed enough people and you've got enough followers, you would show up on any of the top Twitter people list.

You were able to get thousands of followers without doing any work. Twitter had to make it a little harder; so they imposed some limits as to how many people you are allowed to follow. You are now allowed to follow as many people as you want up to two thousand, so you can basically have one person following you and you can follow two thousand people. Now, that looks really odd and it makes you look like you are a spammer when you do that. So, I wouldn't recommend anybody go anywhere near that particular level.

Go slower and in smaller steps. And again, Twitter is actually taking some steps to try and combat that. If they see a lot of activity, they will go and flag an account. Then you will have to ask for a manual review for them to let you back in. So, add your followers slowly in the beginning. Once you hit that two-thousand level, you are blocked.

The only way you are able to get past two thousand followers is to have at least two thousand people following you. You are allowed to follow approximately 1.1 times as many people as are following you. Until you get more than two thousand followers, you are not going to go beyond this. So basically you are able to grow your follower account by 10% per day.

It's up to you if you want to un-follow people when they stop following you. If you are trying to grow an account and you are using some sort of software to follow people, you are probably going to have to drop the people who aren't following you. You have to play the numbers' game.

Eric Enge: Right, absolutely. They are basically limiting your ability to grow.

Michael Gray: Right. And, they sort of put a governing mechanism in, and I understand why they do it. But, you've got to understand what your motivation is. I wouldn't sit there and say, "oh this person un-followed me, sod I am going to un-follow them tomorrow." That's not necessarily the way to do it, but if you are looking to grow your subscriber base you are going to have to start dropping your followers. That's just a reality of the game.

Eric Enge: Right, absolutely. So, let us just focus some more on the types of commercial purposes. What are your thoughts on networking and using it as a tool to meet people?

Michael Gray: There are lot of people who are doing these things they call Tweet-ups , which are basically meetups of people are at a particular event. Usually it's a group of people who have a particular shared interests and they are all going to be in the same area doing something together. I have even seen Tweet-ups before movie premiers where a bunch of people get together, hang out and talk about the movie, and then they go to see the movie that night.

That's another interesting little thing. It's basically a way to try and build your brand within the particular community. It's hard to attach a dollar value to something like that because there is no real conversion. But if more people knew you when you come out of that particular event, then that's probably a good thing.

The key thing everybody wants to know is who are the "A-players" in their space. If they are on their radar and they write an interesting post, A-players will hopefully link to them and mention them, that sort of thing. That's a game that everybody is playing, where interacting within a community is a good thing. So, if Robert Scoble is an A list player in the tech space, whether you like him or not or whether you think he is important or not, is irrelevant. If he has enough people following him and he Tweets something about you, a big mass of people are now going to know who you are and go follow that link. And that's really the ultimate goal of this, to get him to Tweet about you.

Eric Enge: The re-Tweet process.

Michael Gray: Now here is an interesting thing with re-Tweets. I will give a little secret to everybody who is reading. If you are looking for people to re-Tweet what it is that you are Tweeting, do a search for the words re-Tweet, or people sometimes abbreviate it RT, and whatever your keyword is. So, if you have a website that says movie reviews, go to the Twitter search and search for everybody who says "re-Tweet movie review." That will give you a list of people who found movie reviews interesting enough that they re-Tweeted them, and you should probably friend all of those people.

Eric Enge: Right. They might be interested in your topic and get some of them to follow you. , You might get a re-Tweet out of that, and now you are exposed to their audience.

Michael Gray: Right. And, it kind of grows exponentially. If you can get a really good network of re-Tweeters following you that can really grow your exposure. It's that old Malcolm Gladwell thing; you've got to get the connectors.

Eric Enge: Yes, well absolutely. So, just to go back to who you mentioned before, if you know what Robert Scoble re-Tweets and you have something related, then you might be able to get into his huge network.

Michael Gray: Yes, he has something like 60,000 people following him, so if even one tenth of those people go and re-Tweet your link, that's a huge amount of traffic to your website. Hopefully people might think it's interesting and subscribe to your blog or start following you.

Eric Enge: Right. Now, is that the basic strategy for using Twitter to get traffic to their site?

Michael Gray: That's one of the basic strategies. The key is to give away an interesting bit of information. You've either got to solve someone's problem or, if you are doing something commercial, give them a really good deal. The better the deal, the more likely someone is to re-Tweet it. If you are going to tell somebody, "hey, there is a cruise next week and it's $5,000 a person," you are probably not going to get any re-Tweets.

But, if you offer them a 7-day cruise for $200 because the ship is having a hard time filling space, you are probably going to get a lot more people re-Tweeting because there is value-add there. So, you've just got to measure what it is that you are giving out. There are a lot of people that say you should never ask for a re-Tweet. I am of the opinion that no one is going to know you want to be re-Tweeted if you don't ask. A lot of people have that personal threshold that doesn't let them actively sell. If you are trying to get something across and you want someone to do something, you should suggest it at the very least.

Eric Enge: Do you have a suggested way of doing that?

Michael Gray: If you are going for the re-Tweet model, there are a couple of things you have to keep in mind. One is to leave some space. Twitter basically gives you 140 characters, so you want to leave some extra space so that people can re-Tweet, add your name, or add their own little message onto something. Don't max out at 139 characters because that's not going to be as effective. Somewhere in the 100-character range would be ideal. You've just got to try and be as direct as you possibly can.

And you have to practice that a little bit, but it's a skill that you just pickup slowly over time. And again, ask people to re-tweet if they find the information helpful, that's a really simple way to do it. There are some people who a little bit more not straightforward about what they do.

One example that I love to bring up because it's always so amusing, is Matt Cutts at one point posted a link to a Digg page and said, “maybe someone should call their congressman about that.” Now, when I read something like that, I put on my Judge Judy hat and say you don't send me to a Digg page unless you are looking for me to Digg an article. Matt just claims that he was really, genuinely interested in the local politics of that particular case, and I will leave that up to everyone who is listening and reading to decide. But I am of the opinion you don't send me to a Digg page unless you want me to Digg something.

Eric Enge: Well then, the advice might be that if you really do have an interest, you should be a little more explicit?

Michael Gray: Yes. Now, the one thing I will say about social media is that there is an upside and a downside to it. If you ask most normal people to stumble or vote something up, they will do it. Sometimes within the SEO world, there are quite a bit of haters there. So, if you have an account and you are known as an internet marketer or an SEO, I would be extremely careful about asking people to Digg, mix or stumble upon your stories, just because there are a lot of people who hate on you. They will specifically go and watch your account just to figure out who your clients are, what websites you own and they will specifically go and down vote you.

Eric Enge: Other traffic strategies for Twitter?

Michael Gray: Don't push your followers too much, and also be aware of time-zones. If you are Tweeting something at five in the morning, there are probably not a lot of people who are going to see your Tweet. Most people are now following more and more people, so they are not going to go back and look through all of your Tweets.

Tweet your important stuff during primetime, when people are at work. Be aware that there is an east coast and the west coast. Sometimes if I find something that is important, I prefer to Tweet it two or three times a day, just so people all across the country can see it. The other thing I will say is don't Tweet only about yourself. If every single thing coming through your Tweet profile is a link to your blog, that's probably not adding a lot to the space. Tweet other people's stuff in there too. Sometimes just Tweet out stuff that is interesting or funny, just to make your scream a little bit more interesting to be a part of.

There is a little bit of a debate over whether everything should be a single-person account or multiple-people accounts. If you are running multiple-person accounts, let people know at some level. I know a lot of the bigger, commercial accounts like Jet Blue will change their status to say now Tweeting Jennifer or something like that. This is just so people know who is actually there so they don't come back and say, "hey, you told me to do this four hours ago," and the person at the other end of the screen has no idea what you are talking about.

So, if you are going to be doing that kind of thing and you are big enough, let people know. The other thing is to be aware that this is the social space and some negativity goes on here sometimes. . You do sometimes get people who don't know where the line is and they cross that, so be careful about what information you are giving out. Never give out anything too personal, such as your phone number or your address. For 99.9% of for the population on Twitter this is not an issue, but there is always that possibility.

Eric Enge: Right, your troll?

Michael Gray: Yes. You've got to be aware of that.

Eric Enge: What about using Twitter as a tool to build links, is that something that can work as well?

Michael Gray: It is hard asking directly for links. You are probably better off working it on a social angle and just saying, "here is an interesting piece you wrote that helps you do this." Asking directly for links usually doesn't work. Twitter does most of your things through a tiny URL that is no-followed, so you are not getting any real link juice through there.

I believe search engines are monitoring that sort of thing and toolbars are actually capturing the URLs that are going through there. There are actually some services that will go through and reverse-engineer some of the tiny URLs to let you know how much things are being tweeted. This way you can gauge the effectiveness of a website. If you actually put someone's URL in Quarkbase, it will go through and scan Twitter to see if people have tweeted that particular URL. So, if you are looking at someone's website, maybe you want to see if this person is someone that you want to try and engage a relationship with based on how active they are in Twitter. If the person is active in Twitter and you are looking to grow, there are some keys that will tell you if you should try and engage this person a little bit more or not.

Eric Enge: So the way I think about it is absolutely don't go and directly ask for links, but if you have built a good following and you are able to periodically get other people to re-tweak your stuff, then it is like a PR mechanism.

Michael Gray: Yes, it basically just carries forward. Again, if you know that there are bloggers or reporters who have Twitter streams and they are covering your sector, do what you can to get your URL in front of them. I have a couple of different bloggers who have picked up things in front of them and linked to it because they thought it was an interesting piece. So, it is definitely worth doing.

Eric Enge: Yes, well very effective overall. Any additional key tips you want to give people before we wrap up.

Michael Gray: Don't be afraid to push hard, but don't push too hard. You don't want to come off as being super-aggressive, but you should definitely get involved. Go through your referral logs, see if there are people who are re-Tweeting your stuff who are sending you traffic and make sure that you are friends with them. Go out of your way to thank someone for re-Tweeting your link. That's the kind of thing that people appreciate.

At the end of the day, it is a social mechanism and people are involved. People on the other end of the screen are guilty of the same things we are. They like to have their ego stroked, and every time you mention them you make them feel good. Controversy is a tricky angle, not everyone can pull it off effectively. It is very powerful if you can find out what other people are monitoring. If you happen to be a pro-Bush person, whenever an Obama speech is on, go ahead and aggressively engage the Obama supporters.

That way they are going to start re-Tweeting your name out there and that's going to get you some exposure. Again, you've got to make sure you know what you are doing, you have to know where the line is and you have to have thick skin.

Monitor the key words that people are searching for in your subject. If you are a TV blogger, look for people who are blogging about Lost when Lost is on. If you are an entertainment blogger, look for people who are Tweeting about the Oscars when the Oscars are on. Use those hash tags sometimes, which is when you put the numbers on and you put the word after. People would use them to group everything together. Basically, the hash tag is your way of saying that your tag is part of this subject. Do everything that you possibly can to be creative about getting traffic.

Eric Enge: So, if you are in Twitter and you want to find all the people who have searched or Tweeted on the Oscars, what exactly are the mechanics of doing that?

Michael Gray: There are two ways you can do it. There is a service called hash tags, and you will go back to hash tags, put in the word "Oscars" and it will show you everybody who has used the Oscars hash tags. The way that is easier and probably going to get you some more people, but not as refined, is just going to the Twitter search mechanism and typing in the word "Oscars." It is going to list everybody that's in there doing it. There are a couple of Twitter monitoring services that monitor for trends on Twitter.

So, if you are looking to play that game, you should definitely be subscribing to those and they will tell you what is actually hot. Again, there are a couple of services like Twitter grader and Mr. Tweet that are trying to match people together who have similar interests. Sometimes they work, sometimes they don't; it all depends on how big your particular niche is. If you are playing in a big area like an iPhone, you are probably not going to have a problem finding anybody, but if you are doing something like a knitting school, it is probably going to be a little harder to find people who are interested in that particular thing. Sometimes people just aren't there.

Eric Enge: Excellent. Anything else?

Michael Gray: Oh, I think that's pretty much it. Again, the opportunity is here. Don't squander it just Twitting about what you had for lunch and what you are doing with your cat. Use it to find ways to grow your exposure or find ways that you can turn it into commercial enterprise. As an example, I had a commercial account that I started last year.

I pretty much ignored it and it babied up three hundred followers. I had some free time studying in January, and I started working the account. I basically went from 300 followers in the middle of January to just shy of 7,000 at the end of February. I worked the account, I got people who are re-Tweeting commercial links and I e got people who are re-Tweeting affiliate links. So again, if you are willing to put the time in to experiment and try different things, it can work.

Eric Enge: Great, well thanks a lot, Michael. I appreciate it.

Michael Gray: Thank you!

About the Author

Eric Enge is the President of Stone Temple Consulting. Eric is also a founder in Moving Traffic Incorporated, the publisher of Custom Search Guide, a directory of Google Custom Search Engines, and City Town Info, a site that provides information on 20,000 US Cities and Towns.

Stone Temple Consulting (STC) offers search engine optimization and search engine marketing services, and its web site can be found at: http://www.stonetemple.com.

For more information on Web Marketing Services, contact us at:

Stone Temple Consulting
(508) 485-7751 (phone)
(603) 676-0378 (fax)
info@stonetemple.com

Omniture's Chris Zaharias Interviewed by Eric Enge

Mon, 2 Mar 2009 08:52:17 -0500

Published: March 2, 2009

Chris Zaharias runs Sales for SearchCenter SEM solution, and is a seasoned sales executive whose working knowledge of search marketing dates back to the late 90's. Chris has more than a dozen years of ecommerce, online marketing and enterprise technology experience, including domestic and international management positions at Netscape, RealNames - an early pioneer in the paid keyword space - and Efficient Frontier. Chris is an avid SEM writer and blogger and loves nothing more than knowing what's new in search.

Interview Transcript

Eric Enge: What are the areas where search and traditional marketing are intersecting?

Chris Zaharias: Advertisers have realized that there are a lot of interactions between search and other forms of marketing. If we were to talk about this four years ago, I would have said the interactions between search in the other forms of marketing and an advertiser's own business were immaterial, because the greatest opportunity for growth in search had nothing to do with that.

It had everything to do with building out keywords sets, making sure that you have the right ad copy, sending the traffic to the right page and doing all of the SEM 101-type of things that most advertisers did. What's changed now is that advertisers, at least in the US, are already buying most of or all of the keywords that they should be. They've done a lot of things, like structuring their campaigns correctly and participating on all three or four of the major search engines, as opposed to just one or two.

Those types of organic growth opportunities have dried up. But at the same time, the overall growth in search as measured by metrics like the number of searches per searcher have also come to a virtual standstill. There has been a lot more focus on how to properly manage search campaigns and what to do to set yourself apart from the competition. The competition has figured out all of those SEM 101 types of tactics because of all the writing, the conferences and the increased focus on this particular.

So this is where it really gets interesting to try and understand search marketing as part of an overall marketing mix and to use that knowledge to get more out of search specifically.

Eric Enge: To summarize, growth was sort of brainless for a while, but that is not good enough anymore.

Chris Zaharias: Yes. I used to always say that all people needed to be successful in search in the first few years was a pulse. It really wasn't that hard because it was such a green-field opportunity.

Eric Enge: Right, but in the current environment, where a lot of that is more mature, it's not sufficient. Your company might get some gains out of that, but it is not differentiated.

Chris Zaharias: Right. There is also the dynamic of the search engines themselves. Because of the slowdown in growth and the continued imperatives on their part to grow their businesses, they have now started to operate their engines with an eye towards yield maximization. There are a lot of things that search engines have introduced that increase their monetization but require more complexity in campaign management.

They have also introduced more requirements for precise and holistic management of the campaigns. This is done in order to create yield management on the part of the advertiser to combat the monetization moves that the search engines are making.

Eric Enge: So you can still get a competitive edge by better managing these second generation opportunities (problems) than the next guy.

Chris Zaharias: One of the key opportunities for growth and improvement of campaigns is conversion optimization. When you look at any given sector in paid search, you will find there are up to order of magnitude differences in the conversion rates that advertisers are getting. SEM tactics being equal, the advertisers that can convert the traffic at the highest rate are going to win.

Whereas the focus for conversion rate improvements used to be just iterating your ad copy or trying to see which page converted best, there are now requirements to get much more scientific about that conversion optimization process. One area that jumps out is known as Multivariate Testing.

Multivariate Testing is testing all of the potential variations of a landing page or a session. This can include the ad copy, landing page, and all of the different pages in the conversion funnel. And on each of the pages of the site there are multiple elements that can be tested, such as the offer itself, page layout, placement and size of graphics, font size, color schemes, and much more.

Breaking down all those variables into a set of tests often yields conclusive results as to what combinations of elements best convert the traffic.

That's the reason Omniture acquired Offermatica in late 2007. Omniture realized that the opportunity to make big strides in conversion optimization was there, but that it would require technology that marketers could manage to take control of and continually iterate through the testing process. So, one big area is conversion optimization.

That industry has been getting more refined in understanding how to manage paid search campaigns. Historically, people started out optimizing paid search to clicks to pay for their traffic. The next wave was from 2003 to 2007-2008, and it concentrated on conversion for people who are optimizing to a cost-per-order or CPA metric.

Now a new area of opportunity for greater efficiency is getting more fine-tuned in the metrics. When you look at a typical keyword campaign, you will find that high-volume keywords tend to have enough data to optimize your conversion metric.

But for the midsection and the tail of your keyword portfolio, you tend to not have enough data to optimize to conversions and you essentially have to use your best human intuition to make decisions. That keeps you from having the efficiencies you want in your campaigns.

We are seeing a lot of people use micro-conversion events, also known as leading indicator metrics.. What that might mean in a retail setting is rather than just optimizing to the conversion event, you might try to understand the ratios of events such as shopping cart fill or viewing a product detail page. You can assign values to the leading indicator metrics. You can optimize using a lot more data than if you were to just look at conversion data.

Eric Enge: That can include things like newsletter signups or contact box requests.

Chris Zaharias: There are literally hundreds of metrics, but the point is defining and assigning values to those metrics. Then you have to optimize to those metrics, which is something that very few people are doing, but something that everyone should do. It just increases the efficiencies of your search marketing campaigns that much.

As result, integrating your web analytics with your SEM Campaigns becomes very critical. If you can't be aware of those metrics or analyze data to understand the ratios of those micro-conversion events then you can't optimize.

The third area is this notion of trying to understand SEM within the context of your overall set of marketing initiatives and your overall business. There are consistent patterns of interaction between TV advertising and search that you can track, analyze and optimize. One example could be overlaying search traffic and conversion data against TV advertising data. It's clear when you look at this data that the TV activity drives search activity in a meaningful way.

What that means from a campaign management standpoint is that you need to integrate those other forms of marketing into how you manage paid search. If you know you are going to be running a set of TV campaigns, for example, you may want to adjust your bid management rules in advance of the expected rise in traffic during that time.

You could say the same thing for whatever medium you are advertising in, they are all going to have an impact on search. For that matter search is going to have an impact on other forms of marketing.

Another thing to remember is that you can't be truly efficient in the management of your search campaigns unless you take into account the direct and indirect online value that you accrue from those activities. But you also have to take into account things that might happen offline. This has been a big challenge, particularly for the SEM community, because it is comprised primarily of SEM-specific tools, vendors or agencies.

When an advertiser sees that there is a competitor that has a big offline presence that appears to dominate the paid search space for their sector, they will always ask how can they afford to consistently spend 40% more than me when I am doing everything right and taking into account all my online value that I am getting from paid search?

The answer is that the advertiser is able to understand the value that they are getting from their search campaigns in their physical stores or something along those lines.

Eric Enge: The classic scenario is that someone does a search for "women's clothing, and then, they want to go see it, they want to touch it. They decided what they wanted to buy based on search, but then they went in the store to complete the transaction.

Chris Zaharias: Yes, the online-only retailer is always going to be at a disadvantage, because they can't consummate the subset of transactions that are going to happen offline. Or, if you are looking at two advertisers that are both in search and have online and offline presences, the one that can instrument their campaign management to actually understand the exact relationship between a particular keyword and a particular offline transaction is going to be much more efficient than one that is limited to knowing that 20 percent of its business happens offline and therefore assumes that each search driven online transaction has a value of 1.2. That would be a very approximate way to do it.

What advertisers need to do is integrate their point of sale systems and their physical stores with their analytics and search marketing systems so offline data can be piped into their analytics and SEM environment. If they do this they can actually take into account the direct relationships between a query and the ultimate offline transaction.

Eric Enge: That sounds like a fairly complex process. Say someone comes to the website and they fish around for a little bit without leaving a name or anything like that behind, how do you create the correlation to when the transaction takes place in the store?

Chris Zaharias: It requires a very specific method of tracking. Let's use a call center as an example. If we are talking about a phone order, what you might do is have a particular 800 number or coupon code that you show to the person that came through Google campaign X.

This way, when they call in you can correlate that back to a particular keyword or campaign just by virtue of the number they call. If you are trying to entice people with some sort of promotion, you might want to tie back that promotion to that promotion code or a particular keyword, or particular campaign, or particular engine.

All you are doing is correlating with the particular click-through that happened from paid search to make sure that you take the value into account. The main reason I came to Omniture is because I felt like the system to do this in a much more exact fashion was going to become critical. The acquisition of Visual Sciences gives us a system that integrates with the point of sale system or the call center system. Then it sends the data from that system to our SiteCatalyst infrastructure, which is the data hub for analytics.

Because Omniture SearchCenter is integrated with SiteCatalyst, we can have SearchCenter act on any source of data, including that call center offline data as it figures out how to bid on keywords.

Eric Enge: Right. Calling a custom 800 number is actually much simpler than capturing the coupon code at the point of the sale because it is fully automated. The other thing to realize is that even though you have improved your situation greatly, you are still only capturing a percentage of the transactions. You are not capturing a hundred percent of the transactions, you are just capturing a chunk of them.

Chris Zaharias: That is true. But every increment of efficiency is good to have.

Eric Enge: Absolutely, the more information the better. Even though we know that analytics is an inexact science and everybody talks about how relative measurement is king, you are still better off every time you remove a source of error.

Chris Zaharias: Right. So, in addition to taking offline call centers and what data you have into account, to understand its impact on the search marketing campaigns and react to it, there is also the continual goal of targeting to or taking advantage of all the targeting capabilities that exist in search.

What you might do is try to use that offline data to understand the impact of search in a particular geography. We have certain clients correlating conversion data back to the geo-targeting they are doing in Google to perform geo-analyses of offline conversion data. Because the response from search tends to drive offline conversions of different amounts in different geographies, that's going to affect how a company actually implements and iterates through geo-targeting in the search engines.

Eric Enge: You might spend more on your Boston campaigns then your San Jose campaign, for example?

Chris Zaharias: Correct. That's one of the main areas where Omniture is going with search and integrating with both SiteCatalyst and the other pieces of Omniture's platform.

Eric Enge: People used to believe that the CPA was everything. Don't get me wrong, it's a great and reasonable way of measuring things, but why not use real revenue goals instead? Once that occurs, the next logical thing is to use gross profit. On each step along the way, you are eliminating a source of error.

Chris Zaharias: Exactly. I just had this conversation with someone in the credit scoring space a couple of weeks ago. They are spending $25 million plus a year on search, but the one thing that they haven't been able to do in their campaigns is optimize something other than CPA. They obviously want to optimize to profit, which requires that they take Lifetime Value (LTV) into account.

Ideally we would take the LTV data, get it into SiteCatalyst, and then use bid rules that take into account the variable profit of each transaction that they are going get from search. Say you sign someone up and they are paying a monthly fee for credit monitoring services. Compare someone who stays with them for just a one month trial then cancels, with someone who stays with them for three months, with someone who stays with them for 12 months and buys an additional product.

They are very different values, and unless you take those different values into account, you are never going to be as efficient as you could be in your search campaigns.

Eric Enge: The people who convert as a result of one group of keywords may be more likely to be long term customers than those that convert on another group of keywords.

Chris Zaharias: Right. Before the whole macroeconomic shift, the sole focus of paid search campaigns tended to be top line growth. But now we are hearing a lot more people who don't care if it makes them money or saves them money, as long as they increase profits. These types of discussions about increasing efficiency, regardless of where they come from, are a lot more prevalent.

You talk about how the search engines are trying to monetize their inventory. The need for analyzing site traffic data to find and implement negative keyword strategies is a lot more acute. And you could argue that an advertiser who is doing everything right should have as many negative keywords as positive keywords. Typically, however, an advertiser will have maybe one negative keyword for every five or ten positive keywords.

If you can help people who are making use of broad match, find all of the areas where the search engines are not properly matching the ad to the query, you can help them build out those negative keywords. This way you can drive a lot of efficiencies that the search engines are unlikely to show you.

Eric Enge: There are a lot of flavors of those kinds of things. It could be as simple as deciding between broad match versus phrase match.

Chris Zaharias: The key to finding the right match type strategy is to think of a broad match type as a baseball farm team. You have a lot of players in there, some of them are going to pan out and some of them aren't. But you need that farm league system to continually grow your campaigns. It's going to feed both the positive keywords as well as help you find all of the negative keywords that you want to avoid at all cost while using broad match.

Have you heard of this automatic match beta that's been going on at Google? It is very emblematic of the direction that the search engines are going in. Some people are discussing whether or not bid management is dead. The real discussion that should be going on is whether or not the keyword is dead. In an ideal world, search engines would go to an advertiser and say, you are in this vertical, let us figure out all the keywords and all the traffic that's appropriate for that vertical.

That would be great for the search engines because it would allow them to cut out the middleman and get a lot more revenue than the conversion data would merit. The reality is that's the opposite of what advertisers should do. Advertisers should take advantage of the ability to target at the individual keyword level to find the right buying intent to go after and capture. So search engines and advertisers are increasingly at odds with each other from a strategic perspective. This is where the online business optimization platform that Omniture is bringing to bear becomes much more compelling.

Eric Enge: Right. So, for an example let us say that you are an online shoe site and you focus on selling men's shoes. Let's say you don't sell slippers, so you really don't want to be matched up with them and you never use the word ‘slippers' in your campaigns. But, with automatic matching, you might end up showing up for that.

Chris Zaharias: Right. Think of Automatic Matching as a beta system whereby Google looks across accounts for unspent budget and then broadens the ad-to-query matching as necessary to spend the remainder of the budget.

Eric Enge: Let's talk a little bit more about TV driving search. Let's say you run a Super Bowl ad. You should see a pretty good spike in your search traffic as a result, particularly if you are smart and prominently featuring your webpage address in the ad. But in addition there will be people who search on your brand name. So, what's the best-practice for trying to measure that kind of impact?

Chris Zaharias: Well, I think that being a very unique advertising scenario, where there is an immense amount of money you are spending in an incredible short period of time, it's definitely a bit different than just the standard ongoing TV campaigns. If you are going spend that much money for a 30- or 60-second spot, you need to be confident that you are going to be able to convert that traffic at the absolute highest rate possible.

In that specific case you would be criminally negligent if you didn't do some multivariate testing in advance of that TV spot to make sure that it converts at a very optimal rate. The very first thing to think about would be testing to make sure that you have the right flow, the right offer and the right formation of pages that optimize that expensive traffic you are going to be paying for.

Another thing from an SEM management standpoint is to know exactly when your ads going to run. You are going to want to make sure you implement a set of management rules for your SEM campaigns that take into account what's likely to happen that day in advance of the Super Bowl.

You might have to up your budget for this period, or you may have to impose a different set of bid rules for a period of time after it airs to take into account what is likely to be a much higher conversion rate than you typically get.

I am also going to revert back to my long standing keyword management principles as the effect of the Super Bowl advertising trails off. And that requires some specific thought and advancement in infrastructure. Let's say you go into SearchCenter and you look on the calendar, and you impose a specific set of bid rules for this specific period of time. That could either be a period of days or a specific set of hours within the day that correspond with when your ad is going run.

Depending on the type of TV advertising that you do, you are going to have to think about different keyword sets, both positive and negative. If ever there is a time when you need to be aware of all possible misspellings of your brand name, this is it. People that are four beers into watching the Super Bowl they might recollect your brand name correctly when they see an ad or they might have variations; making sure you capture all of that is going to be critical.

Likewise, making sure that you have all the negatives set up to avoid getting traffic that you don't want becomes even more important if you are spending about 2 million bucks for a 30- or 60-second spot. You might have specific reporting that you need to see during and after the Super Bowl ad is run.

Obviously, if you are an SEM manager at a company and all of a sudden you are going to be spending 2 million bucks on a Super Bowl ad, you can bet that the reporting requirements of your senior management around your SEM campaigns are going to be very different during that time than it has been to date. You need to be able to provide near instantaneous detailed executive reporting so that they can understand the value of that investment.

Eric Enge: Right. So, you can do differential measurements, correct? You can compare the results on the website for the 7 days following the event verses the results for 7 days previous to the event, for example.

Chris Zaharias: Right.

Eric Enge: So even though we may not have a precise number on the income that it brought in, we do know that income was 60% higher during that 7-day period, than the prior 7 days.

Chris Zaharias: Right. Every metric that you can imagine from transaction volume, to average transaction size, to geographic measurements to conversion rate is going to be looked at.

You need to be managing your SEM campaigns in a manner that takes the interplay into account, not just with offline marketing channels, but with other online channels as well. There is a value in someone who might sign up for a newsletter on your site as a result of seeing the ad or who might play a game that you develop on your site specific to that Super Bowl ad, and whose goal is engagement and brand awareness with the target demographic.

Being able to have your SEM systems integrated with your email campaign management systems becomes very important if you are going to achieve the engagement and conversion goals that you have.

Eric Enge: What if we have an ongoing TV campaign rather than a spike event?

Chris Zaharias: In that case you really need to make sure that you are understanding, capturing, and reacting to the value that you are getting from the offline campaigns. You will want to have your SEM management set up to be able to bid in the way you want to during the times those ads are running. Obviously, you will typically get spikes in traffic and changes in conversion rates relative to other times when you run those ads.

If you just let your system react to it, the system is inevitably going to react poorly because it has no reason to know that anything is about the change. It will react slowly and then efficiently to changes in traffic conversion rates.

What you should do is tell the system that you want to bid up or relax your CPA requirement in anticipation of a higher conversion rate or higher average order sizes during this period. Doing that analysis and implementing the rules to coordinate your TV campaigns with your search management is going to yield better results than just letting it ride.

Eric Enge: Right. It could be something as simple as being willing to take a lower margin because you expect a good spike.

Chris Zaharias: Yes and SEM systems act on historical data. It's great to use historical data, but only if it's a good representation of likely future performance, which isn't necessarily going to be the case in TV advertising.

Eric Enge: Right, absolutely. So this sort of gets back to Omniture CEO Josh James's presentation at SES Chicago; the notion of the search marketer being the quarterback. Do you have any comment on why it is that the search marketer should be the one driving this process rather than someone else in marketing? Search Marketer as the Quarterback

This screen shot from Josh James' presentation as SES Chicago reprinted with permission)

Chris Zaharias: Search has the largest, most measurable aggregation of buying intent that has ever existed. Because the opportunities for optimization are so much greater, the controls are at the fingertips of the advertiser. This is all important because you don't get concentrations, of buying intent, expression of buying intent, and variations of buying intent in any other channel in the way that you do with search. So that aggregation of intent, the ability to analyze the data, the ability to optimize along multiple parameters, , that's likely to be the path that the rest of marketing will take.

If everyone today is worried about Google becoming a monopoly, those worries are overblown because Google actually already is a monopoly for all intents and purposes within the world of search. It has an 80-95% market share in Europe. If you count their distribution partners, they're almost at 80% in the States, and they keep taking more market share.

What's really important to know is that the real time action marketplace that's measurable and actionable within search is likely to grow into all other forms of marketing. We are a couple key technological advances away from advertisers being able to understand individual reactions to things like radio advertising, or print advertising, or billboard advertising. At which point, those offline marketing channels become measurable and actionable marketplaces where advertisers can react at the data they are seeing.

It's much more likely that the search engines with the most efficient advertising platforms will grow into those other areas. And you are already seeing that with Google's acquisition of DoubleClick, with some of the interesting tests they are doing around deployment and measurement of radio advertising.

When you see things like quick response codes in Japan, which led to tens of millions of consumers reacting to print advertising by scanning a barcode with their cell phone, it becomes very real to think about the systems in the action environment that is search. The role of the search marketer as a quarterback is not just important because it's this biggest aggregation of buying intent right now, but also because the measurement, the real time optimization and the auction environment within search is likely to be the environment that's subsumes the rest of advertising, not just online advertising.

Eric Enge: So even though the rest of advertising dwarfs search from an economic standpoint, the accountability and measurability of search will eventually become the predominant way of doing everything?

Chris Zaharias: Right, exactly.

When you think about the opportunities for advertisers to take advantage of the inventory that Google, Yahoo and Microsoft are aggregating, it's pretty astounding. It is obviously incredibly complex, and it becomes all the more complex for every additional channel that you have to look at.

Eric Enge: Thank you Chris!

Chris Zaharias: Yes, thanks Eric!

Have comments or want to discuss? You can comment on the Chris Zaharias interview here.

Other Recent Interviews

Google's Sandra Cheng Interviewed by Eric Enge

Mon, 16 Feb 2009 09:39:13 -0500

Published: February 15, 2009

Sandra Cheng is a Product Manager at Google where she is responsible for the overall product strategy and roadmap for Google Website Optimizer. She also manages ads optimization related products for AdWords. Previously, Sandra worked at Amazon.com, where she led product management for Endless.com.

Interview Transcript

Eric Enge: So why don't you start with an overview of the program that Google Website Optimizer did with these four companies?

Sandra Cheng: We launched Website Workout last year to raise awareness of website testing and improve the ease with which you can increase the value a website delivers to your prospective and existing customers.

So, during a two week contest entry period, we had asked thousands of advertisers why their sites needed makeovers. Out of all of the entrants, we then chose four of the most compelling cases.

We paired each of those four advertisers with one of our Website Optimizer authorized consultants, who then helped them set up experiments to optimize their website in order to generate more leads, sales or signups, whatever their conversion goal may be.

Eric Enge: So an authorized consultant is a third-party organization like ZAAZ, or someone like that?

Sandra Cheng: Exactly.

Eric Enge: When did you actually start working with the companies?

Sandra Cheng: We announced the contest last June and started working with each of the companies shortly thereafter.

Then we went through a couple of different series of tests before we spent some time to shoot the videos. And we just announced the results two weeks ago.

Eric Enge: Let's dive into one of the scenarios. Can you talk about Extra Space Storage and maybe tell me a little bit about what they felt their situation was, the scenarios that they tried and the results that occurred.

Sandra Cheng: Extra Space Storage is the second largest operator of storage space in the US, and is based in Salt Lake City, and they get about 40% of their customers online.. And as they stated themselves, marketers and developers guessing what customers wanted designed their website.

This is something that I feel is pretty common in website design today, where people tend to guess what's best based on cool ideas or a gut feeling, but never really knowing what's actually going to resonate with their visitors. So, Extra Space Storage decided to test their facility pages, which are the pages that allow customers to find the storage unit that's best suited for their needs.

The original version of the page included pictures, driving directions, the description of different unit types and some big select buttons to choose which one you are interested in.

They then tested three different variations. In the first variation, they increased the size of the picture, added a map thumbnail to driving directions and decreased the size of the information that they felt was less critical to decision making on that page.

In the second variation, they made the picture smaller and tried to bring more things above the fold. In the third variation, they moved the picture, the directions, hours of operation and everything else over to the left-hand side, and had the unit information front and center with larger select buttons.

Really what they were doing was testing some pretty drastic layout changes where the content more or less stayed the same. The first version that they tested led to an increase of almost 10% in conversion rates.

Eric Enge: Right, so 10% is a pretty significant lift for those pages.

Sandra Cheng: Absolutely.

Eric Enge: So, you talked about three different types of strategies that were tried. They tried pretty significant changes, and, there is an argument that could have been made for any of those scenarios.

Sandra Cheng: Absolutely, I agree. We see that a lot in our own testing, and the moral of the story there is that things that are generally regarded as best practices aren't necessarily best for your particular website. They are the result of someone else's success, but they may not apply to you.

Eric Enge: Right. And, I think the underlying theme that drives that is that best-practices may be best-practices when you are dealing with a completely homogenized, un-described audience; but that's not your audience.

Sandra Cheng: Right.

Eric Enge: Your audience has come to the site by a specific path with specific things in mind, and you need to be very, very aware of that.

Sandra Cheng: Agreed. I think that goes back to the earlier point that website testing takes out the guess-work of website design. It not only develops better pages for users because they get a better web experience, but it's also good for site owners because they get higher ROI.

At Google we fashion ourselves as being pretty well informed web designers, yet we are constantly surprised at what customers tell us they prefer. We use Website Optimizer on a lot of our own Google properties, such as Gmail, AdWords, Website Optimizer, Google Earth, Google Maps, etc.

We are constantly testing Website Optimizer's homepages to try to get more people to try it out and help them launch their first experiment.

Eric Enge: Basically what Extra Space gave you was the site that was designed based on the best thoughts of groups of developers and marketing people, but there wasn't a specific science to it. I think it's important for people in the Web Marketing world to really understand that the chances that they will get it perfect that way are basically nonexistent.

Sandra Cheng: Absolutely, I agree. You are paying money to drive traffic to your webpage, and not everybody that visits the page is going to do what you want them to do. So ultimately, it comes down to conversions and money. Every visitor is precious, so why not convert as many of them as possible and let them tell you what is and what's not working for them.

Eric Enge: Can you tell us about the Colonial Candle example?

Sandra Cheng: Colonial Candle is a 100-year-old candle company based in Minnesota. They sell a large variety of candles and seasonal items, and they tested their product page by changing font size, headers and product photos. Their original site didn't have a clear call-to-action and they had inconsistent imagery.

They had some product shots, some were stylized, some had colored background, and some didn't. They ended up increasing the font size and changing the image size, the headers and the colors, and they had two drastically different pages to test. One was shorter and had more stylized photos, and the other was a longer page that had blue headings which helped organize the page.

Everyone thought that the shorter design would win because conventional wisdom is that less is more. It turns out that the longer page won. They saw a 20% lift from that page, which is more than $20,000 a month in revenue. So, now they are a big fan of Website Optimizer and they run Website Optimizer tests on a regular basis.

Eric Enge: So if you want to have a quarter of a million extra dollars a year, use Website Optimizer.

Sandra Cheng: That's what I am saying. If you are not testing, you are leaving money on the table.

Eric Enge: Absolutely. The idea that less is more is certainly conventional wisdom. People think that things that are over-cluttered are going to not perform very well, and simple, even sparse, designs are best. But that didn't prove to be the case here.

Sandra Cheng: Right. You just never know. In some cases less is in fact more. Website testing really is just like having insurance against making the wrong decision.

Eric Enge: Sure, what you are trying to do is figure out what the best practices are for your audiences.

Sandra Cheng: Exactly.

Eric Enge: So, when you have websites with more than one page, you should basically cycle through pages in some sequence?

Sandra Cheng: Yes. We find it really helpful to start with your primary landing page, because that's where people are first. You want your goal of that test to get them to move towards your conversion. Maybe it's to learn more about your product, or to signup for a newsletter or for an account.

It could be to dive deeper into product categories and spend more time on the sites. You can also run tests to test how long people are staying on the site, and optimize for that. But, what we generally recommend is that you start with your landing page and make some pretty big changes there, and then start to refine your tests.

Maybe you start with A/B Testing to test a couple different variations of the page, and then move on to multivariate testing, where you are testing different sections of the page. You might try different headings, different images different content, different text or different copy and then move down the conversion funnel until you test and optimize the entire process.

Eric Enge: One thing that you said that was important there was to not be afraid to be radical.

Sandra Cheng: Absolutely. Don't be afraid. I think if you are going to test, you want to test something bigger than a semicolon versus a period; you want to be able to stand 6 feet away from the monitor and still be able to see the difference. Once you get the general layout correct, you can start tweaking the insides.

Eric Enge: Right, so you get the big gains from the big change, and once you are there you can do some follow-up things. In the case of Colonial Candle, they got 20% and maybe now they can refine it and pick up another 5%.

Sandra Cheng: Right. Sometimes small changes can result in big increases in conversion as well. Something as simple as changing a button color could make a difference. We have one customer, NerdyShirts.com, that tested a white button versus an orange button, and they saw a 7% increase in sales just due to that change in button color. You just never know. Small changes can make a big impact, so it's worth testing any idea.

Eric Enge: Let's talk about the mechanics of implementation. Can you outline what's involved from a technical perspective?

Sandra Cheng: Okay, great. Website Optimizer offers two different types of testing. The first is A/B Testing, where you are testing different variations of the single page. The way that works is at the top of your page you paste in some Javascript that we call the control script, and that piece of Javascript will take care of cookie-ing the user and choosing whether or not to redirect the user to a separate page.

Then, you put a tracking script at the bottom of each of your variation pages, and a conversion tracking script at the bottom of your conversion pages.

Eric Enge: So basically, the control JavaScript goes at the top of the current pages?

Sandra Cheng: Right, the control script is at the top of the current page and the tracking script goes at the bottom of each of the different variations.

Then at the bottom of your conversion / goal page, you put a third piece of Javascript that tracks the fact that the conversion occurred.

Eric Enge: Of course, you also need to brainstorm the different versions of things you want to test.

Sandra Cheng: Exactly. We give you all the Javascript. All you have to do is copy and paste them in the right spot and upload the pages.

Eric Enge: So there is no need for the Website Optimizer to actually get intertwined with the nature of the changes.

Sandra Cheng: That's right. Everything is pretty much controlled by the control script at the top of the original page. If a browser doesn't have Javascript turned on, it just behaves as if everything was normal, the customer will just continue seeing the original page.

Eric Enge: And that doesn't bias the test because they don't have Javascript turned on.

Sandra Cheng: Exactly, it just doesn't count. If they transact or not, it doesn't matter.

Eric Enge: Putting aside the need to design different scenarios, it sounds like this is something that could be done in a day.

Sandra Cheng: Oh sure, it can be done in 5 minutes if you have access to the pages. The other kind of testing is called Multivariate Testing, and it basically allows you to change different sections of your page at the same time. You could say, ‘I would like to test four different copies for my header at the top.' For your front and center image, you could decide to test two different images. For the text on the right-hand side, you may have three different examples to test. So instead of creating a brand new page for each of these sections, you would put a control script at the top of the page and then wrap each of the sections that you want to test with the section script.

And then, within Website Optimizer, you define what the different variations are for each section.

Eric Enge: Right. So that sounds all very simple. It's slightly more complex than a simple A/B test, but that's just because you are allowing Website Optimizer to decide how to mix and match the elements. Do you literally test every single possible combination?

Sandra Cheng: We do. We do what is called a full-factorial test.

Eric Enge: Say I own a new company and I am interested in using Website Optimizer. How do I start?

Sandra Cheng: I think the most important thing is to have a very clear conversion goal. What number are you trying to drive?

It's important that you have a fairly high rate of conversions, and by fairly high I mean about five conversions per day. Fewer could work, it just might take longer to finish the test.

Eric Enge: Right, you have to wait longer for the data to be statistically significant.

Sandra Cheng: Exactly. And then test a small number of variations. Our rule of thumb is to test no more than one variation per approximately 100 conversions. This is really just to make sure that your test doesn't take forever to run.

Test a small number of big changes. If you can't visually see the difference between two combinations in 8 seconds, visitors probably won't either. Then all you have to do is set up the test and wait; don't jump to conclusions. Wait for about two weeks and focus on the absolute difference in conversions. Once you have a winner, move on to testing other pieces of the site or refining the test in general, maybe getting more granular in the changes that you are making.

Eric Enge: Do you have any advice or tips on how to decide what kinds of things to test for? What are typical things that people play with?

Sandra Cheng: A couple of things we have talked about, like page layouts for example, are great candidates for testing. Things like font size and color are important too, especially for your first test. Then, what images should you show, how big should they be, where you should place them, etcetera.

You can test your call to actions, button sizes and messages, where it is placed, etc. You can test different types of images like testing traditional product images versus action images, for example.

Extra Storage Space did a good job of this. They tested glamour shots of their storage space, using pictures of a space with people and furniture instead of empty storage containers, and they found that the glamour shots worked a lot better. You should test and see which works better for you. So that's just the start, but really you can test anything.

Eric Enge: Does it make sense to try to approach these things with a philosophy for example, one philosophy would be the less is more philosophy right, another might be hit them between the eyes, so does it make sense to think this way when trying to think about different scenarios?

Sandra Cheng: That's certainly one way to go about it. I think there are an endless number of combinations that you could come up with, and ultimately I would say testing is not a one-time, hit-or-miss experience.

It's really a mentality, it is a way of life for website design. Pick a couple that you think might work and test those. If you are not incredibly happy with the results take the winner and test it against some others, and keep iterating.

Eric Enge: Particularly for paid search marketers, where the margin tends to drive towards zero as everybody squeezes everything they can out of it, it is a great thing to accomplish a 20% lift. You have got a pretty substantial competitive advantage in that situation.

Sandra Cheng: Right. And, the web is constantly evolving, so what works for you last year may not be the best option for you this year. Especially in 2009, with the economy being the way it is, I think it's really going to be the year where testing goes mainstream. People are looking to get more for their advertising dollars and landing page optimizations is one of the few ways you can do that without spending more.

Eric Enge: So the lesson there is that market forces can change the behavior and mindset of your customer?

Sandra Cheng: Exactly.

Eric Enge: So, one of the things people sometimes talk about is the personas of visitors. Some people are very disciplined or competitive, some are logical, some are spontaneous, others are emotional. People come to your site with many different mindsets.

Is that something you recommend people think about when they are considering how to try these scenarios?

Sandra Cheng: Sure, I think it's great to try to design your website in such a way that appeals to your target customers. If you pick designs that you think will speak to each of those different types of customers, you have a really quick way of learning who your primary audience is and what is the most effective tactic to use. You could also argue the flip argument though, which is somebody who normally isn't very spontaneous could come to your site and feel compelled to take action right then and there.

Eric Enge: Right. People who talk about this theory point out that everybody falls in all four of those groups at one time or another, depending on a lot of circumstances.

Sandra Cheng: I think the key is just to figure out what's going to work the best for your visitors.

Eric Enge: Is there anything else that you want to say?

Sandra Cheng: I think that Google Analytics and Website Optimizer are both about helping people succeed online and improving their user experience. As I mentioned earlier, we expected 2009 to be the year where website testing goes mainstream. So in 2008, we saw most of our adoption from SEMs, web development firms, e-commerce companies and other online marketing experts.

This year we are already seeing adoption across the board, from political campaigns, bed-and-breakfasts, to stationary companies. So, testing is going mainstream, and if you don't test, you are going to get left behind.

Eric Enge: Thanks Sandra!

Sandra Cheng: Thank You Eric!

Have comments or want to discuss? You can comment on the Sandra Cheng interview here.

Other Recent Interviews

Google's Cedric Dupont Interviewed by Eric Enge

Thu, 5 Feb 2009 14:38:19 -0500

Published: February 4, 2009

Cedric Dupont is the Product Manager for SearchWiki at Google, Inc. He has been with Google for almost 2 years (since June 2007). Prior to Google he worked as a consultant at Bain & Company, where he worked on strategy and due diligence for Private Equity deals for funds in the US and Europe. Prior to Bain he worked as a Senior Research Engineer at Volkswagen in a research capacity.

Interview Transcript

Eric Enge: Can you provide an overview of what SearchWiki is?

Cedric Dupont: SearchWiki is a feature we announced a few weeks back. It is a feature that gives users more control over their search results. So, you can re-rank results or you can add a result that you feel is missing. You can delete results that you don't want and you also can add notes to specific results for yourself.

The next time that you do a search, we'll respect that new modification, so we'd show your modification again. So, the changes that you make only affect your own rankings, but you can also see how other people have ranked or modified their searches by going to a special page, what we call the All-Notes page.

You can go to the All-Notes page, do a search if you think it's popular and you'll see a bunch of websites that have been promoted and deleted for this search, and a bunch of comments that have been added as well. What we show in terms of rankings is an aggregate number of deletions and promotions from our other searches.

For comments, we associate the comments with a nickname that is pulled from your Google account. The reason we are doing this is because the web is becoming more and more participatory in nature. So, people interact, they are no longer just passively pulling from each corner of the Web. People are interactive, they post stuff, they comment on things, they want to change what they see. Search is adapting to this phenomenon, and therefore becoming more interactive and more participatory.

Eric Enge: What were your goals were for the whole program?

Cedric Dupont: Our goal, as it always is, is to make our users happy. So, this is taking customization to the next level. Say there are results that you don't want to see for certain queries, you can delete them. If there are results that you think are missing, you can add them, and we'll respect that every time you are searching in the future. So, we want to give you exactly the results that you would like to see for a query. Some people are probably already happy with what Google does for them normally, and they'll be even happier in the future with all the search improvements that we are making everyday. But, some people just really want their search results the way they want them, and for these people SearchWiki brings them that level of control.

The other thing that we've seen people use SearchWiki for is repeated searches. I am sure you can think of things that you keep searching for over and over again and you still don't remember the URL. SearchWiki will remember you're searches and make sure that you'll find whatever you're looking for much quicker and easier, by putting that site on top. So, that's the behavior that we are seeing, and it's the behavior that we have also seen in the experiments that preceded this one, so there are basically no surprises here..

Eric Enge: Right. So, do you anticipate allowing a more refined type system where the user can pick the first result, the second result and the third result, and re-order at that level?

Cedric Dupont: Yes. There are ways that you can order it in a very granular fashion. And, it just depends on what order it is in. Every time you promote a result, it gets back to the bottom of all of those results you have promoted so far. If you promote it again, then it will go to the top, so there is a way of playing around with the order so you can basically do the exact image that you want. But really what we've seen in user tests is that people just say, no, this thing is part of the bucket of things that should be on top, and this thing is something that I want to see. It's very weird as people want to take result #8 and say no, this should really be result #5. That's not something that we have seen people demand, at least not in user studies.

Eric Enge: Right. So, you mentioned that one common use is repeated searches. Are there other common uses that you can comment on?

Cedric Dupont: Yes there are. We also see that users use SearchWiki as a means of expression. So, one experiment that we launched was during the election, there were websites that were created just for the proposition 8 campaign. When you would search for proposition 8 and go through SearchWiki notes, you could see a similar amount of opinions that go one way or another. You would also see it just as a means of expressing an opinion, which is interesting in this case of course.

People start using Google as a bookmarking tool, and this makes it just much, much faster to find what you want. We also found people that are really out there to cleanup the web. They want to clean spam out of their search results, and they really go in and have a large volume of deletions of websites that they consider spammy. So, these are the major cases that we observed.

Eric Enge: Right. So, obviously this is a major step towards personalization of search. Do you see interaction of this program with other personalization initiatives?

Cedric Dupont: Personalized search is a little different. We are trying to adjust search to the profile of the searcher. Here, it's much more direct, so it's a much simpler feedback tool. I mean, you are telling us this is what I want to see on top and this is what I don't want to see, and we give that back to you every time you search. So, at this point it's completely independent of personalized search.

Everyone starts with a different page, and you start doing SearchWiki actions on those from a different page.

Eric Enge: Is there a plan to offer people who want a way to opt out?

Cedric Dupont: SearchWiki is a feature of search and we typically don't let people opt out of features. You can't just say, well, I don't want any news results, or I don't want any book results appearing on my page. I wouldn't talk about opt out in any case, but what I can imagine is that people who don't want to see SearchWiki for one reason or another could choose a setting that turns that off for their particular account. But, at this point we don't see an opt out feature being available, and I think the numbers show that it is not something that is widely requested or needed.

Eric Enge: Right. So, a while back at the Web Conference in Europe Marissa Mayer commented a bit on the potential for SearchWiki to be used as a ranking signal for other peoples' searches. Can you talk about that a little bit?

Cedric Dupont: We are constantly looking at every signal that we can imagine to improve our search. SearchWiki is a new feature and it will provide a new signal. We are carefully looking at it to see if it's usable. we just launched a few weeks ago, and at this point there is really nothing to announce. That doesn't mean we are closing the door on using SearchWiki as an additional signal though.

At this point, we are mostly focusing on making the SearchWiki users happy, and increasing the usefulness of that feature more than anything else.

Eric Enge: But in the aggregate you could potentially collect some interesting data, right?

Cedric Dupont: A simple example is if we suddenly get a very large fraction of dissatisfied users with a site that is shown across the board, we could manually review that site. These are some very simple things that we can imagine doing early on, but again, at this point we'd like to improve the usefulness of the features before anything else.

Eric Enge: Right. Now, that certainly makes sense. I mean I could see that it certainly has the potential for spamming tactics that people could try to implement. It's part of the landscape of what you would have to deal with in using it as a ranking signal.

Cedric Dupont: And, that's not unlike what happens in the web in general. So yes, we are certainly very much aware that we have to create a robust spamming control.

Eric Enge: Sure. But, you guys are used to that.

Cedric Dupont: Yes, exactly.

Eric Enge: Even without the issue of spamming, you have to consider the general noisiness of the signal, and how good an indicator it really is of what should be the best search order.

Cedric Dupont: Right. It's important to understand that the first order improvement is what you created for yourself. And then, the second order is whether the signal is powerful for one thing or another. So, really how people are using SearchWiki and how is it improving their results, that is what's really important to us right now.

Eric Enge: Are you going to be doing some things in terms of evaluating the reputation or participants who provide comments?

Cedric Dupont: We are always looking at things like comment quality, quality of ratings and things like that. Evaluating the trustworthiness of each of these inputs will have to be part of any signal that we would use for ranking. So yes, we'll have to do it right if we want to use the signal more broadly, and not every rating or comment will be considered equal if we do our job right.

Eric Enge: Right. There should be some method that allows you to basically track user behavior, and the kinds of things that they are doing and seeing. Maybe evaluating whether or not they are a set of users that are considered trustworthy enough to be part of that bigger evaluation.

Cedric Dupont: Yes, and these are very simple things. Behaviorally you can imagine the type of spamming that you would get. For instance, comments that are posted many, many times across many, many sites would be a warning signal. Maybe a comment that has been posted many, many times to many, many sites is less valuable than another less-posted comment. Again, it's still very early and we have only been released for a few weeks, but these are the types of things we are observing carefully and we are very optimistic on how well we can use them to limit abuse.

Eric Enge: Right. It's good that you're giving users what they want by itself right out of the box. Given the problems with ranking signals, the most famous being links of course, being able to diversify sources of information is a good thing. I mean, you could see an interaction that somebody seems to have all kinds of great things happening in the world of links. Everybody encounters their site in the results, but a large percentage seem to be removing it. So, that becomes suggestive of bad behavior.

Cedric Dupont: Yeah, it's an interesting idea.

Eric Enge: And, of course you can do it vice-versa, right? If large numbers of users are promoting their site, but the links aren't following. I mean, those are two very simple examples, but it's really good to have multiple signals to offset each other.

Cedric Dupont: Right. So, we are really looking forward to the types of things that we'll see. And again, it's a very recent launch, and we have few things that we want to deploy and that we think will make it more useful for users. We are very excited, and we are looking forward to this being a great thing for searchers.

Eric Enge: What are you doing to deal with inappropriate comments today?

Cedric Dupont: Well, today when you come across inappropriate comments you can flag it as inappropriate. And, what that does is it begins the review process. If the comments need to be taken down from public view, they will be. Of course we also have automated tools that review content.

Eric Enge: So, the automated tools look for things like bad language?

Cedric Dupont: I can't share specifics, but I can imagine that that's one of the things that you could look at.

Eric Enge: Are you considering any other kind of comment verification?

Cedric Dupont: No at this point. Any filter would have to be very, very scalable, because we have a lot of users and a lot of searchers. So, at this point we haven't considered that, but we do have automated systems that try to figure out whether a comment is appropriate or not. Eric Enge: Have you thought about implementing something where you could keep private notes which aren't available to the general public?

Cedric Dupont: At this point, no. Comments are publicly visible, but they are associated to your nickname, which obviously is anonymous in a sense, because you can choose whatever you want as your nickname. And, it doesn't point to a specific URL or something that can tell me that this John Dale is the same as that John Dale.

Eric Enge: Sure.

Cedric Dupont: Your comments are as anonymous as you want to make them, depending on how anonymous your nickname is. And so, we feel like that strikes the right balance.

Eric Enge: It would also be interesting to potentially have a way to get all your comments through a permalink or something like that.

Cedric Dupont: That's an often requested feature, so what people would like to do is to imagine that somebody asked you about what's the best LCD TV to buy? And, all you'd have to send is a permalink of your search LCD TV, and you have promoted and demoted the sites that you think are relevant or not. You may have added a few notes here and there, and you just sent that as a link. And so, now the person receiving it has the full context, including your notes. That's a very, very frequently requested feature, and we think it would be valuable, yes.

Eric Enge: This would allow people to share their conclusions, and you can see that playing into question and answer sites where people ask these questions.

Cedric Dupont: Yes that request often comes up in forms. People want to be able to just send a link instead of having to write and then save a bunch of URLs. It would be something valuable.

Eric Enge: Right. And, what can you say about how the order of the comments is determined?

Cedric Dupont: We try to rank the better comments first, but we also try to show some variety and try not to show duplicate comments.

Eric Enge: Or, perhaps extremely similar comments.

Cedric Dupont: Right. To be completely honest, there is a lot of work to be done here. These are things that are pouring in right now, and we're improving the ranking system as much and as fast as we can. But, this is definitely an area where you will see changes, because if you play with it, you will see that it can be improved pretty dramatically. So again, whatever I would say today would probably be wrong tomorrow. All these things we're talking about are really only relevant for extremely top level sites where there are lot a comments.

Eric Enge: Do you take into account user votes on the comments?

Cedric Dupont: Things like that, yes. There are many things that come into this ranking function, and it will improve quickly because it is so young.

Eric Enge: It's a pretty novel thing, because a lot of places on the web display comments in the order they are entered, so you don't attempt to rank them.

Cedric Dupont: Yes, and there is a benefit to that but there is a drawback too, which is if somebody is really motivated you can make sure that his comment will always be there, because he updates them very quickly. On the positive side, it guarantees some freshness and some discovery, so chronological order is not a bad thing to do as a start. We find it useful to rank according to relevance, but I don't think chronological is bad at all.

Eric Enge: They are two different things with different goals.

Cedric Dupont: Yes. Exactly.

Eric Enge: When you have a traditional-threaded discussion there is a reason for the chronology because of the way things feed off of each other. Here you are trying to surface to the top the comments that add the most value to the search results.

Cedric Dupont: Right. You mentioned something that's very important I think.

You write notes to yourself, and you don't see the context of the other notes at that point unless you go to the all-notes page and then do it there. But, there is no guarantee that your comment would be following this other comment by this other person that you are responding to. That's an important thing to note. For our purposes, the conversation aspect of things is less useful and less clear. It's not like you are on a certain web property where you have some assurance that other people will go with the same interest. So, we don't really worry about supporting a conversation here.

Eric Enge: The goal is different.

Cedric Dupont: Yes

Eric Enge: Have you thought about broadening SearchWiki so that users can do more than just remove a site from a query, but say I don't want to ever see this site.

Cedric Dupont: Yes, that's another good suggestion. It is a tricky problem, because you may find one site completely relevant for one search and completely irrelevant for another search. It's difficult to just delete everything across the board. In addition, the site itself could improve over time.

Eric Enge: Right. But, I think the biggest thing that you hit on already is that the user may not know what they are doing. If they don't think about all the scenarios where they might be searching, they won't realize that a site may be a perfect query for them in another case.

Cedric Dupont: Right. So, the choice that we made is that deletions are query-specific. So, when you delete a site for certain query, we assume you really know what you are doing at this point. You did a search and deleting a particular site made complete sense. When you promote a site, we do something a little bit more complicated. These sites may be promoted for similar searches, because there is no harm in promoting them again for queries that are very much related to your first query.

So, the short answer is we do this sort of thing for promotions but we don't do it for deletions at this point. The risk of what could happen when it doesn't work is much larger from deletions than it is from promotion.

Eric Enge: Thanks Cedric!

Cedric Dupont: Thank You Eric!

Have comments or want to discuss? You can comment on the Cedric Dupont interview here.

Other Recent Interviews

Market Motive's Scott Milrad Interviewed by Eric Enge

Tue, 27 Jan 2009 09:30:05 -0500

Published: January 26, 2009

Scott Milrad is the Director of Online Education at Market Motive, developing Internet marketing courses and curriculum for Market Motive's online education programs. Originally on track to go the traditional collegiate teaching route, Scott completed his Masters Degree in literature and creative writing at San Diego State University before diverting his scholastic attentions and finding himself as a multimedia developer at the heart of the dot com boom. Always vowing to get back into academia, Scott landed at Market Motive with the promise of combining his passion for teaching with the unlimited possibilities of the Internet.

Interview Transcript

Eric Enge: Can you provide us an overview of your role at Market Motive?

Scott Milrad: My title is the Director of Online Education, so I put together all the course curricula and I work with the faculty on new content. This includes planning out new videos, filling in the gaps where we have problems in our curriculum, putting together the learning paths for extended courses and certifications and then running the Master Certification Program.

Eric Enge: So, you are the in-charge of the curriculum?

Scott Milrad: Yes.

Eric Enge: Can you give us an overview of the Market Motive Master Program then?

Scott Milrad: Currently we have four Master Certification tracks - SEO, Web Analytics, Pay Per Click and Online PR. What we have is a program that takes a group of people through one or two of those over a three month period that includes online video classes quizzes, live workshops with the faculty and a final peer review.

So, in our first month, each member goes through those series of videos, take the quizzes and at the end there will be one big comprehensive test of about fifty to one hundred questions, depending on the discipline. They need to pass that test to move on to the next level. After that, there will be two or three longer essay questions relating to the program or the discipline, which they have to answer. After that, members complete an hands-on project that represents a culmination of their learning. All work is reviewed by peers and the dream team faculty.

An SEO, for example, will build a website from scratch with SEO in mind. Starting from a blank template, they have to build title tags, Meta description tags, page copy, do some page sculpting, build the internal linking program and the site architecture.

Then they have to write an essay about why they did certain things, their strategies and how to go about doing various things within the SEO program. After this, they go through a peer review. We exchange projects and one person will evaluate another one based on their criteria for building the project. Finally, we finish with the all-faculty panel review. This is where the faculty looks at everybody's projects. And then, we'll ask a series of questions on something that someone has done and ask them a little bit about that. Along the way, each of those projects has to have a basic level of high quality to move onto the next level. It then culminates with the faculty panel, where our students must earn three faculty endorsements to graduate.

Eric Enge: It sounds like the SEO is focused on onsite SEO type practices, is that right?

Scott Milrad: For the hands-on project, it is practical to focus on onsite SEO,

Eric Enge: Link building isn't currently a part of the curriculum then?

Scott Milrad: Link building is definitely a part of the curriculum as is keyword strategy and more. We have several training classes on off-site strategies and Todd Malicoat gives some fantastic video classes on link building strategies. We initially were going to include that as part of our other project, but we would have had to extend the program longer, so we decided to focus the hands-on project on straight onpage SEO.

Another example of a time-sensitive hands-on project is out Online PR people having to contact a series of bloggers to persuade them to blog about our program and the Master Certification Final. Interestingly, online PR is a short form of link building . It also crosses over into SEO; the cool thing about a lot of this stuff is that there is definitely cross over.

For example, Landing Page Optimization and Pay Per Click are integrally tied together in the same way that Online PR and Social Media are. You can make an argument that there is crossover in most of the disciplines.

Eric Enge: Right. And, which of the programs have you found most interest in so far? PR, Pay Per Click, SEO or Analytics?

Scott Milrad: SEO is definitely the most popular and I think Online PR is a close second. SEO is so popular because that's where people feel they should start out. So then, they add Online PR, or Analytics, or PPC. Generally, you don't have people doing SEO and PPC together, but I'd say many of the candidates are recognizing the need for cross training.

Eric Enge: Who is signing up to participate in these programs?

Scott Milrad: That's another really good question. So, we've got people that are running their own business. We have senior directors at significant public companies. We've got people that work for agencies. We have people that already work for the specific discipline that we are teaching. We have people that are in the analytics track that work for analytics companies and want to be taught by the best and truly master analytics.

For this next round, we have a lot of agencies who want to get their interns through the program and up to speed so that they can actually put them on the contracts and have them do some real work.

We also see some young folks right out of college who want to get into Online PR and Social Media.

Eric Enge: Right. So, do you offer a lower-level certification or is it just the master certification?

Scott Milrad: We do have a lower-level certification; we have a practitioner certificate program that we send people through. It's actually a part of our monthly subscription model. So, people who are existing members at the all-access level can participate in the practitioner certificate, and it's similar to the learning path of the Master Certification Program.

It has approximately the same amount of videos, weekly workshops and quizzes, but a less demanding test at the end and no hands-on projects, peer review, or publicity. If they pass that, they get a practitioner certificate to show that they've gone through a program and learnt the basic of whatever discipline they've gone through.

Eric Enge: So do a lot of people start with the more basic stuff and then move on to the next level after that?

Scott Milrad: Several of our candidates were already in a program. Because they've already seen some of the basic classes and they may have even gone through the practitioner track, they were ready for the next level, which is what master certification is all about, going to that next level.

I would say people that are true beginners can still do well if they are really dedicated, but I think you kind of need a little bit of a base if you're going to just jump into a higher level.

Eric Enge: What about industry endorsement of these programs, has there been any of that?

Scott Milrad: I think that the industry has been keeping a really close eye on us, and, the program. We have some pretty interesting people that have already registered to be part of that, both observers and A-list guests that are going to come and participate, But yes, the industry has received us pretty well because it's not just some slides and a test. It includes hands-on real life projects and a peer review of the top names in the industry.

Eric Enge: I would think your goal would be to get some rock stars to come out of the program.

Scott Milrad: Absolutely. I mean, now you are hitting on what this is all about. We don't take it lightly, we don't put people through the program and just go, "oh, we took your money and we'll give you a certificate." We are putting our own name behind this and we are expecting that when people come out of this they have the ability to go and run an agency or on their own and put the things that we taught them in the program into practice effectively. So yes, we are definitely looking for some superstars to come out of it.

Interestingly enough, the interactive nature has resulted in a really great group of people and they've got to know each other well. We do a weekly office-hours conference call where we get on and talk about what's going on depending where we are in the program. We discuss the classes and projects in Online PR, or PPC or Analytics; we just get into discussion about whatever anybody is having problems with.

What we found is that people are actually really starting to help each other, so we have a few people that have already excelled at the program and we got them to help us out with it a little bit. Then it becomes a symbiotic relationship and they help each other. So yes, we are really excited and hopeful about people coming out of here and being true superstars.

Eric Enge: So in October you started this latest round of Master Certification, right?

Scott Milrad: I think the program actually started at the very beginning of November.

Eric Enge: Can talk a bit about the specifics of what people have been through during the past months and how it's going to culminate in your interactive event?

Scott Milrad: Mastering internet marketing takes a significant commitment, and a three month program is just about right for folks with a fulltime job, and have a fulltime family too. . If we were all in college then this would just be a nice little semester or quarter course. Being out in the real world is a bit of a time commitment, so not everyone has what it takes to finish. Those that stick it out are already benefiting from the results.

Essentially we spend the first month reviewing the online classes and attending live online workshops. There are approximately ten to fifteen hours worth of videos that candidates must attend, depending on which discipline it is.

The second month is dedicated to articulating what you have learned via essays and planning for the hands-on project, taking everything you have learned and putting it into practice. And then, the final part is going through the peer reviews. People are really turning around and are now able to access another person's project from the outside.

That kind of goes to the idea of not just being in-house, but being able to look at a company's existing program and evaluate it from the outside.

On the other hand I think if you actually reduce this program by 50% and made it a six-week program, people could still get through it. They would have to commit a lot more, but there is always a balance in trying to figure out the right amount of time to give people so that they have to get on it early enough, yet at the same time we have to make sure everyone has enough time to get things done.

Eric Enge: For the SEO project people get a starter website from you, and then they have to flush it out. Are they supposed to do a certain number of pages or they are just working on that particular website?

Scott Milrad: There isn't a specified page amount. I give them a front page and it's really just a shell, with a couple of images, a few hundred words of page copy and a secondary page so that they have two pages from which to build. We then ask them to modify that to build a site with five to ten pages so we are able to see what architecture they put together and to see how the internal linking.

Things like title tags, Meta tags and page sculpting can be done in one page or two pages. But, the rest of it, inner linking, overall site architecture, is going to obviously require more pages. But, they can put a dummy copy for the final five pages if they want as long as we see the architecture they are putting together.

Eric Enge: Now that their projects are nearing completion, you are planning an event. Could you tell me a little bit about that?

Scott Milrad: We are calling it the Faculty Panel Gauntlet. It's sort of "American Idol meets SEO certification". Right now eight candidates have made it to this level and will participate in the event. There will be at least eight faculty members, plus a few associate instructors as well. As I said, a couple of surprise guests will be on there as well. What we are going to do is a lightning round type of event where there is going to be a series of questions from each faculty member to each candidate.

Michael Stebbins, the CEO of Market Motive, will host the event, and he will decide in what order the faculty members will ask their questions. After each question to one of the candidates, the candidate has a minute or two to respond. We will just keep going that way for about an hour.

At the end of it candidates have to get three faculty endorsements to go on. So, at the very end Michael will be asking for everybody's opinion and get the thumbs up and thumbs down. All we'll do is ask if people are ready to move forward or not.

Eric Enge: This is going to be broadcast?

Scott Milrad: It is going to be broadcast; we are doing a live webcast of everything, a screen share and over the phone where everybody can call in and listen in on the whole thing. It's interesting, because all the faculty members are on it, and they are all over the country, and our candidates are all over the country as well. So, we are going to have a lot of people that are calling in from all over the place participating in one big event.

Eric Enge: What's the date of the event?

Scott Milrad: That is going to be January 27th at 9 am Pacific Time.

Eric Enge: Why did you decide to make it interactive?

Scott Milrad: Because this is how we perform in the real world. As consultants and experts we perform under the scrutiny of a client. Perform under the scrutiny of the know authority in the market, and you have a good chance of serving your clients well. Also, we aren't hiding the fact that this is fun and it generates some publicity for Market Motive.

Eric Enge: You are practicing good SEO promotion.

Scott Milrad: We are practicing good Online PR promotion. It's not a surprise that part of Online PR's final project is to promote this event. It's very circular in that way, and they need to release a press release and try to get the most people to their landing page about the event. By the way, Mario Bonilla at PR-Web was most generous to our candidates.

In the end, it's just an effective way to bring what we are doing to the surface and show the public what we are doing. And again, it's supposed to be fun; we have no intention of trying to embarrass anybody on this call, and we tell people that it is not required that they be on it. If you are uncomfortable with speaking in public or you just don't want to be grilled, they don't have to.

It's also a great way for them to get public exposure, and we are going to publicize the people who finish the program on our site. Part of the idea is to promote the graduates, have fun and to get the word out there about what we are doing.

Eric Enge: Right. It is a lot like site clinics at the major trade shows.

Scott Milrad: Yes.

Eric Enge: Outside people who want to look in on this can think of it in the same way they would think of a site clinic, as an opportunity to see people who really know their trade grow and dig into the issues with a given site implementation. Scott Milrad: Yes, exactly. When you get the best and the brightest up on the panel debating an aspect of Internet marketing, it's pretty exciting. And, we haven't had the entire faculty on one call since that first conference call two years ago. So, it's a good opportunity to get everybody in one place, and we've got some personalities too so it should be interesting.

Bryan Eisenberg is certainly not shy, and Avinash Kaushik is an absolute pleasure to listen to at anytime. Lots of opinions and authoritative people... It should be a lot of fun.

Eric Enge: Thanks Scott!

Scott Milrad: Thank You Eric! I enjoyed it.

Have comments or want to discuss? You can comment on the Scott Milrad interview here.

Other Recent Interviews