Rich Mironov at Product Bytes weighs in with a thoughtful post about measuring product managers.  He had a recent experience in Sweden where the product executives he worked with shared the same concerns as product executives in the U.S., Europe and the rest of the World, namely: what is the right way to measure the performance of a product manager?

This is a really interesting and difficult question to answer.  Rich breaks it down into three areas: product-level metrics (product revenue, profit, customer sat), team-level metrics (are the other teams getting what they need from product management), and individual-level metrics.  The first two metrics are fairly well defined – there is a strong history of measuring products and teams with these criteria and there are MBA programs with entire curriculum defined about measuring product performance.  Unfortunately measuring individual product manager performance is more tricky as these metrics are not as clear.

Lacking traditional or clear metrics, most companies rely on product-level metrics such as product revenue, margin, or customer satisfaction to measure the performance of an individual product manager.  The thought process is defensible – you want the product manager to be motivated by metrics that lead the product toward success.  Unfortunately, product managers typically have very little control over the outcome of these metrics, or at best, indirect control.  For a product manager measured on product revenue, the product manager could spend months designing the perfect product, training the sales channel, and working with marketing to design a lead generation campaign – only to wake up one day and find that the executive team had made a strategic decision to make a large acquisition and change Sales resourcing.  That change would have a huge impact on the product manager’s metrics, at no fault of the product manager; effectively they would be penalized for doing the right things.  For a product manager measured on product margin, I always ask “Do you have the ability to outsource the development of your product to a 3rd party if your internal engineering team delivered an estimate that was too expensive?”  For most product managers, the answer is a resounding “no.”  Customer satisfaction suffers from a similar control issue – there are lots of reasons beyond the product that customer sat may suffer, and most of those reasons are not something a product manager can impact.

The other issue with measuring product managers on product-level metrics is an issue of timeliness.  If you are measuring your team on product revenue, you can make an argument that it may take up  to 18-24 months to know if the product manager is doing a good job.  Consider this – if you hire a new product manager, before he or she can impact product revenue, the following will need to occur:

• The new product manager will need to go into the market and research the market’s problems (up to 3 months)
• The product manager will then write what they have learned into a business plan, and get it approved (1-2 months)
• Then the product manager will write requirements for engineering (up to 3 months)
• Then engineering will build something (6-9 months)
• Then in most products there will be a Sales cycle (3-6 months)

Only then will we have the data to understand if the decisions made on the front-end of the process are “good.”  Hopefully we will be more agile and most faster, but the point remains – using product-level metrics to measure individual performance is insufficient.  We need a better way, and there is a better way.

What I have settled on for measuring individual performance is to measure the activities required for a product manager to be market-driven:

1. Setting a quota for market visits (10/quarter is a good start IMO),
2. Being able to defend an updated business plan in front of me and their peers every quarter, without using the phrases “I think,” or “In my opinion” (which requires higher order thought and research).
3. Keeping their finger on the pulse of the business by defining and measuring the right product level metrics and communicating them in the form of a dashboard report monthly.

I like these metrics because they require a product manager to get out from behind their desk and be outside-in focused, and the product manager can control them, as opposed to revenue, margin or CSAT which the product manager cannot control.

All things being equal, it would be preferable to measure outcomes as opposed to activities, but it is wicked hard to separate and quantify the inputs of a product manager vis-a-vi all the other variables that go into making a product fly.  Activities are a sufficient proxy to measure a market-driven product manager to supplement product-level metrics and understand if a product manager is doing their job – without having to wait two years to find out.

Most of the readers of this blog are probably familiar with ProductCamp, a networking event for product management and marketing talent that I founded in Austin and have helped seed around the world. One of those seeds took root in London, where ProductCamp London merged with another group to form ProductTank, a group for London product talent to meet, drink, and network. My friend Janna Bastow is one of the leaders of this group and is coming to Austin for SXSW – and is bringing some of her ProductTank’s charm with her!

ProductTank describes themselves as:

ProductTank provides an opportunity for Product Managers working for (or wanting to work for) web companies in London to exchange ideas and experiences about Product Management, Business Modelling, Metrics, User Experience and all the other things that get us excited.

Sounds fun! U.S. product people might not have the chance to experience this event, but if you are going to be in Austin, you’re in luck! ProductTank is hosting a happy hour at SXSW on March 11th! If you are in town and interested, you can go register (it’s free). I hope to be there, so come say hi.

Every year for the last decade, Pragmatic Marketing has run a survey of the product management and marketing community in order to assemble a profile of common industry practices, team sizes, responsibilities, and compensation.  This year, I administered the survey, and we got some very interesting data!  Over 1800 of your peers participated, and for the first time explored some new areas around soft skills and compensation.  I hope that you find the results as interesting as we did.

First, you can download the complete report in all of it’s PDF glory from our website.  Here are some nuggets to whet your appetite:

• 52% of respondents report directly to the CEO/COO or a Product Management VP.  Product Management is becoming its own department in the majority of companies.
• There are only 0.48 Product Owners per Product Manager.  This tells us that Product Managers are being asked to take on both roles – a recipe for potential disaster.
• Over 80% of respondents indicated that they spend at least half a day per week (or more) in meetings with Engineering (such as daily standups).  Whereas only 30% indicated they spend at least half a day per week visiting sites without Sales.  Where is the time to be in the market?
• On the softer skills of product management, over 45% rated their ability to challenge and negotiate with executives to be a weakness, an opportunity for improvement.
• The average compensation is $98,068 plus a $13,501 annual bonus.

If you participated in this year’s survey – thank you!  Also, a special thanks to Steve Johnson for his valued assistance and guidance in assembling this year’s results.

If you have additional questions, drop them in the comments below and I will attempt to answer them.

It’s the most wonderful time of the year!  No, not the holiday season, it’s time for the 5th (almost) annual “You Might Be a Product Manager If…” list!  Favorite entries from past years include:

• You might be a product manager if your wedding included a PowerPoint presentation.
• You might be a product manager if you stack rank your children.
• You might be a product manager if you ask your kids “what problem are you trying to solve?” when they ask for a new toy.

Last month, we ran a quick survey about how you are using social media in product marketing and management.  The goal was not to develop a comprehensive picture of this broad topic, but to recognize some trends and validate some areas that we have heard out in the field.  Over 130 of you responded to the quick, nine question survey, and the results are now in.  Keep in mind that this isn’t scientific!

How often are you using social media?  Not surprisingly, product professionals who answered this survey are prolific users.  Nearly 90% said that they use social media “Daily” or “Often” to keep current on issues.  Interestingly, non-work related usage scores lower on all counts.

How often do you use social media for the following...

Since we know that product professionals are making use of social media, the natural follow-up question is: which ones are they using?

Which social media do you personally consume?

This is a really interesting chart.  As usual, there are the Big 3: Twitter, Facebook, and LinkedIn.  But amongst product professionals, LinkedIn edges out even Facebook!  This correlates with the finding above that product professionals are using it more for work-related items; they probably aren’t going to friend their customers or competitors on Facebook.  Even at a fraction of the size and reach of Facebook, LinkedIn’s focused ability to reach into professionals and customers drives the highest usage.  Google+ also rated highly, considering its relative reach compared to the other networks.

Do you use social media for the following activities?

How do product professionals actually put social media to use?  Obviously, Thought Leadership activities came out on top: driving traffic to your blogs, websites, whitepapers, and landing pages is a huge priority.  Live event support was a little surprising as number two – product pros are tweeting to connect with customers and partners and keep up the engagement at tradeshows and events.  Competitive research also scored highly.

How do you rate the quality of information you get from social media?

Most product professionals still take what they get from social channels with some skepticism.  About 30% said that they rated the information they receive from these channels as “Mediocre” or “Poor.”

Does your company have a social media policy?

With social media becoming ubiquitious, you might think that all companies have a defined social media policy.  You’d be wrong!  25% of respondants indicated that they don’t have any kind of defined policy for how to use (or not use) social media.  About half have a policy in place that is liberal enough that they feel free to post whatever they want.

Who owns social media efforts at your company?

When it comes to having a clear owner of social media efforts in the company, Marketing has taken on this role.  In fact, Marketing outsourced all the other options combined.

Scenario: Someone is actively complaining about your product on social media. How do you react to this situation?

One of the biggest challenges of social media is interacting with a customer who is upset and complaining about your product or service online.  With Twitter, Facebook and other channels, these customers have a huge bullhorn to create brand damage if they aren’t handled appropriately.  On the other hand, companies that have mastered how to use these channels effectively are creating loyal customers who reinforce their brand and spread stories about how good the company was to them.

For this question, we posed a scenario: what would you do if someone was actively complianing about your product on social media?  Almost 40% said that in this case, they’d follow a set policy.  35% would respond directly, indicating that there is still a lot of personal ownership of social media efforts at most companies.  What would happen if your “social person” leaves tomorrow?

Scenario: Someone is actively praising your product on social media. How do you react to this situation?

Next, we flipped it around – how would you react if someone was actively praising your product?  More people ignore these, but do we risk appearing tone deaf if our customers tell us we’re doing a great job and we react with silence?

How do you measure the impact of social media?

Last, we were interested in how product professionals measure their social media efforts.  At Pragmatic Marketing, we are big believers in the mantra that everything needs to be measured, especially in Marketing.  Unfortunately, over half of respondents are not measuring their social efforts today.  15% are still trying to figure out what their measurements need to be, and 9% measure by number of generated leads.  Social media is reaching an inflection point: very soon it will cross the chasm from the shiny-object stage of “ohmygosh we’ve got to do this social media thing!” to “What’s my ROI” stage.  Social efforts that can’t demonstrate positive ROI will be the first things to be cut.

We hope that you found this information interesting and helpful.  If you’d like to help us construct a more complete profile of product professionals, please fill out our 2012 Annual Product Management and Marketing survey.

If you are interested in attending a Pragmatic Marketing seminar that I will be teaching, you will find me in Orlando from Dec 7-9 and Vancouver, BC from Dec 13-15 teaching Practical Product Management and Requirements that Work.  I hope to see you there!

Every year for the past decade, we at Pragmatic Marketing run a State-of-the-Union survey for product management and marketing.  The survey goes out to the 75,000 alumni who have been to one of our seminars and is also open to the general public.  The annual survey seeks to help us define how the role of product management and marketing is changing, with regard to titles, level of education and background, roles and responsibilities, compensation, and how you work with others.

The 2012 version of the survey is now open!  If you are a product professional, go take it and let us know what your work life looks like.  The survey typically runs from mid-November through mid-December and we crunch the numbers and do analysis and post the results on our website early in the new year.  If you think that your team might be under resourced, you can use the survey to compare against other companies.  If you think that you might not be compensated correctly, or want to get a leg up on your compensation discussions for next year, you can use the survey to see how you stack up against your peers in similar industries, education levels, and other vectors.

Thanks for participating, and watch this space for analysis of the survey in December and January!

If you are interested in attending a Pragmatic Marketing seminar that I will be teaching, you will find me in Orlando from Dec 7-9 and Vancouver, BC from Dec 13-15 teaching Practical Product Management and Requirements that Work.  I hope to see you there!

In an effort to learn more about how you use social media in product marketing and management, we’ve created a 9-question survey that we’d love for you to take. We’ll share the results of this survey next month on this blog and the others you see listed on the right hand sidebar.

Thanks in advance! Here is the link: Social Media Survey

If you are in Austin tonight and want to come out and get your networking on, check out ProductParty Austin, from the team that brings you ProductCamp Austin.  ProductParty is a networking event with drinks, followed tonight by a 1-hour session that I will be MCing called PowerPoint Karaoke, or PPTK.

For the uninitiated, PPTK is like Karaoke, with slides.  You get up in from of the room and have 2 minutes to present from a slide deck that you’ve never seen before.  The slides are totally random, pulled from various presentations from the Internet.  Your job is to create a story to go along with the slides.  The session is fast paced, fun, and a great ice breaker.  It is also good skill building for your presentation skills and ability to think on your feet, in front of a friendly supportive audience.

I will be there tonight, as well as John Milburn of Pragmatic Marketing – we hope to see you there!

Register for ProductParty Austin.

Amazing design speaks to us on an unconscious level.  We appreciate it without knowing all of the pain and suffering that the designer went through to get to the final product.  We appreciate it because of its utility or simplicity and its ability to add value to our lives through beauty, or helping us solve a problem in our daily lives.  This is why Apple products resonate: they solve problems we all have (technology is confusing and hard to understand) and do it in a way that is beautiful.

This evening, as I sit in the Admiral’s Club lounge in LAX, I was browsing through the beautifully designed products on Dwell.  Some of these products are high art, and are priced as such.  Some are crafty reboots of timeless classics, with added details of beauty or utility.  One such product is the Tip Ton Chair, a take on a chair a school child would use.

The Tip Ton is a beautiful, useful product, but what caught my eye wasn’t the chair, it was the quote from the chair’s designer, Edward Barber:

“We knew what the problems were but weren’t yet aware of the solutions”

This quote perfectly captures the process of design.  Wonderful designs aren’t created, they are out there, waiting to be discovered.  Only in order to discover them, you first have to understand the problem that needs to be solved.

A student in my class last week in San Francisco sent me a great article from the Stanford Knowledge Base about how a products’ price impacts the value that a customer sees in the product.

We’ve long known that products that carry a higher price are generally perceived by the market to be higher quality.  Researchers from the Stanford Graduate School of Business and CalTech have taken this hypothesis to the next level.  They ran an experiment where they asked participants to consume wine at different price points from $5 to $45, while undergoing a fMRI (Functional Magnetic Resonance Imaging).  fMRI is used to detect blood flow in the brain and see how much the “pleasure centers” of the brain are engaged by different activities or products.

The hypothesis: that the higher priced wine would engage those pleasure centers of the brain more than the lower priced wine.  Which is exactly what happened…except unknown to the participants they were tasting the same wine each time, and being told that it was at different price points.  Wow!

Click here to view the embedded video.

Baba Shiv, one of the researchers, recommends that we pay less attention to the price and weigh the opinions of experts more heavily when evaluating wines.  If we just look at the price, we may unduly enjoy a wine that actually isn’t as good as the price it commands.  Even more fascinating – our expectations tied to price extend to medicine.  Researchers have shown that when we perceive a medicine to be more expensive, it can have a positive impact on how well it works.

Remember, when it comes to the value that someone sees for your product in the market, their perception is truly their reality.