In an organisational setting, learning to grow is all about improving performance and this can only be achieved by individuals learning to improve their performance. An organisation comprises the people who work together, so organisational learning is all about people learning. Over time, through this growth an organisation will develop processes, culture and expertise that can help it differentiate itself and help it deliver its mission. With a good corporate memory, it will know how and why it operates the way it does.

For organisational learning to be effective it has to start with the end in mind. It is therefore important to ask:  what performance outcome are we trying to achieve? Can we be clear about this? And, more importantly, can we help each individual understand what difference in performance we would like to see and how the learning will help them achieve that? As Robert O  Brinkerhoff says, we then need to hold the learner accountable for using the learning to deliver the performance growth we are looking for.

Suppose the growth you sought was to deliver your organisation’s projects more effectively because you have identified that they are consistently late and over budget. After examining the reasons, you conclude that better performance management skills and the ability to have difficult conversations (rather than technical project management training) is the learning that is required. The next steps would then be as follows:

1. Talk to the people who need to learn to make sure they understand the performance outcomes you seek, and explain how the learning will assist them. They should agree that the learning is what is required for them to deliver the improvement.
2. The learning intervention takes place.
3. Another conversation is required to check that the learners now feel equipped to grow and develop their performance and to ensure that any further support is provided.
4. The learners are held accountable for using the learning to develop and grow.
5. Review of outcomes.

Sometimes, like the elephant seal, learning assumes all growth is good and, in the process, damages the profession. We need to start with the end in mind and work in partnership with line managers to deliver real personal, organisational and, eventually, economic growth.

(Blog: originally published through Civil Service Learning) 

Last week was a big week in the calendar for the CIPD. As well as hosting the HRD conference and exhibition inLondon, it also released its annual survey report on Learning and Talent Development. The fact that the report is released at HRD meant that I couldn’t help but reference the results as I walked around the event.

The headline for HRD 2012 was ‘adding value to your business through L&D’. It’s great to see the recognition of learning and development as a contributor to organisational performance. This is a view very dear to my heart and one which some providers (like GoodPractice of course) also adhere to. So it is encouraging that the CIPD are getting behind this practical, performance-led mentality. In fact, ‘greater integration to drive organisational change’ is seen by CIPD members as the biggest area of impact in L&D in the next two years, which highlights the importance of linking learning activities back to the strategy and goals of the business as a whole.

The event felt like it had a greater focus on technology than in previous years. There was a ‘Technology for Learning Zone’ in the exhibition area and the free seminars on this topic seemed to always be packed out. Up in the conference there was a whole stream of sessions also concentrating on this area. As we walked around the event and chatted with attendees, this felt like a smart move by the organisers as there was significant interest in what technology can do to really aid L&D and drive the overall performance of an organisation. There were in particular a lot of conversations around social learning and how best to harness it to maximise knowledge transfer within organisations.

Results vs reality

However, the buzz around L&D technologies was at odds with the results of the Learning and Talent Development survey. A mere 15% of respondents cited that a move towards web 2.0 technology will be a major change within L&D in the next two years. Only 22% felt that ‘bite sized’ learning and smartphone apps would have a major impact, whilst only 24% believe there will be a greater use of e-learning across their organisation. Given the focus on these areas at the event (and indeed across the industry as a whole) these figures are extraordinarily low. Feedback from attendees indicates that we still have some work to do as an industry to bridge the gap between the technologies themselves, and their effective implementation, which might explain the disconnect.

Leadership deficit

Another stand out insight from the CIPD report is that nearly three quarters (72%) of organisations report a deficit in leadership and management skills. The main skills lacking are performance management, leading and people management and skills to manage to change. And most organisations (80%) are intending to address this with some form of leadership development activity in the next 12 months.

As you can imagine, there were dozens of exhibitors on hand to talk about their leadership development solutions. But what struck me was how many fail to delve into what actually creates this shortfall we’re experiencing in leadership and management skills. It’s one thing to acknowledge that we have a deficit in this area but what’s the root cause? And how are organisations supposed to practically address it?

According to GoodPractice stats of over 250,000 web-based searches, these are the most in demand subject areas for learners and managers:

1. Leadership
2. Coaching
3. Communication
4. Change management
5. Feedback
6. Interviews
7. Career development
8. Time management
9. Appraisals
10. Mentoring

The GoodPractice team is about to start conducting some in-depth research with its customers and are looking forward to exploring what really works for leaders and managers in practice. It’s only by truly understanding the everyday challenges they face that we can begin to address the deficit in skills that UKorganisations are experiencing at a senior and line manager level.  I’m looking forward to seeing the results and getting stuck into the detail which we’ll be sharing with you over the coming months. And hopefully the CIPD survey results will paint a different picture by the time HRD 2013 rolls around.

Useful links:

HRD Learning and Development Conference 2012: quick-fire links: http://www.personneltoday.com/articles/2012/04/30/58505/hrd-learning-and-development-conference-2012-quick-fire.html

HRD 2012: Leadership framework helps Asda achieve trust:
http://www.peoplemanagement.co.uk/pm/articles/2012/04/hrd-2012-leadership-framework-helps-asda-achieve-trust.htm

HRD 2012: A quick review: http://www.trainingzone.co.uk/topic/hrd-2012-round/174347

The traditional predictive approach to project management is being superseded by Agile

This is not just a faddish buzz-word, instead it is a commitment to get clients the software they actually need - on time and within budget.

The Problem

There is a problem with the delivery of software. The more complex a project the greater the chance the project will be delivered over budget and behind schedule. As a project grows in complexity there comes a point where this potential for failure becomes almost a guarantee. Most experienced project managers understand this and strain their sinews to prevent it from happening and most experienced programmers have lived through the intense disappointment of seeing their work fail to achieve its initial promise. Yet time and again, despite the best efforts of genuinely talented and motivated people, software projects are delivered late, cost too much and do not function as the client expected – Why is this?

For each failed software project the problem typically turns out to be the plan. Now that may seem trivially obvious. Looking back over a failed project it is easy to suggest that if only the plan had been more precise then the project could have been more controlled and so more successful.

This is not correct.

The problem does not lie in the quality of the planning, the problem lies in the type of plan, specifically the attempt to create an up-front plan that covers the entire project life-cycle. This is not so obvious – how can you run a project without deciding what you need up-front?

To understand why up-front planning impedes the successful delivery of quality software it is first necessary to understand what is meant by a plan in this traditional sense, and then see how this concept can be dispensed with and replaced with a new type of planning mechanism.

What’s in a Plan

At the start of a traditional project there is the familiar requirements-capture phase. This typically involves the writing of various specifications, a user specification that outlines the requirements in the language of the client, a functional specification that outlines the requirements in the language of the programmer and then perhaps a fully detailed technical specification that describes the requirements in a pseudo programming language.

Once complete, these detailed specifications provide the basis for all future work. They allow predictions to be made about the project’s costs as well as its anticipated schedule. Specification documents also serve a secondary function. They give both the client and the engineers a form of ‘contract’ that, upon project delivery, allows everybody to compare what was promised with what was actually delivered.

This up-front planning process is often called the ‘waterfall’ model, it is a highly structured methodology that steps through requirements-capture, analysis, design, coding, and testing in a strict, pre-planned sequence. Progress is generally measured in terms of deliverable artifacts: requirement specifications, design documents, test plans and code reviews.

The Waterfall is Broken

There are good reasons why traditional, up-front planning fails. Unfortunately these reasons tend to make both clients and engineers feel uncomfortable so they are rarely spoken out loud.

Firstly, up-front planning means that the specification documents are written before any software is built. Experts, using all their intellectual powers and experience, attempt to imagine the software and in doing so mentally traverse all of its myriad details. Since no software has yet been built, the hypothetical assertions contained within these documents cannot be tested experimentally. In science an hypothesis that cannot be tested is called pseudo-science and by the same token a specification whose assumptions cannot be tested should be considered pseudo-planning.

Secondly, at the start of any reasonably complex project there is always an inescapable knowledge gap. This gap exists between:

• The business knowledge brought by the client
• The technical knowledge brought by the engineers

To begin with these two bodies of knowledge do not mix well as the clients do not really understand the language of software engineering and the engineers do not really understand the language of the client’s specific business. This will change as time goes on and eventually the distinct bodies of information will mix and become one shared information landscape. However, at the start of a project when traditional up-front planning occurs, this inevitable knowledge gap leads to two critical and incorrect assumptions:

1. The client knows what they want their new software to do
   Many clients come to a project with good idea of what they want, perhaps they have spent time and effort working this out, perhaps they have a legacy system that shows them much of they want and what they do not want. However at the start of a project the client cannot know what they want in sufficient detail to create a complete and precise plan. They can provide a business vision and they can provide business constraints but they cannot state in detail the processes required to deliver their vision because they have not yet absorbed the necessary details of the engineering environment. A superficial understanding can be gleaned during the initial planning meetings but this will not produce a sufficient understanding of the software they are commissioning.
   
2. The engineers know how to implement the client’s business vision
   Many engineers come to a project with a good idea of how to build business systems. They will have spent considerable time and effort building other, perhaps similar systems. However at the start of a project engineers cannot know how to implement the precise details of a specific business application because they have not yet absorbed the detailed business knowledge brought by the client. A superficial understanding can be gleaned during the initial planning meetings but this will not produce a sufficient understanding of the software they are being asked to deliver.

Predictive planning fails because an accurate plan requires a genuine, non-superficial understanding of both the client’s business knowledge and the engineer’s technical knowledge. Traditional specifications are created at the start of a project when both parties have not had enough time to come to such an understanding. It takes much effort to synthesize the two bodies of knowledge into a coherent whole, far more than can reasonably be assigned during the requirements-capture phase.

This means that plans created at the start of the project cannot be more than partially informed guesswork. Given that the nature of complex systems make them particularly sensitive to changes in small details, a plan for a complex system created with incomplete knowledge must perforce be a recipe for failure by degrees.

Does this really make up-front planning redundant? Is there a way to make the synthesis of the client and technical knowledge more efficient, perhaps by using advanced planning software? If this could be achieved then perhaps the planners could write effective up-front specifications that lead to accurate long-term costings and schedules.

Unfortunately there is another, more fundamental reason why detailed specifications must fail – regardless of their precision.

A specification is a description that attempts to outline features and functions in a natural language such as English. Yet software is actually written in the very precise syntax of a machine language. Engineers know that only computer code can truly express the details of a software vision, a natural language specification cannot be logically accurate enough. This means that natural language specifications must leave many implementation details open to interpretation forcing the engineer to skilfully choose from a set of implied options. Yet complex systems are sensitive to precisely these sorts of technical details, different choices will lead to different systems and, as often as not, unfulfilled client expectations.

Therefore, even where a specification guesses correctly, the natural language descriptions will contain subtle choices and hidden contradictions. It is only when the fuzzy language of the specification is transformed into the precise reality of the code that these choices and contradictions become apparent.

This leads to a profound truth about the nature of specifications: Greater precision does not lead to greater control. Instead the greater the precision the more varied and subtle the choices and contradictions become.

Planning For Success

Understanding these fundamental flaws at the heart of traditional software delivery, many forward looking managers and engineers are now moving towards a new project control methodology. In contrast to up-front or predictive planning this new methodology uses repeated bursts of short-term adaptive planning.

Agile Software Development throws out long-term planning and with it the traditional concept of a specification. Instead agile projects start with everybody discussing and sharing a simple vision of the end product. The vision is really no more than a mission statement that, at this early stage, explicitly removes the need for engineers to fully understand the business and for the client to fully understand the technology.

This means that an agile project can get started almost straight away, with the absolute minimum of requirements-capture. Instead of a long, costly and ultimately self-defeating planning phase, the engineers get to work building the first version (iteration) of what will become a rolling beta. Armed with a very short term plan covering just one or two weeks of work, the engineers build the first iteration and deliver it to the client for discussion and criticism. The rolling-beta is still only a sketch, an outline of the most important functions and how they might fit together. Mistakes and incorrect assumptions will have been made, indeed given the knowledge gap they cannot be avoided, but the mistakes are identified and quickly eliminated as the rolling-beta is regularly assessed by the client and engineers in close collaboration.

Once the first iteration is signed-off then the process begins again, a new short term plan is created and work begins on the second iteration. This iterative development continues and as the knowledge gap closes so the requirements and hence the software become ever more detailed and coherent.

Embracing Function Creep

As this hands-on process continues the client comes to properly understand the technical environment, what is expensive and what is possible, and as their knowledge grows so they begin to see new possibilities.

Clients changing their minds or adding new features during development is traditionally called function creep and remains the enemy of traditional planners. Yet to suppress this is to deny that clients can learn and modify their expectations as they see their software progressing. Rather than trying to ignore the client’s input, the agile iterative process welcomes it as new and valuable knowledge.

Thus the client is encouraged to re-specify their product as it is being written. This is the ultimate guarantee that, in the end, the client will be satisfied. It is hard for a client to be surprised or disappointed with their software if they have played an active part in designing and deciding the goals at each iteration.

Equally, as the iterative process progresses the engineers will also come to a genuine understanding of the business. This allows the engineers to discuss the business processes with the client in a manner that allows a useful exchange of knowledge to take place. Questions to the client can be appropriately framed using the business terminology both the client and the engineers now share. Since the frequent iterations and short-term planning means that any incorrect business assumptions are quickly discovered, such mistakes can be corrected with the minimum of effort.

Engineers too, once they come to a genuine understanding of the business, can start to usefully contribute to the re-specification of the rolling-beta. New ideas and inspirations, whatever their source, can be welcomed, discussed and possibly incorporated as the software adapts over time.

Job Satisfaction

In summary, an agile software system evolves under the twin constraints of the client’s business vision and the engineering environment’s technical limitations. As the client and engineers come to a mutual understanding so new ideas bubble up and are incorporated as bad old ideas are identified and discarded. Before starting each iteration everybody discusses, negotiates and quickly reaches an understanding of what is actually required to fulfil the next set of short-term goals.

Thus an agile system organically grows its natural complexity out of a fundamental simplicity. As a result there are fewer surprises, the project risks are minimised and the client is more likely to get software that works.

I saw it again yesterday (and if I’m honest I also participated) during a very good conference with Tim Harford and Freek Vermeulen, the break was announced and 80% of the audience pulled out their phones or Blackberry.

One part of the answer may come from David Rock in a recent article, ‘Are Our Minds Going the Way of Our Waists?’ ‘To the brain, simply receiving new information tends to activate the reward circuitry: information itself can be rewarding, which prompted neuroscientist* Jonah Lehrer to coin the term ‘information craving.’ Thus people can easily become addicted to getting information quickly and often’.

I wonder if we are equally addicted to being busy, to doing; to a feeling that we are making progress with the todo list. Does this also provide a reward to the brain. (In a quick search I couldn’t find a study.) How many managers and leaders do you know who aren’t too busy doing?

I suspect, it is also extremely hard for an individual trapped in this cycle of doing and information craving to recognise that they are losing their perspective on themselves and their place in the world.

It is here that I think coaching has a role to play, to hold the mirror up and encourage leaders to make the space for thinking and reflection. After all how many good ideas have you had when rushing around and how many when you have stepped back and given yourself space?

Our good friend, Geoff Bellman was explaining to me the other day that he now always sets up his leadership retreats in beautiful places. “Because it helps people to become immersed in another world and an inspiring and peaceful environment helps participants see the larger picture.” Apparently, it also helps to hold the event somewhere where there is no mobile signal!

One of Geoff’s gifts is questions. He poses questions for people in a wonderful way, that use the space and time to help you step back and reflect. His next leadership retreat will be ‘Leadership in Turbulent Times’  and is to be held on a sail boat from Bali with a small group of senior executives from around the world. It sounds fantastic and I wish I could go, (there are a few limited places left if you know someone who would benefit from some time to reflect) but for now I will have to content myself with reflecting and not doing when I get a chance to take to the Scottish mountains this weekend.

Here’s my question to reflect on from Geoff:

To survive, to thrive, you must be acutely aware of who you are, what you do, and why you do it.

Knowing yourself makes success more likely and sustainable. This calls up questions important to many of us …

■ What does real organizational transformation look like?

■ What am I doing to help or hinder my organization to make the necessary transitions?

■ How can I create order in the midst of disorder?

■ What do I need to know about myself to succeed at work?

■ How can I artfully bring my vulnerability and strength together?

■ What can I do to liberate myself from past patterns?

Great questions for me to ponder as I climb a scottish mountain in the rain, mist and cloud or at any time when I get the chance to reflect.

* Jonah Lehrer is a contributing editor at Wired and the author ofImagine, How We Decide andProust Was a Neuroscientist. He’s also contributed to the New Yorker, the NY Times Magazine and WNYC’s Radiolab.

This year, I wanted to be a bit more thorough and we made some changes to the process. So I asked a member of our Editorial team, the wonderful Jess Adair, to summarise what we did. This will give you an idea of what was involved and might help if you want to carry out a similar activity in your own organisation as part of a learning needs analysis.

You’ll find the results and some analysis at the end of the post, but how we got there is just as important as what we found.

Over to Jess …

How we analysed the search data

Our first stop was to use our analytics package to generate a list of all the individual terms our users searched for in 2010 and 2011 [1]. We knew, however, that this alone would only go some way to tell us which management topics were most popular with our users. Because the report returned every unique term our users searched for, the list generated was, unsurprisingly, extremely long and with a good deal of overlap between some terms (e.g. ‘change management’, ‘change’ and ‘Change Management’ were all listed separately [2]).

Faced with such a long list, we needed to decide on a sensible cut-off point for our analysis; we knew that looking at every single search term, including the thousands that had only been searched for on one occasion, would not be particularly helpful. We noticed that after a certain point in both the 2010 and 2011 lists, the numbers of ‘unique’ searches dropped off quite dramatically. With this in mind, and given that many subsequent search terms were actually synonyms of the more popular words and phrases, we decided to restrict our analysis to identify the30 most searched for terms in 2010 and 2011.

Our next job was to develop a method for identifying all of the synonyms for each of the search terms we wanted to analyse. To do that, we took the following systematic approach:

1. Identify a key word for each of the search terms (for example, ‘time’ in ‘time management) and search for other iterations of this in each list. In 2010, ‘Time Management’, ‘time management model’ and ‘Time management’ were all popular terms (depending on what analytics package you use, the search terms might be case sensitive), along with ‘time management’, which appeared at the top of the original list.

2. Disregard any results that aren’t relevant to the management concept to which the original search term refers. For example, ‘time sheet’ appeared in the results for ‘time’, but was not included because it does not directly relate to ‘time management’.

3. Look for other words and phrases that relate to the original search term. For time management, we looked for phrases such as ‘workload’ and ‘deadlines’. As before, we disregarded any results that weren’t relevant, and included those that were.

4. Record the total number of synonyms and add this figure to the number of searches for the original search term to create an aggregated total for each of the first top search terms.

5. ‘Rinse and repeat’ to generate an aggregated total for each of the search terms displayed in the original analytics reports.

So what impact did all this have on our results? Quite a substantial one! An initial review of the data would suggest that ‘time management’ was the most searched-for term in 2010. However, after taking all the synonyms into account, change management shot to the top of the list, with almost twice as many synonymous searches as there were for the term itself. The results took a similar twist in 2011, with leadership – originally the tenth most search for term – soaring to the top spot after we counted its synonyms.

Undertaking this exercise has given us a much clearer picture of the topics that our users were really searching for in 2010 and 2011 – you can see what this looks like in the two lists below (those who are more visual can feast your eyes on the Wordle images we produced at the end of the post [3]). But what else do these results tell us about the day-to-day learning needs of leaders and managers?

Owen will carry on the story …

The truth is sometimes a bit boring!

It may seem a little boring, but the most striking thing about the search analysis exercises we conduct is how little the results change from quarter to quarter, year to year. An interesting story would be how the financial crisis affected what managers were looking for support with, but the truth is that we’ve seen pretty consistent results over the years. Two key things have stayed the course:

1) Managers seek out content on the ‘basics’ more than any other kind of topic. Change, leadership, communication, coaching … they’re all the core components of what you’d expect to find in any management development programme. A question that arises from this is: if these are core subjects that are covered by management development programmes, why are managers looking for additional support with them so frequently?

2) The on-demand needs of managers are incredibly diverse. In both the use of our content, and the searches that managers make, we see an extreme example of ‘long tail’ demand. For example, in 2011 over a third of all searches across our products were terms that were completely unique i.e. they were only searched for once by one person. Over half of all searches were terms that were used three times or less. Even when you wrap up the synonyms, the content requirements are very broad. To show you just how extreme this ‘long tail’ demand is, this is a to-scale graph of the search term frequency:

What might this mean for you?

This information is available in your organisation. Every search term and every click on your intranet or LMS can be analysed to draw out hidden patterns and stories that can and should inform your strategy. All it takes is putting the request in to the right person and taking some time to do the analysis.

Although we see a similar picture across most organisations we work with, there is some variation. In a very few cases, this variation is significant. So, it would be wrong to look at our results and assume they automatically apply to your own organisation. However, they do provide a very strong indication that your organisation’s mangers are likely to have very similar needs to the ones highlighted in our results. There’s only one way to find out for sure …

[1] If you’re not sure how to do this, your IT department will be able to help you. It’s a five minute job, don’t let them tell you different! For those that are interested, we use both Google Analytics and server side logging to track usage. However, for the purposes of this exercise, Google Analytics was easier to use.

[2] The analytics service is case sensitive when it comes to search terms

[3] The wordle representations of the results, where the more frequent the search, the larger the word appears:

2010 results

2011 results

Measuring trends in organisational learning

For two years now, we’ve worked with INL Consultancy and Reed Learning to produce the Learning Trends Index; an attempt to track the general attitude and spending trends of those responsible for learning and development in the UK. Thanks to the talents and hard work of our research lead on this project, Stef Scott, we now have a useful historical record to examine longer term trends in the provision of learning in organisaions and are able to highlight some of these in the report along with all the data.

Learning Trends was originally inspired by the ASTD’s Learning Executives Confidence Index (LXCI) report, but focused on the UK market and with a slightly broader question base.

While the comparison with the equivalent questions from the US respondents is quite interesting in itself (you can find the latest LCXI report on their website), what’s really caught my eye is how consistently our respondents have highlighted technology based learning, informal learning and social media as key areas that they expect to grow.

Intent versus delivery

This makes perfect sense. The economy is tough and everyone is being asked to do more with less. Technology, in particular, has always been a way for society to get more done with less effort/resources. However, one interpretation of the results of the Learning Trends Index is that, whilst the desire is there, the reality is proving slower than anticipated.

For example, we’ve seen a consistent 30% or so of respondents say they expect to see a significant increase in technology based learning and around 50% report that they expect a minor increase. Those that expect to see a reduction barely even register.

But why has this number not changed over the last two years? Respondents are asked to state what they anticipate happening over the next six months, so for over two years we’ve seen consistent reporting that increases in the use of technology based learning solutions are just about to happen. For some, surely after two years of significant increases in technology based solutions, the use would be no different to the previous six months – that it would have become ‘business as usual’?

I suspect that what is happening is that many of those responsible for learning provision in their organisations see the need for change and want it to happen, but are perhaps a bit unrealistic about how quickly it might actually embed. After all, it’s a significant change which necessarily involves a steep learning curve for those in the learning function themselves.

However, that’s just one interpretation of one of the many trends we’re starting to see and one reason we. Indeed, that’s one reason why we and our partners wanted to start this report – to stimulate debate and discussion. So, please do go and download the report – it’s free – and let us know what resonates with you, what surprises you and what trends we might have missed.

Ah, statistics. If ever there was a topic guaranteed to turn off an audience, this would be among the favourites. But it’s important, so I’ll persevere since someone, somewhere, might find this little primer useful.

But before I go into the detail of the what and the why, it’s worth making two points about statistics. The first point is that statistics is the worst taught subject in the education system. This is simply because those teaching the subject have neither the aptitude nor the enthusiasm necessary. The second point is that statistics is the best tool we have to see the world for how it really is, rather than as the comforting, but false, picture that our brain paints for us.

As learning and development practitioners become more focused on performance, and identifying ways to improve performance, it follows that we are going to be working with performance data more often.

This presents some challenges and requires those working in L&D to develop their statistical knowledge and skills. We don’t need to be experts, but we do need to know the common pitfalls that people naturally fall into and the right questions to ask when we want to establish whether a workplace learning or performance intervention has had a real world impact, or if an intervention is even necessary.

Our internal error machine

Our intuitive reaction to some information is incredibly accurate, especially in those areas where we have genuine expertise. However, understanding the true nature of cause and effect, especially when presented in the form of numerical information is not one of those areas.

Science, and especially the medical profession, has developed tools to overcome our innate psychological problems in this area. Unfortunately, it’s mostly people who have received scientific or statistical training who are equipped with these tools; not the natural catchment for those working in L&D.

To someone who hasn’t gone through that kind of training, it can seem like the person who insists it’s necessary to find/use a control group, or that a larger sample is needed, is causing unnecessary additional work. And that’s quite simply because what makes sense to us intuitively is so powerful. What feels right in these areas is so often wrong that having a basic grounding in statistics can help prevent you from making mistakes and your organisation from wasting money.

Starting out

To start the ball rolling for anyone wanting to develop their statistical knowledge, I thought I’d put together a list of resources worth consulting. These aren’t dry textbooks, but blogs, articles and book chapters from engaging and passionate writers who can help get you started on the road to statistical confidence.

Nathan Green’s S Word is a series in the Guardian that aims to demystify the basic tools of statistics and explain how to use them: http://www.guardian.co.uk/science/series/nathan-green-statistics.

Michael Blastland’s Go Figure column on the BBC’s News website takes news stories and explains how the reality is more complicated than what’s reported. It’s a brilliant way of introducing critical statistical thinking through familiar stories.

Several chapters in Ben Goldacre’s book Bad Science use familiar medical examples to explain complex statistical concepts like ‘regression to the mean’ and ‘confounding variables’ in straightforward language. The following chapters are particularly valuable:

• Chapter 4: Homeopathy
• Chapter 8: ‘Pill Solves Complex Social Problem’
• Chapter 12: Why Clever People Believe Stupid Things
• Chapter 13: Bad Stats

Daniel Kahneman’s book Thinking, Fast and Slow (which I will admit to being almost obsessively impressed with) covers ‘regression toward the mean’ and ‘the law of small numbers’ in an engaging and easy to understand way.

Super Crunchers by Ian Ayres gives real life examples of where data has been used to make predictions and improve business by overturning long-held beliefs. In it he covers the key techniques that can be used to interrogate data along with some key principles such as the dreaded standard deviation. By giving context to these concepts, he makes it easier to understand how they can be used in everyday business.

To help with what to focus on initially, concepts that you should understand in order to truly understand performance data are:

Statistical significance (p-values)

One of the most surprising lessons you learn when you start learning about the statistical nature of our world is just how many things can occur simply by chance. So many events that we think are remarkable, or ‘must mean something’ are simply the result of random variation. Significance is a way of determining how likely it is that an effect occurred simply by chance alone. Why is this important for us to understand? Well, let’s say that a sales training programme resulted in sales improving by 12%. Calculating the significance of that result will tell us how likely that increase is to have occurred simply through natural variation in sales performance.

Correlation (is not causation)

This is the most famous term to come out of statistics, and is simply a mantra against the most common mistake that people make when interpreting data, which is to assume that just because two things happen together, that one causes the other. It’s so easy to make that mistake, because it’s often true. But terrible mistakes can be made, and have been, by getting this wrong.

Post hoc ergo propter hoc

Literally meaning ‘after [this], therefore because of [this]’, post hoc ergo propter hoc describes the temptation to assume that just because an event (such as an improvement in sales performance) follows another event (such as the running of some form of sales training) that the second event occurred because of the first, rather than because of other explanations such as random variation or ‘regression to the mean’. Which brings us to …

Regression to the mean

Someone starts to perform very poorly. Their manager gives them a stern talking to and their performance improves again. The manager pats themself on the back for a job well done; their talking to did the trick. Right?

Regression to the mean is probably the most difficult concept to truly understand, but it’s important if performance data is to be understood properly.

The most famous example of regression to the mean is the so called ‘Sports Illustrated curse’, which suggests that sportswomen and men who appear on the cover of the magazine are jinxed in some way so that the run of great performance that led them to get on the cover mysteriously ends.

Sample sizes, confidence level and confidence interval

You want your evaluation results to be credible. So, if you have a learning solution that 1000 people participate in, how many people’s performance should you be measuring before and after the learning intervention so you can be confident in your results? Well, thankfully there are calculators out there for that, and understanding exactly what a survey or sample data is, and is not, telling you is an incredibly important skill to master.

The law of small numbers (or hasty generalisation)

The regions in which the sales penetration of a brewing company’s beer are highest are mostly rural, sparsely populated and located in traditionally Conservative voting counties in England. How would you explain this?

In the few seconds you thought about that you formulated hypotheses, rejected many of them and may well have settled on the explanation that perhaps the company’s brand plays better in rural areas than urban areas, or that there was no simple explanation for this based on the information given.

However, the key information is that the regions are sparsely populated, which means that random variation is more likely to throw up extreme results. Understanding the incredible impact that small samples in larger data sets can have is incredibly important in order to avoid drawing the wrong conclusions about the causes of excellent or poor performance.

The very best method for thoroughly analysing data ever

I left this to the end, almost as a reward if you made it this far! Reading the blogs, columns and books I’ve recommended and understanding the concepts I’ve highlighted is a good start. But there’s an incredibly valuable technique you can use to make sure you’ve truly understood what the data is telling you, whether that’s data you’re using as part of a performance analysis or the evaluation of a learning intervention: get help from someone who’s already got the skills.

You’ll find them in finance, business analysis, R&D, marketing or IT. Nearly all organisations have experts in analysing data who would be able to help out and make sure that you’re not missing anything obvious. Being from another area, they may also have an alternative viewpoint that could prove useful. Don’t be a martyr and struggle on with that Excel sheet for hours when a colleague in another department might be able to work out that tricky standard deviation in five minutes.

It’s important to note that all of these will highlight areas where there are performance needs. Whether the solution is some form of learning intervention or a change to some other part of the workplace performance ecology is the subject for a future post.

* Interview a cross-section of managers (including senior managers). It’s important to note that you should not be asking the managers what L&D interventions they think their staff need, but rather ask them about what they need to achieve in the coming year/quarter, what the key deliverables of their business plans are, what challenges they think they and their teams/departments might face, and what they personally have found to be the most challenging aspects of their jobs in the last three months. When you interview them, you should solicit real life examples rather than vague generalisations. Recording the meeting and making thorough notes afterwards will allow you to look for any trends across the business or in particular areas. [1]

* Survey a cross-section of all your (target) employees. To get some quantitative data, a survey is a very easy way to get useful information to feed into your analysis. As with the interviews, it’s critical that you don’t ask questions like “How useful would you find a course on presentation skills?” or “What training would help you to achieve your goals?” because you won’t get at what the genuine needs are. Rather ask about what’s challenging about their jobs now, and what they believe future challenges will be based on the future strategy. SurveyMonkey (http://www.surveymonkey.com/) is a brilliant tool for collecting and analysing this type of information.

* Analyse the development plans of a sample of your target population. You don’t need to review all 1500 development plans, just a significant enough a sample to identify the key development requirements across the employee base. You can do this by textual analysis or even a simple tallying process. Again, look for trends and discount any plans that simply have ‘communication skills course’. You’re looking for what the real needs are, not what managers and employees think is the right solution. You’re the L&D expert.

* Review search terms on your LMS and/or intranet. Your IT department will definitely be able to get this information for you and it’s a rich source of information. Take a look at this video where the Head of Google Learning Labs describes what happened when they conducted a similar exercise (http://www.youtube.com/watch?v=KJ-bMbynst0 skip to around 15 minutes in). He and his team did an extensive traditional TNA and compared the top search terms on their learning portal against the top learning needs identified in their TNA – they matched exactly! I wouldn’t want to rely on the results of search terms on their own, but they’re definitely worth reviewing to see what they can tell you.

* Review business plans for the key areas in your organisation. You’ll get this information from the manager interviews, but it’s worth reviewing them to make sure that you (or they) haven’t missed anything.

* Review key business metrics and trends. Understanding where the organisation is failing to hit its key metrics can lead you to some hidden needs. If you work for a large organisation, the people responsible for producing the organisation’s annual report usually have this information to hand. Otherwise, you may need to get it from key departments individually. The kind of danger signals you’re looking for are things like:

• increased customer complaints
• increased absenteeism
• lower sales
• increased product defect rates
• increased staff turnover
• deteriorating financial indicators

You don’t have to do all of these if you’re pushed for time, but a training/learning/performance needs analysis should be thorough. It’s the basis of what you spend your budget on and you want to make sure that your budget is spent as effectively as possible.

[1] Alternatively, you may want to try running a few focus groups. This has the advantage of getting greater numbers, but there’s the danger you run into groupthink and, ultimately, I think interviewing is a better approach overall.

A few years ago now, Richard Wiseman conducted research into the key differences between resolutions that were kept and those that fell by the wayside. You can see a summary of his findings on his website, and get a fuller run down of the research and its findings in his book 59 Seconds [1].

What’s striking about his suggestions is how mundane they are; how lacking in the wow factor. There’s nothing in the list of actions you can take that makes you think “I’d never have thought that”. It’s all pragmatic, common sense stuff which is why a great many people will simply ignore it. It’s too much like hard work; too much effort is required; there’s no short cut.

It’s the long term that counts

At the same time this advice is being ignored, ‘detox’ diets will be all the rage. Now, don’t get me wrong, any diet that encourages you to eat more fruit and veg, and less red meat, dairy and fat is most likely a good thing. However, changing your diet for a week or even a month doesn’t make a difference. Most lifestyle changes only have a significant impact over a lifetime. Stopping smoking just for a month is pointless; stopping drinking for a month might make you feel better in the morning over that period but will make no overall difference to your general wellbeing; ditto for the gym. Long term changes are good. Short term changes simply make you feel psychologically better for a very short period of time.

Silver bullets vs hard work

These are just examples that real, lasting, positive change is usually hard work. That there are no silver bullets out there to magically improve things with little or no effort. Helping social learning take off requires effort on the part of dedicated users, changing the practice of managers in the workplace requires effort on the part of those same managers and those in the organisation who believe change is necessary.

However, there are ways to make it easier. Not easy, but easier. Being informed about what has worked for other people helps. Learning from your past mistakes helps (though you have to be wiling to admit to yourself that you’ve made them). Getting support from friends and colleagues helps.

Is that a negative viewpoint? I really don’t think so. I’m saying the most impactful approach you can take to any endeavour is to put in the effort and take a sensible approach, backed up by evidence where possible. Luck plays a role in almost everything, but you can tip the scales in your favour.

Happy New Year!

This year, I realised that I only had two books that I wanted to highlight and so I decided to tap into the expertise of the rest of the team at GoodPractice to find out what they would recommend.

Below, you’ll find our list of the best books on leadership, management and learning and development in 2011.

Adapt by Tim Harford

Every now and then, someone pulls together the strands of many different ideas and research across a wide range of disciplines to create something that makes you go ‘Of course! That’s so obvious!’ And that’s the trick, it’s only obvious after you’ve read it.

At the start of 2010, Dan Pink did that on the subject of motivation with his book Drive. This year, Tim Harford achieves the same feat with problem solving in his book Adapt: Why Success Always Starts with Failure. In Adapt, Harford describes how problems in complex systems are never solved by clever people coming up with the answer straight away, but through a natural process of trial, error and adaptation.

It’s a truly brilliant book that everyone who ever has to solve a complex problem should know about.

Thinking, Fast and Slow by Daniel Kahneman

Along with his colleague Amos Tversky, Daniel Kahneman paved the way for a new understanding of human behaviour. Rather than thinking of people as rational machines, Kahneman and Tversky uncovered the foibles and frailty of human cognition that inform so much of our modern understanding of decision making, judgement, risk and behavioural economics.

Thinking, Fast and Slow is Kahneman’s summary of of how we think. It’s a tour de force, and understanding the contents is essential for anyone working in jobs that require an understanding of human behaviour. Luckily, it’s not a dense, technical read and I’ve seen no better description than in the Guardian’s review:

“It is an outstanding book, distinguished by beauty and clarity of detail, precision of presentation and gentleness of manner.”

Future Babble by Dan Gardner (recommended by Stef Scott, Editor)

In Future Babble: Why Expert Predictions Fail and Why We Believe Them Anyway, author Dan Gardner takes a critical look at the accuracy of so-called ‘expert’ predictions from across the fields of business, politics, environment and economics. He examines the psychology that explains why people are so compelled to believe and rely upon these predictions, even though they consistently fail.

The New Learning Architect by Clive Shepherd (recommended by James McLuckie, Learning Engagement Manager for Eden Tree)

With so many different learning theories and ideas already in existence, it’s often difficult to get excited about a new one. However, renowned learning designer Clive Shepherd provides a rich argument on behalf of the ‘new learning architect’ concept. Shepherd outlines how learning professionals are failing their clients and customers unless they design environments that allow learners to take advantage of all available opportunities for development, be it formal or informal, or online or offline. Profiles of some of the world’s top learning architects help make this a thoroughly engaging and thought-provoking must read for anyone working in learning and development.

Nail It Then Scale It by Nathan Furr and Paul Ahistrom (recommended by Peter Casebow, Chief Executive)

This may not seem like an obvious inclusion, but there is a lot to learn from the evidence based approach suggested by the authors both for organisations and specialist areas such as Learning and Development. If you are looking to stimulate innovation or make sure you are delivering what your customers need then this is the book for you. It’s full of counter-intuitive wisdom.

A thought on external memory

I just wanted to share one other thought I had when I compiling this list. When I did this last year, I had to rack my brain to remember when I had read specific books. I asked myself whether I had read Drive in 2010 or before [1], and browse my bookshelves at home as well as the ones at work.

This year was different, because every book I’ve read has been on my Kindle, or consumed via Audible (the audiobook service from Amazon). I simply had to check through my history on those services to be reminded of the books I had read. It saved me time and was probably a much more accurate process. It was also a really nice way of reflecting on what I’ve learned over the year, because I read certain books at certain times, which came back to me as I was reviewing them online.

Anyway, those were some of our best reads of the year. What about you?

[1] Luckily that was easy to check because it wasn’t released until January 2010