You’ve hired the best sales team and created a top-notch training experience. Now it’s time to use the information and analytics you already have to help your team live up to their potential and meet sales goals on a consistent basis. Here are six reasons why a learning dashboard is the missing piece of your performance puzzle.

Nothing happens until something gets sold

Sales is the life blood of any organization. A strong sales pipeline is more than a revenue stream – it’s the foundation for your ongoing success. Keeping your salespeople in sync with the needs of your customers and your company is a key element of sales success.

Yet in today’s business climate, staying on track for the long run can be a challenge. How do you track results and drive motivation in a way that keeps your sales team both engaged and focused? Start by mining your training programs and learning management system (LMS) for the data you need to build a learning dashboard.

What is a learning dashboard?

A learning dashboard is an information management tool that tracks a variety of metrics relevant to learning, job performance, development goals, or business strategies. A dashboard aggregates information and displays results in an easy-to-understand graphical format.

A learning dashboard may include several different types of data, including:

• Operational information, such as training assignment completion and learning goals;
• Strategic information, such as training KPIs and personal development goals; and
• Analytical information, such as team performance and individual learning results or rankings.

Learning dashboards typically provide several different types of views. The individual view allows users to track their progress towards completion of their individual learning goals. A team view provides people leaders with the ability to track the learning progress for each of their direct reports and to compare or contrast overall performance against other peer teams. The leadership view allows learning leaders to view results across the global learning population.

Sales learning dashboards versus sales management dashboards: how do they compare?

Sales management dashboards provide valuable information about the status and expected results of your current sales pipeline. Sales management dashboards are outcome-focused by nature, so they tell you what’s happening in terms of activity and revenue.

But what if your sales activity is not producing the expected results? A sales management dashboard usually cannot tell you why your sales team is not producing results, because it does not focus on the sales input process.

A sales learning dashboard, on the other hand, tracks the knowledge and training activities that support your organization’s sales process. When you’re not seeing the kinds of sales results you expect, look to your sales learning dashboard to see the knowledge and performance gaps need to be filled.

Let’s look at the six ways a sales learning dashboard can help improve your sales team’s skill set.

1. A sales learning dashboard enables real-time training results

Successful organizations constantly try to change for the better.

The best salespeople know that training is an ongoing process. Salespeople can use their learning dashboard to confirm that they are always up-to-date on the knowledge and skills they need to do their job. People leaders can use their team view in the learning dashboard to view the overall learning journey for each of their direct reports.

Real-time reporting helps to ensure that benchmarks for new hires are met and that compliance goals are achieved.

2. A sales learning dashboard motivates sales performance

Add gamifying elements to your learning dashboard by creating a leader board and assigning a score to various learning tasks. For example, you could track completions by date and award those users who were first to pass a course. Turn knowledge based assessments into games or activities and stack-rank your player participants according to level of difficulty, job role, team, or location.

Salespersons are competitive by nature. Transforming learning from a task to a game helps to show your team members how to leverage their knowledge and skills for a win!

3. A sales learning dashboard validates the quality of your overall learning program

High quality sales training programs don’t happen overnight; they’re usually the result of years of experience gained through trial and error. Still, there’s no sure-fire recipe for sales training success. What worked last year may fall flat this year.

Sales learning dashboards can help you find the right recipe for learning success. Compare real-world results to the learning activities for each job role to see how well your training programs align with your sales goals. View this data across teams or locations for a big-picture view of your program success and track it over time to get an idea of the “life expectancy” of your overall learning program.

4. A sales learning dashboard can identify trends and find gaps

Learning results tend to vary over time, and a learning dashboard can help you figure out why something works – or not.

A major change in learning results or performance outcomes can be a signal that your training program needs attention. Are your new hires doing a lot better than expected? You may be hiring at a stronger skill level – or perhaps your existing training is out-of-date and a bit to easy. If performance is much worse than expected, your training program could be out of line with your current needs. Take a moment to review your current job descriptions, KPIs, sales goals, and strategies. Use this information to find and fill in gaps that may have appeared in your existing programs.

5. A sales learning dashboard motivates personal development and skills growth

This is one of the biggest benefits for your end users.

A sales learning dashboard offers more than just a “where am I today” snapshot of each user’s progress on assigned training. Add a personal development tab to your dashboard so that each learner can zero in on the knowledge and skills they need to develop to advance their career.

Make it a policy to allow your salespeople time to complete both the job-essential training courses they have to take as well as the personal development courses they want to take. Providing this balance will ensure that your people have an enjoyable and engaging experience with learning.

6. A learning dashboard links learning and performance

A learning dashboard is the logical place to connect, analyze and link learning and performance results together. Your LMS can provide relevant data sets for learning; performance and customer data can be imported from a CRM (such as Salesforce). Over time, you can use predictive analytics to link performance results back to the learning activities that support each skill or KPI.

What does a sales learning dashboard include?

The look, feel, and functionality of a sales learning dashboard is customized to meet the specific needs of an organization. Common elements of a sales learning dashboard typically include:

• Curriculum structure, training categories, training levels, and type of training
• Overview of learning program and curriculum structure
• Details of all courses a learner is registered for
• Status of all learning activities, completions, badges, and rankings
• Personal development activities
• Business outcomes

 

Building a sales dashboard from scratch will require a bit of planning and preparation. A good way to start is by familiarizing yourself with the kinds of data and reporting functions that your current LMS setup provides.  Once you know your capabilities, it’s just a matter of figuring out what information you need.

The graphic below offers a high level overview of the dashboard creation process.

Ask Earthly

Earthly Systems, your LMS professional services partner, has created the Accelerate Sales Learning Dashboard. Accelerate provides a comprehensive view of your learning program, detailed reporting, and customized views for learners, people leaders, and learning executives.

Contact us today to schedule a presentation or to learn how we can work with you to design the perfect learning dashboard for your organization.

Learning Management Systems have evolved from a simple tool to manage, deploy and track learning activities to become the “nerve center” of an organization’s overall talent management strategy. Advanced reporting capabilities enable us to capture and quantify a wide array of learning activities and program results.

Many learning organizations struggle to find ways to access and interpret the virtual gold mine of data contained within their own systems. In this post, we’ll take a look at four typical LMS reporting challenges and recommend actions you can take to resolve them.

1. Who did what?

The typical LMS setup does a good job of capturing basic assignment and course completion data.  While completion results are important for documenting activities such as compliance training, onboarding, or other legally required topics, it’s less critical for other learning programs.

Learning administrators often point out that finding data on course incompletes is rarely an LMS reporting default option. However, incompletion reports can provide critical and actionable details on who is not finishing a course and also identify the page where an individual exits the course. Leaders can use this information to coach learners, manage deadlines, and identify potential issues with content deployment.

Solution: Customized reports

You can obtain noncompletions by filtering a standard Completion Report to exclude those who did not successfully complete the module. If your focus is on finding noncompletes or stopping points, it makes sense to customize a report to generate that data. The parameters for determining course incompletes can be set as follows:

[Overall training audience] – [Completed] – [Not yet begun]

Your new report will include all persons who were assigned the content and opened the module but did not register a completion benchmark within the system.

If possible, be sure to include the last page accessed by each learner prior to exiting the course. Knowing where your learners left a course gives you important information about content deployment and learner engagement. For example, if the vast majority of incompletes stopped at page 17, you might check that point for technical issues that impact progress, such as a page advancement error or bad hyperlink.

Be sure to save any custom reports you create so that you can access them as needed. You may want to set up the report to run automatically on a schedule. For assistance creating any kind of custom reporting function, reach out to your LMS administrator or Learning Technology team.

2. Pinpointing knowledge issues

Most assessment level reports generated by an LMS are set up only to indicate whether or not learners hit an established benchmark, such as an 80% score to pass a course. Question-level reporting on each assessment item – rather than a simple Pass/Fail indication of the overall results – yields detailed response patterns at both the individual and audience level that are of interest to both operational managers and learning teams.

People leaders and mentors can use question level reporting details to focus their coaching efforts on each learner’s exact point of need. If question level results indicate that learners are more likely to miss a question than to answer correctly, the learning team can use this data to determine if (a) the learning content is effectively written and presented; (b) the question aligns to the content and learning objectives; or (c) the answer tagged as correct is actually incorrect.

Solution: Bridging the gaps

Learning gap reports can be tricky to configure and require lots of effort. Typically, learning team members cannot get this type of data unless the LMS is correctly configured to capture the information. Question level reporting also requires an e-learning module to be packaged as xAPI or SCORM 2004.

Your LMS administrator should know the basic parameters of your LMS configuration. If your system cannot deliver the type of information you need from a learning gap report, you may need to work with your IT team, the LMS provider, or an outside vendor to update your capabilities. Once your system is able to interpret this data, you can design the appropriate report type.

3. Mapping the Learning Journey

Many learning leaders like to present learning results in a dashboard format that aligns skills and competencies according to job-specific themes or outcomes. Tracking and displaying relevant data in a logical manner allows leaders to track the big-picture learning journey, rather than focusing on each individual data point. One popular strategy is to configure this output as an infographic, which is visually appealing and easily understandable.

A learning portal built to display learning results can be customized to show different views to different audiences. Senior executives can see program level learning and performance results, while people leaders and mentors receive each individual’s learning results along with a self-reported “temperature check” measuring each learner’s overall comfort level with the learning materials they are consuming. At the learner level, the portal view will track the individual’s progress against the overall curriculum.

Solution: Learning Dashboards

To start building a dashboard, identify your relevant training themes and topics then:

• Specify the knowledge, skills, and competencies (KSAs) that align to each topic;
• Select the training activities that build knowledge of each topic; and
• Select the training activities that prove or support the transfer of knowledge to the actual job.

Once you’ve identified the relevant KSAs and the supporting learning materials, you can focus on proof of competency data points such as:

• A passing grade on an assessment can indicate proof of competency for knowledge-based materials.
• The successful completion of scenario-based learning, executing the task in a work environment, leading a “teach back” demonstration, or a supervised apprenticeship can show the successful transfer of knowledge to real-world job needs.

Additional data points for your dashboard could include job specific metrics such as Net Promoter Score, sales data, customer satisfaction data, or other measures.

Qualitative data points such as satisfaction results, confidence measurement, and mentor or leadership feedback can give you insight about the level of comfort your learners feel with each topic and their levels of engagement.

You can show a learning journey by configuring your data to display as a series of layers with click-through options for viewing, or as a series of analog-style gauges similar to the speedometer and tachometer on your car’s dashboard. Recommended layers (or gauges) include:

• Training themes or topics
• Completion Status
• Assessment scores
• Mentoring or hands-on activities
• Self-reported data
• Informal self-led training (i.e., articles read, videos watched, conference attendance, etc.)
• Job-related metrics

Your internal technology team or an IT vendor who knows your LMS is your best bet to design and build the dashboard you need. Since business needs are subject to frequent revision, we recommend that you plan for annual updates to ensure that your dashboard reports always support current business needs.

4. Measuring the cost/benefit of a learning program

Learning leaders and instructional designers typically focus on training quality. Business partners, who are responsible for results, are more inclined to notice the impact of learning. Designing, delivering, and maintaining learning programs requires ongoing time, effort, and funding. When training programs fail to deliver the expected results, revenue targets can be missed. If learning costs run higher than budget levels, entire programs can be placed at risk of cancellation, further straining an organization’s potential to grow.

Solution: Calculate the ROI of learning programs

ROI, or return on investment, measures the revenue gained from a planned expenditure. In the past, calculating a Learning ROI may have seemed daunting because of the number of variables required to create a single unit of learning. However, today’s successful organizations have become adept at identifying and capturing relevant quantitative data points.

A simple ROI

A basic learning ROI calculates the net benefit that can be directly attributed to a learning program by the total cost of the program. The net benefit – or return – can be expressed as:

• A financial measurement, such as increased profits; or
• A performance-based ROI.

To calculate a performance-based return on your learning investment, compare the increase in customers attributable to a learning program against annual goals. If the number of new customers gained exceeds your goal amount, you have a positive ROI; any number less than your goal is a negative ROI.

You can take the performance-based ROI a step further and divide the number of new customers by the amount of net profit generated by new customers to see the average revenue per new client.

Learning program costs

To see the true cost/benefit of learning, you’ll need to measure the cost of all of the inputs needed to design, deliver, and maintain a learning program. Learning program costs are generally a combination of labor, technology, and fees. Typical learning program costs can include:

• The number of hours of work contributed by learning team members to create each learning deliverable. Work hours include not only the actual time spent writing and editing the learning materials, but also project meetings, research, communication, content uploading, testing, and revisions.
• The total number of hours of work contributed by Subject Matter Experts (SMEs), Stakeholders, and business partners for project meetings, content reviews, and content approvals.
• A percentage of the direct costs of learning technologies (i.e., software programs, licensing fees, subscriptions, file storage space, etc.) required to create, deploy, and manage the learning program.
• Labor costs per hour for all learning team members and business partners involved in the creation or approval of learning materials.

Learning program benefits

Learning programs can benefit an organization by providing increases in both revenues and productivity. Calculating revenue increases is a fairly straightforward task. Your Finance Department can often provide direct revenue figures for specific programs.

Productivity increases may not show immediately as a revenue gain; however, as your employees grow their overall skills and competencies, you may notice improvement in areas such as revenue growth, client retention, client satisfaction, and a shorter sales cycle. Be sure to include the revenue increases or cost savings attached to each of these data points as a learning program benefit.

Seeing your results

In most cases, you can work with your LMS administrator to design a simple ROI spreadsheet to calculate basic costs or productivity measures. You can set this type of report to run on a monthly basis then track the cost/benefit data over time.

Your internal technology team should be able to work with you to gather internal data for a detailed program-level ROI. A vendor partner who is knowledgeable about your LMS can work with you to design and build a learning portal or dashboard to display your learning program ROI.

A Final Thought on LMS Challenges

In 2013, a new protocol for tracking learning activities was introduced. The Experience API (commonly known as xAPI or Tin Can) allows organizations to track learning activities that happen in any context, both inside and outside of the LMS. With xAPI capabilities, an organization has the capability of measuring not only formal LMS-generated learning activity, but also the informal learning events which are estimated to make up about 70% of all workplace learning.

Some learning leaders consider xAPI to be the proverbial elephant in the room. Most “compliant” legacy LMSs are configured to only gather SCORM data. Taking one or two extra steps to post-process information from end users will pay off down the line by giving you a more complete picture of your overall organizational learning.

A key benefit

xAPI takes reporting capabilities to a new level. With xAPI, you can receive detailed test results, including question-level reporting. The protocol enables you to track learning events completed offline, as well as non-traditional learning activities, games, simulations, informal events, and real-world performance results. xAPI learning results can be reported in the form of a dashboard, learning analytics, or noted as an open source badge.

Organizations should address xAPI needs during their initial set up, in order to make use of all of the benefits the protocol can bring to a workplace. If your LMS is already in place, reach out to your Account Executive to learn how to reconfigure or upgrade your current capabilities.

To see how advanced reporting can really save you time, money and frustration, request a demo today.

When it comes to your SumTotal LMS user interface, the same fact rings true no matter what industry you’re in or who your learning audience is: standard, non-customized interfaces don’t make the cut anymore. Templated, basic, feature-less LMS user interfaces leave learners feeling bored, unmotivated and disengaged across the board.

The importance of customized user interfaces and LMS dashboards is something we talk about a lot here at Earthly, and there’s a good reason for that. Why? Because it’s so important. Delivering learning in a way that speaks to your learners and presents them with something familiar, user-friendly and engaging is absolutely essential… and the best way to achieve that is by developing a customized user interface.

In this post, we use the healthcare learning industry as a specific example but bear in mind that a lot of these suggestions can be applied across the board, regardless of industry or sector. Get in touch with us if you’d like to chat more in-depth about how these customizations can be applied to your organization and its learning goals.

The issue with your LMS

Healthcare academies, including healthcare universities or learning divisions of leading healthcare bodies, often use an LMS as a means to help practitioners share knowledge and earn continuing education (CE) credits.

However, many of those LMS aren’t being used to their fullest potential. They’re left basic and non-customized, delivering a bad user experience and resulting in low repeat visits. Simply put, learning managers are investing tons of time, energy and money into building an LMS that learners take one look at and then abandon it.

This is undoubtedly a frustrating but common problem – and one that can be remedied with a little bit of customization. By customizing your user interface, you’re driving a deeper connection with your learners and showing that you have their best interests in mind. Customization allows you to present the information they need when and where they need it, while also giving you an outlet to target groups of learners with specific messaging or content. It’s a win-win situation.

What is a customized user interface?

SumTotal user interface customization is exactly what it sounds like: it’s the process of customizing the user-facing elements of your SumTotal LMS. It involves everything from your visual identity to data reporting and helps you ensure branding and organizational consistency within your learning platform.

Customizing your user interface goes far beyond just branding it with your company’s colors, fonts and logo. It’s about developing something that is tailored to your learning initiatives and that delivers unique learning value to your users. It takes into consideration your organization and its brand, your learning program and its intended outcomes, and your learners’ needs. The changes can be reflected in your courses, whether they are CE credit courses of product knowledge courses, as well as on your learner homepage dashboards.  

Why does your healthcare university need it?

Engaging in user interface customization is a great way to boost learner engagement, increase repeat visits and ultimately generate greater knowledge retention.

Healthcare universities in particular often have a huge amount of valuable knowledge resources across other platforms – some hidden in plain sight on their websites, others hidden away in email inboxes and faculty projects. By taking these resources and making them available in a customized, user-friendly LMS, you’re going to deliver a superior learning experience and make better use of the content you already have.

In other words, it’s going to help you stop reinventing the wheel.

The result? More engaged learners who actually access the information you’re putting out there, leading to better-equipped practitioners and the actual fruition of your learning goals.

An example: Health Care Academy

To make the process seem a little clearer, let’s look at a visual example of a customized user interface. Below is a sample mockup of a user interface for a healthcare academy. Let’s imagine the academy was created as a means for healthcare professionals to earn continuing education credits and share knowledge among industry thought leaders. It promotes bite-sized learning, live events (like webinars), tailored learning paths, and the ability to learn anywhere, anytime on any device.

Do you think this could all be achieved this through a standard LMS user interface? Of course not. A customized SumTotal LMS allows the academy to deliver the content they want to deliver, while also increasing its brand value through direct, obvious links to the corporation and its brand.

Every detail you see is intentional, geared towards making an engaging and enjoyable experience for the user.

• The branding is clear and consistent, from the visual branding down to the language, offering a constant reminder of the corporation behind the learning platform
• Learning options are categorized according to the topics that interest their typical user, from consumer health and wellness to sustainability
• The learning experience is heightened thanks to interactive, visual content like webinars and videos, introduced using custom video widgets
• An intro video is offered to the learner can easily familiarize him/herself with the LMS
  Navigation for the learner dashboard, transcript download and evaluation options are clearly displayed

Let’s look at each of the customization options in detail.

Custom branding: How can you apply your corporate brand to your customized SumTotal user interface?

If you’ve had any experience in marketing, you know that one of the key aspects of a good brand is consistency. Unfortunately, many learning managers fail to make that connection when developing their LMS.

When creating an LMS, it’s important that the branding be similar – if not identical – to your corporate website. This kind of consistency allows a learner to make a more immediate, stronger connection between the learning content and your organization. When a user is learning within a branded user interface, they are unlikely to forget that the LMS they’re using is associated with that particular institution or company. The logo appears at the top of the page, even when you navigate away from the homepage, the fonts are consistent, and the company’s messaging (like its mission) is transmitted throughout the LMS content.

When helping organizations customize their SumTotal LMS, we ensure that the design incorporates everything from their colors and fonts to their language and tone of voice – while also ensuring usability and attractiveness.

Custom news pages: How can I present customized content to my users?

A hugely advantageous but hugely underused customization feature of your SumTotal LMS is the news page, or news content is on the homepage. When customized, this section of the LMS offers a place to share custom content based on the user’s role or active mode. As a healthcare academy, you could offer up different content to a nurse, a doctor, a dentist, or any other healthcare practitioner based solely on their role and the learning programs he/she would be most interested in.

The key benefit of this is that learners have the exact information they need at their fingertips, without having to dig through content categories or tags to find the programs or resources that are right for them. It also offers you an opportunity to reach groups of people (or even individual learners) with targeted messaging.

For more information on this topic, you can read our post on LMS learner dashboards in the healthcare industry here. You can also check out our post on how customized LMS news pages can be used for a more effective onboarding.

Custom widgets: What kind of content can be included in a customized user interface?

Chances are your organization already has plenty of useful resources for learners just sitting on your website or in various digital files and folders. The great news is that a lot of this content can be built into your SumTotal LMS, no matter its format, through custom widgets.

Here are just some examples of what forms of content you can include as part of your customized LMS user interface. As you read through them, think about what resources you already have (or that you may currently be developing) that fit into these categories and might have a place on your LMS.

Best Practice Resources

Best practice resources are resources on topics that matter most to your organization and your learners and offer top quality advice on how to address those topics. They are typically specific to a particular skill or issue.

These resources typically take the form of an infographic or a video. They can be sourced from industry leaders or even from top performing team members in your organization. For a healthcare university, you might seek topics from your faculty leaders or top lecturers.

Videos

A custom video widget enables users to browse and watch a playlist of videos. Those videos can be hosted on an external site, like YouTube or Vimeo, or can be uploaded to a private, internal playlist within your SumTotal News Page. Best of all, any new videos uploaded to the playlist will automatically be presented in the widget — no extra step necessary.

In the healthcare industry, you may already have videos detailing best practices, patient stories, or unique case studies. With a custom video widget, it’s easy to gather all of these videos into neatly organized playlists for easy viewing directly in your LMS.

Want to learn more? Click here to download our guide on custom video widgets for SumTotal.

Podcasts

If your organization is already producing a podcast, whether on your own or in conjunction with industry thought leaders or another brand, why not repurpose it as supplemental learning material? Audio files, as well as podcast episode transcripts, can be easily uploaded to a customized area of your SumTotal LMS.

Articles & Blogs

Most healthcare academies will have plenty of useful articles and blog posts floating around – after all, this is the kind of content production that goes on most frequently in these environments. From academic journal articles to media articles and guest blogs, it’s important to look for ways to repurpose these knowledge resources within your LMS. Taking your top-rated posts and sharing them with learners within your LMS user interface provides additional value without any extra work.

Do you see the potential?

As you can see, there are many ways in which going beyond a standard-issue LMS and customizing your SumTotal user interface can significantly impact the user interaction and engagement within your healthcare university dashboard.

• Custom branding can provide a more consistent message across your initiatives and drive a deeper connection between your organization and your learners/users
• Custom news pages can deliver a superior learning experience through personalized news, help learners find the information they need with greater ease, and help your organization target users with the appropriate messaging and content

• Custom widgets can be developed to repurpose the valuable content and knowledge resources you already have across other channels, providing greater value to your learners with minimal additional time or resource commitments

If you’re interested in learning more about how these customizations can work for your organization, get in touch. At Earthly, we’re leading experts in SumTotal LMS and deliver user interface customization options that are perfectly tailored to any organization. Get in touch today.

Struggling in your efforts to understand the value of your learning programs? Maybe you’ve uncovered your ROI, but can’t explain why it happened?

Uncovering the ‘why’ behind your program’s success (or failure) comes down to two things: learning and job performance. In this eBook, we walk you through the best ways to measure learning and job performance — and, crucially, how to analyze your data.

Download our FREE eBook today and you’re already on your way to delivering superior programs (and answering all your stakeholders’ hard-hitting questions).

Learners remember up to 95% of visual content, compared to just 10% of the text. With numbers like those, it’s hard to deny that video has an important place in your learning efforts.  

Custom video widgets are the best, easiest way to incorporate video content in your SumTotal learning environment. Download our FREE guide to learn more.

Want to know how to help your learners learn smarter, not harder? Our FREE guide outlines 50 tried-and-true methods for accelerating learning, from icebreakers to reflection exercises.  

Download your free copy today for better learning tomorrow – guaranteed.

There’s no denying the importance of learning and development. In 2017 alone, approximately $96.3 billion was spent on L&D worldwide. However, as learning professionals, we know there are always strings attached to that number.

No matter your organization’s L&D budget, you will have stakeholders asking for evidence of a return on their L&D investment. In fact, return on investment (ROI) is one of the biggest struggles plaguing learning managers today. After all, it’s not just the stakeholders who want to see evidence of results – most L&D professionals are results driven as well.

The issue comes with the fact that L&D is among the most challenging business divisions when it comes to demonstrating ROI. But what if it didn’t have to be so challenging? What if we’re approaching our ROI measurements all wrong?

ROI measurement today

There are many tools used by modern learning professionals to measure the ROI of their initiatives and, while each of them offers value, they’re often insufficient – particularly when used in isolation.

 

 

One of the most popular tools is Allen’s Model, which measures the impact of a learning program across eight levels:

• Level 1 determines whether students were happy or dissatisfied with the learning program (i.e. participant satisfaction)
• Level 2 determines the degree to which cognitive knowledge was acquired, usually established through testing
• Level 3 determines the degree to which technical skills were acquired, usually established through observation
• Level 4 is attitudinal and assesses learners’ reactions to and beliefs about their pre- and post-learning
• Level 5 determines the degree of behaviour change, usually established through observational feedback
• Level 6 determines whether a learner can show he/she is capable of using the learning and knowledge
• Level 7 summarises the previous six levels by averaging out the learning on a cumulative basis
• Level 8 is an attempt to triangulate the change and measure its impact on the culture and outcomes of the organization

While a useful tool, Allen’s Model is incredibly complex and very hard to scale across a large number of learning initiatives. In other words, it’s not a great option for small teams who already feel as if they’re in over their heads.

 

 

A similar popular tool is the Kirkpatrick Model, which uses four levels rather than Allen’s Model’s eight. Here, we’ll break down the different levels and their pros and cons.

Level 1 involves measuring learners’ reaction and acceptance. This is typically done using simple evaluations like questionnaires or “smile sheets”.

Not only are these evaluation methods cost-effective, they often have high distribution and completion rates as they’re typically issued in the learning setting. However, the answers are usually quite generous. They may also indicate satisfaction with factors we’re not necessarily interested in from an ROI point of view.

Level 2 involves measuring learning through post-learning tests. This could be anything from simple knowledge checks, like multiple choice tests, to comprehensive assessments.

These tests can be very useful in giving a basic understanding of learning success, particularly when combined with pre-tests. However, they are limited in that they only tell you about what a learner is able to recall immediately following the learning.

As we know, knowledge retention tends to decrease rapidly following learning, and a positive test result might not actually indicate the learner’s ability to apply their learning in the days, week or months ahead.

Level 3 involves measuring whether an outside party can observe a changed behaviour following learning. Typically, an objective observer will rate the application of a learned skill, such as in the case of employee/manager assessments or post-training evaluations.

It’s hard to ignore that there are many challenges with this form of evaluation. If you are able to get managers or other observers to agree to participate, you need to ensure that they fully understand what the learners were trained on and how they can identify positive changes. T

These evaluations are also typically hard to scale (as they require a great deal of time and manpower) and there will almost always be an observation bias at play.

Finally, level 4 measures the impact of your learning initiative. It involves determining whether the training actually taught the right things, whether the training was applied, and whether that had the intended outcome.

This is the most important metric to measure and is all about tying your learning back to your organization’s wider objectives. Even if your questionnaires, tests, and observations are positive, a lack of proof of impact indicates your learning program wasn’t a success.

Of course, Allen’s Model and the Kirkpatrick Model are not the only two options out there when it comes to evaluating a learning initiative’s success; however, the above issues remain in most evaluation tools.

Our fascination with causation

The issue with these tools and the way that we approach ROI is that so much focus is being placed on causation.

Learning professionals get caught up in formulas and figures to prove a direct link between learning and organizational success, despite the fact a direct link almost never exists.

By focusing on a need to “prove” that an L&D investment produced a specific result or contributed to an organizational objective, we completely ignore the fact that there is a multitude of factors outside of our control.

Luckily, there is a better way – but it involves a shift.

A multi-faceted learning measurement model

Of course, we can’t simply ignore ROI. Instead, it’s up to L&D professionals to shift the dialogue around ROI from one focused on causation to one focused on correlation.

How? Through data.

It has never been clearer: L&D needs to be a data-driven function and connect its work to existing, valued organizational metrics.

To achieve this, Karen Hebert-Maccaro of O’Reilly Media suggests we shift to a more sophisticated, multifaceted learning measurement model – one that acknowledges the fact that learning is a human experience, with many factors at play.

Her suggested model is founded on correlation (not causation), exploration and monitoring.

Correlate

A correlation-based framework allows L&D to draw direct lines between learning initiatives and business priorities. It is up to the L&D department to look at those priorities, ascertain their relevant human capital needs, and determine how it can facilitate those needs through learning initiatives.

By following this framework, learning professionals are able to give greater focus to determining a correlation between learning engagement and whether the organization’s human capital needs are being met.

The same correlation-based approach can be used to evaluate data that already exists within your organization. For example, you might look to see whether your most engaged learners are also your best salespeople.

Explore

To demonstrate learning ROI in the most effective way, we also need to be selective in the measurements we do.

While it is crucial for learning professionals to measure post-learning behaviour and its impact on the business, it is equally crucial to focus efforts on the most important programs – particularly if your organization offers 100s or 1000s of programs every year.

These will often be high-investment, low-frequency learning initiatives.

Monitor

Finally, it is still important to ensure quality control by monitoring your learning initiatives as they progress through questionnaires and pre/post-testing.

However, under a multi-faceted learning measurement model, it is recommended that data collection in this area be as limited and precise as possible. Hebert-Maccaro suggests using smile sheets but limiting them to a handful of important questions, such as:

• How effective was the instructor?
• Will you use what you learned over the next 30 days?
• What specifically will you use?
• What was missing?

She also suggests we pay greater attention to metrics that help us understand learner behaviour and how best to alter programs to meet those behaviours. This would include looking at:

• Linear (structured) vs. non-linear learning (jumping in and jumping out)
• Deepening (focused learning) vs. broadening (various topics)
• What learners are learning, searching for and not finding (i.e. topical analyses and trends)
• Industry comparisons – what your competitors are doing and how their learners are engaging

Learning Imperative

Hebert-Maccaro’s model also uses an underlying justification – a learning imperative – as a foundation for our commitment to a new narrative around ROI:

This case study outlines how Earthly helped a client in the pharmaceutical industry implement SumTotal Report Bundles within their L&D department to:

• Transform a flawed manual reporting system into something far more efficient
• Better align their learning data with business objectives and strategy
• Get value out of their data faster, resulting in increased information flow, improved decision making, save costs, and more

Download your free copy now for a real-world example of how these report bundles can benefit your business.

 

The future has arrived. Robots can be found in your local shopping malls, your phone can have a conversation with you, and cars are starting to drive themselves. Artificial intelligence can be found in all aspects of our modern lives – and it’s time for learning to catch up.

It may sound daunting, particularly to anyone who’s been in the learning game for a long time, but it doesn’t have to be. If anything, it’s a great advancement that will make your role as a learning manager or provider even more crucial.

Understanding the lingo of AI

We all know that we’ve left the industrial age behind us, moving on to the information age where we have access to more data than we know what to do with. It’s something that’s been talked about time and time again.

Less discussed, though, is the fact that we’re slowly but surely leaving the information age behind for the ‘imagination’ age: another revolution, this one characterised by actually making sense of all that data. We’re finally starting to take all that data, all that information, and all our ideas and making the steps to transform it all into something that actually improves our daily lives.

That’s the foundation of AI. It’s pretty simple when you think about it – and it’s exciting. Where some people get lost, though, is in the various definitions associated with AI. Let’s break them down.

Artificial Intelligence (AI)

Artificial Intelligence, or AI, is the term given to machines that can do things that are characteristic of human intelligence.

Machine Learning

Machine learning is a specific subset of AI. It’s the ability to learn without that learning being explicitly programmed. Breaking that down into simpler terms, it’s when an AI has been trained  to learn things on its own, thanks to a combination a baseline set of instructions and a massive amount of data.

A great example of machine learning is recommendations, like Netflix program recommendations or Amazon product recommendations. Baseline set of instructions + data on user’s previous habits = machine learning in the form of recommendations.

Deep Learning (DL)

Deep learning, or DL, is like machine learning, but taken a step further. If machine learning is

an imitation of the human brain, deep learning is that same brain but with the neural networks built in. It’s all about layering the data in a more complex way than machine learning can handle, allowing us to better differentiate between elements.

In L&D, an example of DL would be not only collecting data on individual learners, but using that data to predict future organization directions. For example, one might collect data on a new hire all the way from his/her onboarding down to his/her continuing growth within the company and eventual exit. Using DL, you would use all that data to predict what skills & competencies allowed that person to perform well in the role, and apply that information to your future learning content (and even the job description and role).

Personalization

As we’ll explain further, personalization is the aspect of AI that has the most significance within L&D. Personalization in this case is two-fold: a combination of personalized information (i.e. information that’s tailored for a specific user) and the personal (i.e. communication that’s centered on a human-like experience).

When it comes to personalization, you can’t just rely on technology: your learning still needs that personal element.

What that means to L&D

L&D initiatives generally started in the form of general, in-person group training. As technology improved (and purse strings tightened), the industry turned more towards e-learning. While we can’t deny that e-learning is efficient and convenient, it has somewhat dehumanised the training process.

What AI and Machine Learning allow us to do in L&D is move back to that personalized, human learning – even if it continues to be delivered via technology.

Like in all aspects of our lives, AI is freeing us up to become more efficient when it comes to learning. AI can be used in your organisation to take care of all the menial, tedious, repetitive tasks that take up so much of your time – so you can go back to spending your mental effort and time on the things only humans can do.

It allows us to introduce that second half of the personalization equation: to create learning initiatives that are compassionate… human… and that actually solve the problems your learners’ problems.

How to personalize your learning

People expect personalization. Nearly whatever your learner does today is a seamless experience – from logging onto their work computer to browsing the web at home. This is so true that whenever we come across a non seamless experience, we notice. It impacts our flow and our productivity.

By personalizing your learning, you’re providing a more seamless experience: one that gets the right content in front of your learners at exactly the right time. There are three steps to achieving this:

1. 1. Analyze your current situation. Take a look at your current learning initiatives – the tools your using, your content, and how your distributing that content. Get to know it inside and out.
   2. Understand the gap between your current situation and where you want to be. You need to understand where you want to go before you can begin to understand how you’re going to get there.
   3. Take action. Come up with a roadmap that outlines everything you need to procure, implement and grow in order to achieve that desired state. Everything starts with a plan.

   We shouldn’t be afraid of innovation. Innovation isn’t going to replace our jobs… if anything, it will make our roles in the learning industry ever more important. AI and Machine Learning give us the opportunity to hand off the menial tasks that take up most of our time (without the cost of additional hiring) so that we can go back to focusing on developing personalized, effective training programs. By following the steps we outlined here, you’re getting your learning ready for the future… a future that’s already here.

You know that your SumTotal LMS is teeming with useful data. The question is, how do you access that it?

We are living in an Information Age, whereby every company – no matter their size or industry – is looking to gather as much data as possible on their customers. It is a fact universally accepted in the business world today. We know that this data is necessary for analyzing how people interact with their products and services, in turn helping businesses improve those products and services and better connect with their customers.

The same is true for those of us in the learning sector, and LMS software is becoming increasingly more complex in order to serve those needs. Your SumTotal LMS tracks valuable learning activity: how many learners have completed a course, the average time learners spend on a course, pass and fail rates, and more.

All this data is undeniably useful; however, only if the data is properly analyzed.

Your learning data is ultimately useless without meaning behind it. Without data analysis, the information is merely numbers. For this information to really be useful, you need to transform your data into something that will positively support your business and help you deliver an even better experience for your learners and team.

This can only be done through LMS reporting.

Your SumTotal LMS comes equipped with basic reporting capabilities that allow you to gather all your information in a single place. These reports are a great start for gathering the data you need to make important decisions and to improve your learning programs.

However, there will likely come a time when you need more sophisticated reporting that allows for more detailed analysis.

If this is frustrating for you, you’re not alone. Organisations are consistently concerned that their LMS is not meeting their reporting needs – in fact, research has shown that this persists as one of their top concerns over time (according to a 2010 study by Bersin & Associates). At the very same time, L&D departments are under increasing pressure to demonstrate measurable results of their learning programs. It is certainly a frustrating cycle.

But there’s a solution.

At Earthly, we specialize in overcoming those frustrations. We’re driven to deliver more and better forms of LMS reporting – reporting that actually suits your industry and your business, not just ‘one-size-fits-all’.

Using Jaspersoft technology and our years of knowledge on what is most valuable to organizations like yours, we’ve created a bundle of advanced LMS reports. These out-of-the-box reports provide you with the analysis needed to really understand your learners, their learning experience, and how to use that information to improve the value and ROI you see from your learning initiatives.

If you’re looking for even more sophisticated reporting, our team is also equipped to help you customise your reports as needed.

Your data is only useful if you have the tools available to mine it and discover the value underneath. There’s no need to be limited by basic reporting. Isn’t it time to overcome your frustrations and start working with LMS reports that get you the information you need, quicker and easier than ever before?

We’re ready when you are.

To see how advanced reporting can really save you time, money and frustration, request a demo today.