When it comes to learning management systems, “success” means different things to different organisations. Are you aiming to boost learner engagement or reach a 100% compliance target? Perhaps you’re looking beyond the LMS itself, striving instead for particular business metrics such as increased sales or improved customer satisfaction.
Contrary to popular belief, “success” is not simply launching your LMS. It’s not about the system working as expected, nor is it about collecting vast amounts of data. LMS success is inextricably linked to your organisation’s success. The questions that matter are: How is your LMS helping you drive real-world results? What impact does your learning have on your people, outcomes and productivity?
To help you evaluate the success of your LMS, we’ll examine some common signs to check if your system is working against you, and using the Kirkpatrick model of training evaluation, we’ll show you how to turn LMS failures into successes.
Common signs your LMS is working against you
If any of these ring true, your system might be undermining rather than enabling your learning objectives:
- Completion rates are your primary success indicator
Your reports focus on who finished what and when. Meanwhile, questions about knowledge retention or performance improvement go unanswered. - You can’t connect learning to business outcomes
The LMS exists in its own bubble. There’s no way to link training activity to sales figures, customer service metrics, operational efficiency or employee retention. - Data doesn’t travel beyond the platform
Your learning analytics sit isolated from HR systems, talent management platforms, business reporting tools and performance dashboards. Each department works from incomplete information. - Managers have no sight of their teams’ development
Line managers can’t see what their people are studying, can’t reinforce key concepts, aren’t involved in applying new skills on the job, and have no input into development priorities. - You struggle to justify the investment
The LMS costs money and people are using it, but when leadership asks about return on investment, you’re left pointing to completion statistics rather than business impact.
Sound familiar? It shouldn’t be like this; your LMS needs to function as one tool within a larger, connected learning infrastructure.
Using the Kirkpatrick model to evaluate your LMS
To help you structure your thinking around LMS success, we will take inspiration from the Kirkpatrick model of training evaluation (and Phillips’ additional fifth stage). As a quick recap for those of you who are unfamiliar with these levels, they are:
- Reaction – what do people think of your learning?
- Learning – how does your learning contribute to the acquisition of new knowledge, skills, behaviours and confidence?
- Application – are your people applying your learning on the job?
- Impact – is your learning having the desired impact?
- ROI – how does the cost of your learning program compare with the results?
These five stages can be used to assess the success of your own LMS, and to help you prove this success when it comes to reporting back to your stakeholders. The better you get at measuring the success of your LMS, the easier it will be to secure further investment in your learning program going forward.
Level 1: What do people think of your LMS?
The good news is that you have a nice easy start with the first level of measurement. What you want to find out here is “Do people like your LMS?”. Was it delivered on time? Was it on budget? Is the quality good?
These are simple questions to answer. Don’t get disheartened if the answer to any of these questions is “no” – it doesn’t render your entire project a failure. Realistically, LMS projects can’t or won’t always be delivered on time or budget.
If you wanted to take it a step further, you could conduct a survey to gather the opinions of learners, managers, administrators and stakeholders. This could sit on your LMS, or you could send out an email asking for initial feedback.
Level 2: Is your learning effective?
So, your LMS is in place. But who is logging on? Are they enjoying the learning experience? Level 2 is about your users, their interactions with your LMS and whether it becomes part of their ongoing development. Some of the data you can collect to measure success here includes:
- The number of learners visiting in the first week vs the next four weeks.
- The number of learners visiting your new LMS vs your old LMS.
- Return visits vs learners only visiting once.
- The length of visitors’ engagements (are people visiting your LMS for 30 seconds or 30 minutes at a time?).
You could also consider adding rating systems within the LMS to help gauge opinions quickly. For instance, once a week or month you could add a question to your LMS asking questions like “How easy do you find this LMS to use?” or “How has your use of the LMS impacted the amount of learning you do?”. Simply clicking a response makes it quick and easy for learners to have their say, and you’ll discover how your learning is working.
Level 3: Is your learning being applied?
Now we move onto measuring the learning itself, rather than the system.
At this stage, you need to prove that the learning process itself has improved. The focus is on measuring whether employees are gaining more knowledge or skills for the same time and money invested.
Your LMS analytics are essential here. Track the total learning across your organisation and demonstrate an overall increase per unit of investment. For example, show that for every additional £X spent on your LMS, learning has increased by X%.
Application of learning can include the following considerations:
- If your LMS enables mobile learning, what is the impact of enabling learning during otherwise less productive times outside the workplace? What is the value of this additional “useful time?”
- What is the value of the learning decisions you have made? For example, scrapping programs with poor satisfaction levels, switching to a better vendor or replacing unused content with fresh content.
- How has your LMS saved time and money spent traveling to and from face-to-face learning sessions? Are people spending more time learning and applying that learning than they were previously?
Level 4: Is your learning having the desired impact?
Level 4 is where things start to get really interesting.
Looking at the impact of your learning means also measuring the performance of your people. Ideally, your LMS will be integrated with a performance management system such as Totara Perform. This will enable you to link learning to employees’ competency frameworks. Over time, as people learn more and apply their new skills and knowledge in the workplace, you should see the number and level of competencies in your organization grow. Ideally, your analytics will show that when LMS usage increases, so too do competence levels.
The more competencies your employees have, the more value they add to your organization as your “greatest assets.”
Level 5: What’s the ROI of your learning?
This is where you can really justify the spend on your LMS.
Which big-picture metrics did you want to impact upon when you implemented your LMS? Do you think you’ve made the impact you wanted?
Importantly, has implementing your LMS left you in a better position than when you started? Has more learning led to more competence, and has more competence led to tangible performance improvement?
While you could invest in expensive correlation analysis or econometric modelling to prove cause and effect, the Kirkpatrick model offers a more practical approach. It lets you plot the progression from engagement to business impact. Your learners liked the LMS → they used it → they applied their new skills → the application had the desired impact → it provided return on investment. When broken down into these five stages, it becomes much easier to see the link between your LMS and the real impact on your organization.
It’s time to start measuring
Ideally, you’ll already have years of rich data behind you, all broken down and ready for analysis. But the reality is that you likely have a patchwork of data tied up in spreadsheets, pulled from different systems and updated inconsistently.
But in the absence of that, the best time to start measuring the success of your LMS is right now. Whether you’re thinking about a new LMS or reflecting on the success of your established system, now is the time to decide what to measure, how and when to measure it and what your measurable goals will look like. It is only through measurement that you can prove that your LMS is a success; not just for the learning team, but for the organisation as a whole.
