When it comes to learning management systems, “success” means different things to different organizations. Are you looking to boost learner engagement? Reach that all-important 100% compliance target? Or are you looking beyond the LMS itself, and instead striving for a particular business metric, such as increased sales, better customer ratings or more repeat business?
Contrary to popular belief, “success” is not simply the launch of your LMS. Nor is it the fact that it works as expected, and it’s not even collecting a lot of data. LMS success is inextricably linked to your organization’s success. How is your LMS helping you drive real-world results? What impact does your learning have on your people, outcomes and productivity?
To help you structure your thinking around LMS success, we will take inspiration from the Kirkpatrick model of training evaluation (and Phillips’ additional fifth stage). As a quick recap for those of you who are unfamiliar with these levels, they are:
- Reaction – what do people think of your learning?
- Learning – how does your learning contribute to the acquisition of new knowledge, skills, behaviors and confidence?
- Application – are your people applying your learning on the job?
- Impact – is your learning having the desired impact?
- ROI – how does the cost of your learning program compare with the results?
These five stages can be used to assess the success of your own LMS, and to help you prove this success when it comes to reporting back to your stakeholders. The better you get at measuring the success of your LMS, the easier it will be to secure further investment in your learning program going forward.
Level 1: What do people think of your LMS?
The good news is that you have a nice easy start with the first level of measurement. What you want to find out here is “Do people like your LMS?”. Was it delivered on time? Was it on budget? Is the quality good enough?
These are simple questions to answer. Don’t get disheartened if the answer to any of these questions is “no” – it doesn’t render your entire project a failure! Realistically, LMS projects can’t or won’t always be delivered on time or budget.
If you wanted to take it a step further, you could conduct a survey to gather the opinions of learners, managers, administrators and stakeholders. This could sit on your LMS, or you could send out an email asking for initial feedback.
Level 2: Is your learning effective?
So your LMS is in place. But who is logging on? Are they enjoying the learning experience? Level 2 is about your users, their interactions with your LMS and whether or not it becomes part of their ongoing development. Some of the data you can collect to measure success here includes:
- The number of learners visiting in the first week vs the next four weeks
- The number of learners visiting your new LMS vs your old LMS
- Return visits vs learners only visiting once
- The length of visitors’ engagements (are people visiting your LMS for 30 seconds or 30 minutes at a time?)
You could also consider adding rating systems within the LMS to help gauge opinions quickly. For instance, once a week or month you could add a question to your LMS asking questions like “How easy do you find this LMS to use?” or “How has your use of the LMS impacted the amount of learning you do?”. Simply clicking a response makes it quick and easy for learners to have their say, and you’ll discover how your learning is working.
Level 3: Is your learning being applied?
Now we move onto measuring the learning itself, rather than the system.
Here, the burden of proof is that the learning process has improved. The measures here need to relate to an increase in knowledge or skills acquired per employee for the same investment in time or money, or over the same time period – think of it as an ROI for learning.
Your LMS analytics will play a vital role in this stage of measurement. You will need to factor in all the learning within your organization and show an overall increase in learning per unit of investment – e.g. for every additional $X spent on your LMS, learning has increased by X%.
Application of learning can include the following considerations:
- If your LMS enables mobile learning, what is the impact of enabling learning during otherwise less productive times outside the workplace? What is the value of this additional “useful time?”
- What is the value of the learning decisions you have made? For example, scrapping programs with poor satisfaction levels, switching to a better vendor or replacing unused content with fresh content.
- How has your LMS saved time and money spent traveling to and from face-to-face learning sessions? Are people spending more time learning and applying that learning than they were previously?
Level 4: Is your learning having the desired impact?
Level 4 is where things start to get really interesting.
Looking at the impact of your learning means also measuring the performance of your people. Ideally, your LMS will be integrated with a performance management system such as Totara Perform. This will enable you to link learning to employees’ competency frameworks. Over time, as people learn more and apply their new skills and knowledge in the workplace, you should see the number and level of competencies in your organization grow. Ideally, your analytics will show that when LMS usage increases, so too do competence levels.
The more competencies your employees have, the more value they add to your organization as your “greatest assets.” Assets cost money… and suddenly, you’re tantalizingly close to your ROI!
Level 5: What’s the ROI of your learning?
This is where you can really justify the spend on your LMS.
Which big-picture metrics did you want to impact upon when you implemented your LMS? Do you think you’ve made the impact you wanted?
Importantly, has implementing your LMS left you in a better position than when you started? Has more learning led to more competence, and has more competence led to tangible performance improvement? Answering these questions requires a leap of faith, which may lead to questions such as “How can you take credit for boosting sales/improving employee engagement/increasing customer ratings?”.
While you could invest in expensive correlation analysis or econometric modeling to prove cause and effect, instead you can use the steps set out in this model to plot the “baby steps” comprising your leap of faith. People liked your LMS, so they used it, so they applied their new skills, so it had the desired impact, so it provided return on investment. When broken down into these five stages, it becomes much easier to see the link between your LMS and the real impact on your organization.
It’s time to start measuring
Ideally you’ll already have years of rich data behind you; all broken down and ready for analysis. But the likelihood is that you have a patchwork of data tied up in spreadsheets, pulled from different systems and sporadically updated.
But in the absence of that, the best time to start measuring the success of your LMS is right now. Whether you’re thinking about a new LMS or reflecting on the success of your established system, now is the time to decide what to measure, how and when to measure it and what your measurable goals will look like. It is only through measurement that you can prove that your LMS is a success; not just for the learning team, but for the organization as a whole.
