Best Methods to Capture and Use Data to Develop Your Learning

With so much evolution and advancement in technology today, we now have more ways than ever for capturing data associated with the delivery, consumption, and impact of learning in organisations. Yet, according to Brandon Hall, fewer than 16% of organisations are able to effectively identify and track a series of metrics, including participation, satisfaction, knowledge transfer, behaviour change, and business impact for any of their learning. One reason for this shortfall is that the idea of identifying, collecting, and analysing all of these potential data points can be overwhelming to even the most mature organisations. One potential solution is to focus those efforts on a subset of data, providing insights that align with the most important outcomes. This creates a “measurement brand” for your organisation. Continue reading to learn the steps to creating your learning measurement brand.

The Good News: Data is Everywhere

The ability to capture data has always been at the core of the measurement challenge. Thankfully, over the past decade this issue has been addressed with new tools and technologies that improve access to data. These include:

• Learning Management Systems (LMS)
• Learner Experience Platforms (LXP)
• Experience APIs (xAPI)
• Google Analytics
• Learning Content Management Systems (LCMS)
• Feedback surveys
• Enterprise BI Tools
• Location and Device tracking

…and other similar tools and processes. Combined with a comprehensive learning analytics strategy, the data from these tools can be used to measure training consumption, assessment performance, social behaviours, confidence and more.

Is it possible to have too much of a good thing?

Ideally, each organisation would capture every bit of data available, analyse for trends, create beautiful graphs and charts to present the findings, and provide regular updates to all the various stakeholders. Unfortunately, for most organisations, it’s not possible to do it all. Different stakeholders will have differing priorities for this data. The Compliance Team may care most about consumption data like starts, completions and time to completion; The Sales Team may care most about starting and ending confidence; The Quality Team may care most about learner’s ability to recall and apply knowledge in a course. Even if you had unlimited resources, providing every data point to every department may leave them overwhelmed. Prioritising your measurement goals, as part of your overall learning measurement strategy, and taking a point of view on what’s most important to your organisation will guide where you put your energy and resources when capturing learner data. This approach will ensure that you successfully deliver on your measurement strategy.

Learning Brand Positioning

So, what’s your measurement brand? As part of a broader learning measurement strategy, here are four potential brand positions to consider:

1. Learner-Centered: Meeting the needs of the learner is your driving force. Do you develop confidence? Did they enjoy the experience? Were they able to apply the material?
   
   
2. Business Outcomes: Delivering an impact on the business. Do those who go through a learning solution perform better?
   
   
3. Behavioural: Do our learning solutions change behaviour? Were we able to get learners to start, stop or continue activities? Can we assign value to those changes?
   
   
4. Consumption: Are people engaging with the learning? How long do they spend? Are they completing it? Do we hit penetration levels, e.g. 80% of potential audience?
   
   

Once you establish the learning brand position for stakeholders, you’ll be able to identify appropriate approaches to capture and measure learner data from your training programmes. How does this “data” get there?

It Starts Long Before Data Collection

So let’s say you work with each of your key stakeholders and nail down their priorities. Is that enough? Probably not. The development of a learning measurement strategy involves more than just creating a list of organisational learning priorities. It’s more elaborate because capturing learner data, and subsequently your measurement strategy, will affect not just the data you collect, but also the process of developing learning – from inception to delivery. Here are some examples:

• If you want to track how learners apply knowledge, you might use xAPI statements to collect the number of correct and incorrect responses to questions that require application of knowledge.
  
  
• If you want to measure how a learner’s confidence changes from the beginning of learning experience to the end, you need to include questions about confidence at the beginning and end of each course.
  
  
• If you want to focus on consumption statistics like # of learners who started a course, finished a course and time to completion, you’ll need to pull data from your LMS.
  
  
• If you want to know if training changed behaviour back on the job, you’ll want to include a manager or 180 assessment before and after training.
  
  

As these examples show, your learning measurement strategy drives your overall approach to learning design and development.

A good set of principles to remember come from Douglas Hubbard, the creator of Applied Information Economics, offering us some insight into collecting and measuring data:

• Rule 1: If it matters, it can be measured
• Rule 2: You have more data than you think
• Rule 3: You need less data than you think

The key to building an effective measurement strategy, therefore, is to set realistic expectations, build relevant data points, and capture and measure specific data from your learning environment. Then, you can measure and re-assess your learning brand and, if required, make informed decisions on repositioning that brand.

Time to Own Your Brand

Designing and implementing a learning measurement strategy can feel overwhelming and unattainable, but it doesn’t have to be. Three keys to keep in mind are:

1. Focus your efforts based on the needs of stakeholders
   
   
2. Plan your data capture strategy early in the project (before you build it!)
   
   
3. Be consistent - once you’ve decided your approach or approaches, implement it on every project. There’s value in seeing trends over time.
   
   

To learn more about all the different data you might consider collecting, and how to measure training data to assess the effectiveness of your learning brand, download our Measurement Guide now.

‍