How to Measure Training Effectiveness in Libraries and Nonprofits

You plan a training session, organize content, invite learners, and deliver the goods—in person or online. Then what?

If you just count who completed the training, you’ll only see a small part of a much larger picture—and not the most important part. The real question to ask is:

Did people learn? 

Knowing how to measure training effectiveness helps ensure training meets its objectives. In this article, we’ll look at why to measure, what to measure, and how to do it.

Why You Should Measure Training Effectiveness

The simple answer is, you measure training to know whether it’s working. But there’s more to it than that.

Training supports mission-critical goals. It helps new staff get up to speed, volunteers feel confident, teams provide better service, follow policies, handle sensitive situations, and adapt when things change.

In other words, training can shape the quality of your organization’s work every day.

That’s why it’s important to measure training effectiveness. Digging deeper helps you answer questions like:

• Are people learning what they need to know?
• Can they apply the training in real situations?
• Where do people still need help?
• Is this training worth the time and money we’re putting into it?
• Do I need to become a better trainer?

In an AIHR article, Erik van Vulpen explains how measuring training “is critical for showing tangible results, making continuous improvements, and justifying training investments.”

Training Effectiveness Metrics: What You Should Measure?

It’s easy to track things like:

• Completion rates
• Learner satisfaction
• Associated costs

But as Christina Kefala points out in an eLearning Industry article, “Completions, satisfaction, and cost are all input metrics. They describe what went into training. They say nothing about what came out.”

So while tracking these metrics can be helpful, you also want to focus on output metrics—like those outlined in Kirkpatrick’s training evaluation model. It looks at training across four levels:

1. Reaction: Did learners complete the course and find the training useful, relevant, and engaging?
2. Learning: Did they gain knowledge, skills, or confidence?
3. Behavior: Are they applying what they learned on the job? Has the training influenced their performance and general attitude?
4. Results: Did the training contribute to a meaningful organizational outcome?

You don’t need to use the model formally to benefit from its underlying idea. The main takeaway is to ask questions that will help you see training outcomes more clearly. 

How to Measure Training Effectiveness—Step-by-Step

1. Start With a Clear Training Outcome

Before you can choose what to measure, you need to know what you want to achieve.

Identifying the desired training outcome is the first step in backward design, a popular instructional planning model. When you know what you want to achieve, it’s easier to figure out how to get there. And once you know how to get there, it’s easier to figure out how to measure progress.

Thinking about what you want people to know or do after training can help you drill down to a specific outcome. 

For example, instead of:

“Train volunteers on safety.” 

Try:

“After training, volunteers can identify three safety risks, follow the reporting process, and explain when to contact a supervisor.” 

Now you have specific actions to measure—making training easier to build, easier to assess, and easier to improve.

If you’d like a little more guidance on backward design and writing clear objectives, check out our free Training Design Workbook.

2. Identify the Skill or Knowledge Gap

If you’re still struggling to define a specific outcome, look at what people are doing now, where you want them to be, and ask yourself:

What do people need to get there?

 The gap might be:

• Knowledge-based - people need to learn a new policy, system, or process
• Skill-based - people understand a concept but need help and/or practice applying it
• Confidence-based - people know what to do, but hesitate in real-life situations
• Consistency-based - some are doing the task well, while others are not, or are using different steps

This step is important because training time is often limited. If people know the basics, they may only need a refresher or a job aid rather than a full training session. Knowing where the gap is can help you define the scope of the training.

3. Choose Training Metrics That Match the Goal

Once you know the outcome and the gap, choose metrics that align with them. 

The key is to avoid measuring just because the data is available. Instead, measure what helps you answer key questions and make decisions. 

Other training goals may require a different focus. If you are looking to improve customer service, you’ll likely want to examine behavior and results more closely. 

You might use metrics like:

• Manager observations
• Performance after training
• Patron/client feedback

The right metric depends on the goal. A quiz can show whether someone remembers a policy. It might not show whether they can calmly apply the policy during a difficult interaction.

4. Collect Baseline Data Before Training

A baseline is your starting point. It shows what people know or can do before training begins. 

You might be tempted to skip this step, but then you risk covering the wrong information and wasting people’s time. Using a quick survey, quiz, or observation to collect baseline data also gives you something to compare later. 

Without baseline data, you may know how people performed after training, but you won’t know how much their performance changed.

For example, if staff score 85% on a post-training quiz, that sounds pretty good. But if staff scored 82% beforehand, the training didn’t add much. If they had scored 45%, that’s a different story!

5. Measure During and After Training

Training measurement works best when it doesn’t all happen at the end.

During training, you can use quick knowledge checks, reflection questions, practice activities, or discussion prompts. These help learners see how they’re doing while there’s still time to adjust. They also help trainers spot confusing sections before the same problem repeats.

After training, you can measure what learners gained and what they can apply. You might use post-tests, scenario responses, surveys, or manager feedback.

You can collect some data immediately. Other data needs time.

If the training is about a new procedure, you may be able to check understanding right away. But if the goal is behavior change—like better volunteer communication—you may need to follow up after a few weeks.

6. Compare the Data and Look for Patterns

Data becomes useful when you compare it; this is why it’s important to collect information before and after training. 

Look at what changed between the baseline and the follow-up. Then look for patterns:

• Did most learners improve? Great—your training is working!
• Did everyone struggle with the same question or step? This pattern may indicate something isn’t working with the training and needs to be fixed.
• Did one group of learners need extra support? Look at the commonalities—some learners may have already been well-versed in the subject.
• Did learners understand the concept but struggle to apply it? Again, this is an opportunity to look at the training and see how you can improve it.

If you’re not seeing any particular patterns, you can always ask learners for feedback. They can provide useful qualitative data, like maybe the training itself was fine, but they ran into technical trouble!

7. Keep Training—and Measuring! 

It’s okay to start small, with one piece of training, one objective, one department.

Once you get used to the practice of measuring, you can gradually apply it to other training efforts so you can see what’s working in the moment and over time.

As Jay Shani with AIHR says, “...measuring training effectiveness should be a continuous process, enabling your employees to feel supported and empowered at work.” 

Training Measurement Tools for Libraries and Nonprofits

You don’t need a complicated system to measure training effectiveness. A few basic tools can help you collect useful data without adding too much extra work.

Pre- and Post-Tests

Pre- and post-tests are one of the simplest ways to measure learning. 

Many organizations use post-tests, but they are much more useful when combined with a pre-test. Starting with a pre-test gives you the all-important baseline data mentioned earlier. The pre-test shows what learners know before training. The post-test shows what they know afterward. Then you can compare the two to see what’s changed.

Formative and Summative Assessments

Assessments can sound intimidating, but they’re simply ways to gather evidence of learning. 

According to UMass-Dartmouth, a strong assessment asks:

• “What am I asking students to learn or do?"
• "What kind of evidence will show that learning?"
• "How will students get guidance while they are still able to improve?”

There are two main categories of assessment:

• Formative assessment happens during training. (It’s helpful to think of the base word “form”—it’s helping to form or inform the learning). Activities like knowledge checks, role-playing, and discussions help learners practice, let them know how they’re doing, and give you a chance to offer feedback.
• Summative assessment happens at the end of training. It helps you evaluate whether the training met its goal. (Think of the base word, “sum”—it’s the sum of all learning). At this point, it’s too late to adjust the training in real time, but you can use the information to see what went well and what may need to change before the next round of training.

Most people don’t think of it this way, but assessments are meant to test the training, not the learners. They help you ensure training meets its objectives.

Surveys That Go Beyond “Did You Like It?”

Learner feedback still matters. People are more likely to complete training when it feels relevant, clear, and respectful of their time. 

Satisfaction surveys are most useful when they go beyond “Did you like this training?”

ICMI  notes, “Learner likes and dislikes are interesting, but our goal is a sound learning event that results in learning transfer.” They recommend asking questions like, “What do you think the impact of what you learned today will be when you return to work?”

The trick is to ask questions that give you actionable information.

Interviews and Observations

Some of the best training data comes from watching what happens after the training to see if learners have retained and applied the skills or information in their work.

Informal conversations are also useful. After a training, you can ask questions such as:

• What information (or tools) have you used from training?
• Did you feel ready to use what you learned?
• What would have made the training more useful?

This approach is especially helpful for skills that are hard to assess with a quiz, such as communication or trauma-informed support.

LMS Reports and Analytics

Using a learning management system (LMS) can make training measurement much easier.

At a basic level, an LMS can help you track completion, progress, quiz scores, and engagement. A stronger platform (like Niche Academy 😁) can also help you organize training by role, assign refreshers, pull reports for leadership, and more.

The goal is to make training easier to manage, easier to improve, and easier to explain to the people who approve the budget.

How to Use Training Data to Improve Future Learning

Your measurements are only helpful if you do something with them; otherwise, you’re wasting time and creating more work by collecting information you’re not going to use. 

So how can you use the data? Here are some ideas: 

• See if training is working. This option is the most basic use—if people are meeting the outcomes you set at the beginning of training, then your training is working—well done! Now you can use the training and measurement model to create new training to meet new skills gaps.
• Share with others. If the training is working well, let others know. Executives and boards are likely involved in approving the training budget, so sharing data demonstrates ROI and sets the stage to ask for more funding to expand the training program in the future!
• See where training needs improvement. When training doesn’t seem to be working, good data will let you drill down and answer important questions. You might check whether all learners missed the same thing. If so, there’s probably a problem with the training. Understanding this can help you gather more feedback about what isn’t working so you can fix it.
• See where learners need more support. If only a few learners appear to be struggling, this could indicate the training itself is working, but some individuals may need more help, support, or resources. To learn more about how training data can reveal what learners need, check out our democast, Using Training Data to Support Learners.

Think of the process as a circle. You start with the training itself, measure it, and then use the data. Effective data analysis brings you back to the start, so you can continue to improve training.

Measuring Training Helps You Make Learning Matter

Training takes effort. Measuring training helps make sure that effort leads somewhere useful. 

While measuring training is essential, it doesn’t have to be a heavy lift. When you look beyond attendance and completion, you can see whether people are learning, whether they can apply what they learned, and whether the training supports the work your organization cares about most.

A training platform like Niche Academy makes the process easier by bringing training development, measurement, analytics, and reports together in one place. I invite you to start a free trial or contact us to learn more.