She enjoys combining in-depth research with expert knowledge of the industry. If you have eLearning insights that you’d like to share, please get in touch.
Natalie TaylorAs a certified iSpring eLearning consultant, I help our clients assess the pros and cons of their training strategies and launch bespoke online training solutions to boost training effectiveness while saving my clients both time and money.
Many businesses train their staff to boost their productivity and work performance. But how can you tell if it’s successful? And how do you identify what needs tweaking in your courses to make them more effective? The ideal approach is to start evaluating your training programs.
This piece will guide you through understanding the training evaluation process, its critical role, and the benefits it can bring to your company. It will also shed light on the best evaluation models and tools you can use to assess your program’s effectiveness.
Training program evaluation is the regular analysis of how efficient and effective training programs are. This process includes collecting participant feedback, assessing job performance improvements post-training, aligning with corporate strategic goals, and much more.
It allows companies to make sure that the time and money spent actually help employees learn and grow. It shows what worked well and what didn’t, so future training can be improved. Finally, it checks if the training helps people do their jobs better, which is the main goal.
Evaluating training effectiveness obviously requires additional resources, but the advantages far exceed the challenges. In short, it helps companies:
To evaluate training programs more systematically, you can use specific training evaluation models. There’s a wide selection of them, with each providing a unique approach to course assessment. We’ve identified 5 models that stand out as being the most popular and trusted over the years.
One of the oldest and, without a doubt, the most famous is The Kirkpatrick Model. Even now, after more than sixty years since it was developed, it’s still widely used.
This model suggests evaluating training programs across four levels: Reaction, Learning, Impact, and Results. Let’s delve into each level.
Level 1: Reaction
Once learners complete your course, assess their reactions. Have them fill out a survey, asking questions such as:
For more nuanced feedback, consider a Likert scale survey. Unlike binary questions, a Likert scale survey offers shades of opinion beyond simple yes/no answers, which can enrich your evaluation insights.
Level 2: Learning
Measure how much was learned in the course. For example, consider using online quizzes to gauge the knowledge and skills your learners gained or missed out on during training. For even more accurate insights, some companies opt for pre-quizzes. This approach lets you clearly understand what your learners knew before starting the course and what they have learned by completing it.
Level 3: Behavior
Observe any changes in an employee’s behavior following the training. The most effective way to do this is to use 360-degree feedback — gather evaluations from the employee’s colleagues, supervisors, and subordinates before and after the course. This comparison reveals the training’s impact on behavior.
Level 4: Results
This level is the cornerstone of training evaluation. After all, better results are the primary goal of corporate training. Assess the impact of your course on the business by assessing subsequent improvements in quality, efficiency, productivity, and customer satisfaction.
Although The Kirkpatrick Model is extremely effective, it has some limitations we need to mention:
Don Kirkpatrick himself recognized these issues and suggested a better way of using his method: begin with the end in mind. By starting from the desired outcomes and working backward through the four levels during the design phase, you can tailor your training program to achieve your specific goals.
You can think of the Phillips ROI model as an upgrade to Kirkpatrick’s framework. It mirrors the levels of The Kirkpatrick Model, with a crucial addition at the end — return on investment (ROI). Unlike The Kirkpatrick Model, which focuses on Return on Expectations (ROE), the ROI model can actually let you know if it was the right decision to invest in a training program.
How to measure ROI with the Phillips model:
Collect data on the status before, during, and after the training to assess its impact on your company’s earnings, productivity, and performance. Then, compare the training cost to the benefits it provides. If the benefits to the company’s bottom line surpass the costs, you’re on the right track. If they don’t, pinpoint which level or levels of training evaluation fell short and refine your training approach.
Building on the Kirkpatrick Model, Roger Kaufman introduced a five-level framework. He split the first level into two parts, combined Kirkpatrick’s second and third levels into ‘micro’ levels, and introduced a fifth level to assess outcomes for both customers and society.
Here’s how to use Kaufman’s five levels of evaluation:
Level 1a: Input
Track the resources, like time and money, that were invested in your training program.
Level 1b: Process
Gauge how participants felt about the course.
Level 2: Acquisition
To assess the specific benefits of your training, check if it meets the goals for individual learners or small groups. This involves finding out whether your learners have gained new knowledge and skills.
Level 3: Application
Assess learners’ ability to apply new knowledge and skills to their work.
Level 4: Organizational payoffs
Measure payoffs for your company as a whole. A payoff can be an improvement in employee performance, a reduction in costs, or increased profits.
Level 5: Societal Outcomes
At the final level, you are to evaluate the impact that your course has on what Kaufman calls ‘mega-level clients.’ By these, he means business clients or society.
Kaufman’s framework isn’t entirely applicable to real-world scenarios. Measuring how much impact your training has on society is often too costly, complex, and impractical. However, Kaufman introduced some valuable concepts, such as dividing the first level into two and evaluating the content you deliver and the resources you invest separately. Some of his levels can serve as enhancements to your basic model.
CIRO stands for Context, Input, Reaction, and Output. This model is designed for evaluating management training. So, if you want to assess management courses, it is an ideal choice.
Stage 1: Content
Assess your company’s current situation. Identify all the factors that could influence the training results. At this stage, also pinpoint where your organization falls short in performance. As a result, you’ll have a list of needs that should be organized according to the following three levels:
The ultimate objective
The ultimate objective is the elimination of organizational shortcomings, such as poor customer service, low productivity, or low profit.
Intermediate objectives
These are the steps needed to reach the ultimate objective. They often involve changes in employee behavior.
Immediate objectivesAs immediate objectives can help change employees’ behavior, they usually involve the acquisition of new skills and knowledge from training, or shifts in their attitudes.
Stage 2: Input
At this stage, your aim is to pinpoint the optimal training intervention. Explore all possible methods and techniques for training. Also, think about how you will design, manage, and deliver your course to your learners. Assess your company’s resources to figure out the most effective way to use them to achieve your objectives.
Stage 3: Reaction
At this stage, collect feedback from your learners about the course. Focus on three key areas:
Your goal isn’t just to find out if they liked or disliked the course, but also to gather insights on any changes they suggest for the training program. Note their recommendations for future improvements.
Stage 4: Output
In this stage, it’s time to showcase the outcomes of the training through four distinct levels of measurement:
Select the level that aligns with your evaluation’s objective and the resources you have.
Unlike other models, Anderson’s Value of Learning Model takes a broader perspective, concentrating on a company’s overall learning strategy instead of just a specific training program. It consists of three stages that help to identify the most suitable learning strategy for your organization’s needs.
Stage 1
Determine if the existing learning programs align with your company’s strategic priorities. Going back to our example, suppose that the strategic goal is to boost sales and strengthen market position. Does the training for salespeople target these objectives? The answer is yes.
Stage 2
Evaluate the impact of learning on strategic outcomes. At this point, our company uses different measures to assess how much effect the training has had on accomplishing the primary goals. By analyzing the data, we find that the program boosted sales numbers. However, it didn’t result in a larger market share for the company. Moreover, as time passed, we observed a decline in customer numbers because the waiting time was too long.
Stage 3
Select the most relevant approaches for your company. The choice of approach depends on stakeholders’ goals and values. Anderson introduced four categories of measurement:
Below is a table designed to help you identify the best approach for your organization.
You should choose a category that’s relevant to your situation and establish an approach that will help fulfill your organizational needs.
Training evaluation tools are what you use to assess training programs. They typically fall into four categories: questionnaires, interviews, focus groups, and observations. We’ll also include an additional one — LMS reporting. For the most thorough and accurate assessment, it’s common to use these training evaluation methods together. Now, let’s look at each one in greater detail.
Questionnaires stand as the most frequently used method for training evaluation. They consist of a set of questions designed to gather valuable insights from participants. This tool is great for assessing learners’ reactions after a program.
