Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs PDF Free Download
1 / 67
/67
100%
How to Measure and
Demonstrate the Impact of
Your Training Programs
Prove and Improve
By Asha Pandey
Chief Learning Strategist, EI Design
1
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Asha Pandey is the Founder
and Chief Learning Strategist at
EI Design. She heads the
organization’s L&D Advisory
and L&D Consulting practice.
She also directs and supervises
the organization’s Business
Development team and plays a
key role in Innovation,
Research, and Exploration
mandates. With 2 decades of
experience, she helps EI Design
customers ensure that their
investments in training lead to
performance gain and business
results.
A regular writer for various
global Learning forums, Asha is
one of the most recognized
thought leaders in the industry.
In 2022, she was recognized as
a thought leader by eLearning
Industry in the list honoring 30
eLearning pioneers. She also
shares her insights regularly on
a range of topics on her blog at
https://www.eidesign.net/insig
hts/.
About the Author
2
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Preface
Organizations put significant time, effort, and funds into creating and
delivering training programs. However, they are highly challenged in
their ability to demonstrate the impact of the training investments.
While tremendous effort goes in during the Training Needs Analysis
phase, majority of the initiatives do not focus on identifying the
approach (data, metrics, and frameworks) that would help the L&D
teams ascertain the impact of training on learners and, more
specifically, business.
As a result, they are not able to demonstrate the value, impact, or ROI
on the training spend. This also means that there are no insights that
can help organizations update, augment, or improve the initiative –
till it meets the required mandate.
3
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Preface
While there are several reasons why this measurement is so
challenging, here are the top 2:
•The trainings are not designed with the right outcomes in mind.
They tend to focus on the narrow perspective of learning
acquisition outcome and not on application.
•Very often, there is a lack of alignment between the gains that
business would like to see vs how the L&D team envisions the
same and designs the training programs.
In this eBook, I show you how L&D teams can align their goals
better to business needs to ensure that the training is focused on
the right performance outcomes and will be able to positively
impact the identified business KPIs.
Additionally, I share unique frameworks created by EI Design that
we offer to help L&D teams demonstrate the impact of training –
on learners and the identified business KPIs.
Read on!
4
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Table of Contents
1Why You Must Develop the Ability to Evaluate
Training Effectiveness and Its Impact
2What Are the Key Challenges in Evaluating Training
Effectiveness and its Impact
3How to Re-orient Your Approach to Training Impact
Measurement –Leveraging a 4-step Toolkit
4Key Models and EI Design’s Methodology to Ascertain
Training Effectiveness and Its Impact
5Resources
5
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
When employees train often and learn
effectively, the results usually show up in
organizational performance. There’s also a
mountain of research-based evidence that
links effective training to exceptional
performance.
Yet, surprisingly, a Brandon Hall Group
research brief indicated that 9% of
organizations surveyed did not see a need to
link training-induced behavioral changes to
business performance. Because many
organizations have entrenched training as
standard operating practices, they, therefore,
lack proper metrics to measure training and its
contribution to positive business outcomes.
Introduction
7
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Here are some compelling reasons for evaluating
training effectiveness and its impact:
Justify training spending and secure further budgets:
Shareholders, Boards of Directors, and Executive teams hold
business leaders accountable for their spending plans. The
better the justification for those spends, the more likely
those funds continue to flow. If L&D managers wish to
secure ongoing funding for their learning initiatives, they
must evaluate the effectiveness of training program
spending and justify the ongoing need for those programs.
Measure the impact of training programs: The primary goal
of any training plan must be to support business objectives.
It is only by evaluating the impact of training programs that
business leaders can get objective metrics on how well
those programs work to support business objectives.
Improve the impact of training programs: When training
aligns with business objectives, an in-depth training
evaluation is the only way that learning leaders can identify
important KPIs to make training more relevant. By
measuring the results of a program, L&D teams can evaluate
those results against pre-determined KPIs and enhance or
improve training plans based on that assessment.
Even though L&D leaders understand the significance
of measuring and demonstrating the training’s impact,
they face several challenges in being able to
accomplish this goal. In the next section, I list these
challenges and define the approach that should be
taken to mitigate them.
8
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs 10
Organizations make steady investments in
training and development programs every year.
As I had outlined in the previous section, today—
more than ever—there is a need to assess the
business impact of these trainings. These
insights can help organizations determine which
training and development programs are aligned
to the business goals and update/modify the
ones that aren’t.
Unfortunately, aspirations to evaluate the
effectiveness of training program
implementations don’t necessarily equate to
concrete and measurable success in determining
training effectiveness. That’s not because those
“aspirations” are flawed! It’s typically because of
inherent organizational challenges that stymie
the fulfillment of those aspirations.
Introduction
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
What are the key challenges in evaluating training
effectiveness and its impact?
You are not alone in facing this challenge. Take a look at the research
from Brandon Hall Group’s Learning Measurement Study.
Reasons Why Companies Do Not Measure Learning Effectiveness and
Its Impact
We don’t have the
proper metrics
We don’t have the
technology to
support
It is too difficult to
assess
42% We don’t have the
time/staff
It is too difficult to
link learning to
outcomes
We don’t see a
need
42%
36% 35%
31% 9%
Source: BHG, Learning Measurement Study
11
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Here are some of the critical challenges that L&D leaders
encounter while embarking on initiatives for evaluating training
effectiveness:
Lack of the
right
framework and
methodology
Organizations that lack employee performance
metrics, are unable to say, with any degree of
certainty, whether training helps or hinders
employee performance. For instance, how do
you link specific training outcomes to even
more specific performance objectives? And how
do you determine if training is helping to
develop your in-house talent pools, or if
employees are successfully applying newly
acquired skills (learned through training) as
performance aids on the job?
Inability to link
training to talent
development,
performance, and
outcomes
When preparing to evaluate the effectiveness
of training program results, most evaluators
begin with a training outcomes measurement
approach. That strategy seldom works because
it doesn’t consider what happened prior, or
subsequent, to training. What’s required is a
methodology that reviews the training
framework across the organization, beginning
with a Training Needs Analysis (TNA) and
culminating with a determination of whether
training meets all KPIs and delivers a justifiable
return on investment (ROI).
12
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Training technology, such as Learning
Management Systems (LMS) and Learning
Content Management Systems (LCMS), have
embedded tools that provide a lot of analytic
data for evaluating training effectiveness.
Unfortunately, many organizations either lack
the right tools and technologies to collect such
data or are incapable of using available tools to
collect and objectively analyze that data.
Many organizations lack the capability or
capacity to link organizational performance
with specific training-related outcomes. This
lack of abilities is reflected in the ineffective
evaluation of training results.
Lack of the
right tools and
technology to
collect data
Lack of capacity
inhouse
13
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
As a result, what happens?
•The TNA phase focuses on the training’s ability to meet the desired
learning outcomes; As an extension, the training impact is
measured largely through L&D metrics.
•Change in thinking or behavior is deemed to be difficult to measure
and may not be attempted.
•Often, the business KPIs are not identified upfront - If they are,
there is no easy way to obtain analytics that can measure the
business impact.
•Even if L&D teams know what should be done, they often do not
have the resources (team and tools) to collate the additional data,
analyze it, and draw actionable insights.
Because of these challenges, measurement either never gets
attempted or takes an inordinately long time. By the time the insights
come in, it may be too late for the business to apply them.
14
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
What should be the way forward to measure the
business impact of your training and development
programs?
The first correction needs to be done at the beginning of the exercise,
that is, during the Training Needs Analysis (TNA) phase. Typically, at
this stage, the metrics that will be used to determine if the training
effectiveness and impact is established.
L&D teams typically look at the following L&D metrics:
1. The number of training registrations and completion rates.
2. Learner reaction/feedback on the training/the trainer.
3. Assessment scores (to assess the training effectiveness).
15
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs 16
However, the crucial missing links are:
Learner Performance Gain Metrics –How will the
change/gain in learner’s performance be measured?
Business Metrics –How will the business see a specific gain
that will influence/impact the business goals, and how
should this be measured?
The correct way forward is by combining the evaluation through L&D
metrics, Learner Performance Gain as well as Business metrics. The
focus on the L&D metrics alone is inadequate in helping you get the
impact on business. Only when you couple them, you will be able to
determine the impact of the training on business.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Take a look at this example to understand the value you get as you
couple both L&D and Business metrics.
17
An organization invests in an enterprise-
wide CRM tool for their Sales and
Marketing teams. The new tool is planned
to replace the multiple legacy tools and
Excel-based trackers over a short period
of 6 months.
Background
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Training need: The entire Sales and Marketing team needs to
undergo training for the new CRM tool. The L&D team maps the
training to the following profiles:
•Set 1: Sales Executives, Sales Managers, and Head of
Sales - Maps to acquiring three levels of tool proficiency.
•Set 2: Marketing team.
•Set 3: CEO and COO - Dashboard review - with a focus on
analysis and actionable insights.
The L&D team plans the training modules to achieve the required
proficiency levels, across the three audience profiles. The
assessment scores are the selected indicators to validate the
proficiency gain.
18
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs 19
What was measured?
•The completion rates are
deemed as metrics for
success of the training
program.
•Additionally, the
assessment scores are the
used as the other
indicator - to validate the
proficiency gain.
What was missing?
The Learner Performance Gain
metrics: The effective
application of the acquired
learning is a pre-cursor to
gain for business. However,
tracking this aspect was
missed out.
The Business metrics: What
wasn’t discussed with the
business (Sales and Marketing
teams) was that once the team
members have the requisite
proficiency in the new tool,
what is the anticipated
business gain, and how should
it be measured.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
How can this gap be offset?
Go back and identify why this training investment was planned (to
address current challenges and achieve additional gains). Then,
shortlist a key factor – For instance, “Achieving higher efficiencies” that
can be measured as:
•Reduced time spent > Leverage to higher target.
•Better analysis > Improvement in conversion rates.
•Better pipeline management > Higher predictability in projections.
20
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
For instance, the Sales reps will save 1 hour/day once they move to the
new tool (instead of wading through multiple Excel trackers), and this
time could be used to step up their target by 12.5%.
Now, this impacts the business directly and is a great way to ascertain
the impact of the planned CRM training on business.
If this collaborative quantification (coupling of L&D metrics with
Learner Performance Gain and Business metrics) isn’t done upfront in
the project, it would be impossible to track and measure the business
impact of the training. This exercise needs a strong collaboration
between the L&D team and the Business team.
•During the LNA phase, the parameters that can help ascertain the
impact of the training on learner’s performance must be identified.
•During the TNA phase, the parameters that can help ascertain the
business impact of the training must be identified.
21
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
It is important to note that while identifying the
metrics for evaluation is the first and the most crucial
step in your ability to measure and demonstrate the
training’s impact, there are several other aspects that
need to be addressed that I outline in the next section.
Take a look at this data from Brandon Hall Group that highlights the
focus of how “High Performing Organizations” are leveraging the right
metrics to measure training effectiveness and impact.
Source: BHG, State of Learning Practices
22
Metrics used regularly or
consistently to measure learning
programs:
High Performing
Organizations Others
Completion rates
79% 63%
Achieving business outcomes
73% 28%
Compliance requirements
64% 72%
Employee engagement
62% 28%
Learning hours delivered
62% 45%
Certification/Recognition
56% 35%
Performance reviews
53% 27%
Coaching and mentoring outcomes
52% 22%
Managerial observations
49% 27%
Skills assessments
47% 27%
Turnover rates
30% 11%
Promotion rates
26% 10%
Behavior
assessments 22% 16%
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs 24
Before embracing frameworks/tools to
measure and demonstrate the impact of
training (on learners as well as business),
organizations need to re-orient many facets.
In this section, I share EI Design’s Toolkit that
will help you realign successfully to be able to
measure the training impact.
Introduction
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
EI Design’s
Toolkit
Step 9
Relook at the
definition of
learning
View the picture from
the learner’s view and
not be limited by the
organizational view
Identify measures
to sustain and
maximize the
impact
Identify relevant
and measurable
metrics (that are
not subjective)
4
3
2
1
25
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Sounds far-fetched?
However, this is the first and the most crucial
aspect in this endeavor. Far too often, the
objective of the training is merely learning
acquisition –with no room to enable the
learners to apply this on the job so that there
can be a demonstrable improvement.
This is what the concept of “learning” must
look like –wherein, learning should mean the
employees have:
•Acquired new information.
•Practiced it.
•Applied it.
Vital to this formula is coaching –to provide
feedback during the practice and application
of new knowledge.
Step 1 –
Relook at the
definition of
learning
26
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Even though the learner holds the key to the
successful impact of a training program, very
often the start point is the only organizational
need for training - rather than aligning it with the
learner need, asks, and expectations.
As employees are now more self-reliant in when
and how they work, they expect similar levels of
flexibility in workplace learning. The fact is that
each learner is different, and each learns and
assimilates learning content uniquely.
However, more often than not, organizations put
“learning” as the sole desired outcome - while
neglecting to integrate learner needs into their
strategies. Keeping learner needs and
preferences in mind allows L&D managers to:
•Leverage learners’ past knowledge and
experience.
•Understand how they prefer to learn and then
offer the most effective training.
Step 2 –View
the picture
from the
learner’s view
and not be
limited by the
organizational
view
27
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
In fact, the training effectiveness and its impact grows manifold –
once it matches the learner expectations and preferences, including:
Short, action-oriented, relevant content.
Relevant (preferably personalized) pathways that match
their career aspirations.
Room for learning within their workflow.
Learning aids and resources that are available on demand
- pull vs push.
28
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Ensure that you have provision for the
evaluation of both L&D metrics and business
metrics. Without the coupling of L&D metrics
and business metrics, it will be impossible to
track and measure the business impact. Strong
collaboration between the L&D team and the
business to identify the parameters that can
help ascertain the business impact.
Remember, successful L&D programs align to
both employee performance targets as well as
corporate strategy and tactics. A training may
seem effective based on basic L&D metrics like
utilization or reaction scores but only
legitimate KPIs identify actionable results. The
evaluation done against correct KPIs ensure
that L&D programs support and drive vital
enterprise strategy and tactics.
Step 3 –
Identify
relevant and
measurable
metrics (that
are not
subjective)
29
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
To arrive at the right metrics, collaborate with key business
stakeholders to:
Leverage L&D expertise to inform about the collaboration.
Understand what metrics business leaders care about.
Focus on objectively quantifiable metrics.
Determine the value of measuring a program’s effectiveness.
Report results honestly, even when the outcomes are poor.
To identify the right KPIs - quantify the Business metrics
and KPIs so that training effectiveness and its impact on
business can be measured.
•Confirm that the training can solve business problems.
•Align the L&D programs with the corporate strategy.
•Identify KPIs that map to results business leaders truly
care about.
30
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Some best practices that will help you successfully demonstrate the
training impact on Business KPIs are:
1. Identify the KPIs before developing the training solutions.
2. Ensure that the data used in the KPIs are available and reliable.
3. Avoid KPIs that can’t be measured - Ensure that training KPIs are
measurable (not subjective).
a. Use Leading KPIs in the early versions of L&D programs to
iterate and improve knowledge or application evaluations.
b. Then use Lagging KPIs a few weeks after the training and
these indicators should include business results like
increased sales or enhanced customer satisfaction.
4. Don't ignore negative results.
5. Wait long enough to measure the Lagging training KPIs.
31
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Here is a list of 5 strategies that will come in
handy in sustaining and, more importantly,
helping you maximize the impact.
Strategy #1 - Set the Right Foundation - Focus
on both L&D as well as Learner Performance
Gain and Business metrics
•During the Training Needs Analysis (TNA)
phase, focus on identifying both L&D as well
as Business metrics.
•During the Learner Needs Analysis (LNA)
phase, focus on identifying both L&D as well
as Learner Performance Gain metrics.
Step 4 –
Identify
measures to
sustain and
maximize the
impact
32
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Strategy #2 - Choose the right Evaluation Model to measure the
business impact
•Selecting the right evaluation model/methodology is
imperative in helping you measure the training impact.
•Correct evaluation or model/methodology helps provide
clarity on.
•What is being measured.
•How will it be measured.
•How will the evaluation outcomes be processed.
Strategy #3 - Spot-check periodically and showcase the impact
and the value to the business
Post the training delivery and completion by the learners,
determine its impact on the business.
•Use the Evaluation Model for the Business metrics and collate
and analyze the data on the impact.
•It is important to showcase this value to the business and
based on the analysis, you can reconfigure or tweak the
approach.
33
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Strategy #4 - Work with a holistic approach to drive employee
performance and behavioral transformation
Adopt an Ecosystem-based approach to:
•Engage and motivate the learners.
•Provide strategies that inspire sticky learning experiences.
•Offer practice zones to gain further expertise in a particular area.
•Apply the acquired learning.
•Strengthen learning in order to offset the “Forgetting Curve.”
•Influence and trigger behavioral transformation.
•Continue ongoing connects (post the successful completion of
the training).
34
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Strategy #5 - Sustain and maximize the business impact
Foster a sense of continuous learning through approaches like:
•Social Learning.
•Communities of practicing excellence.
•Self-Directed Learning.
•Curated content so that learners come back for more.
•Inclusive approach by encouraging active participation –through
User-Generated Content (UGC).
35
As I had mentioned in Section 2, the journey to
measure and demonstrate the training’s impact
requires re-orientation at multiple levels. Now that
you have these cues with you, in the next section, I
will show you how exactly we, at EI Design help our
customers in meeting this mandate successfully.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs 37
Training employees is becoming more
important as economies continue with major
shifts and adaptations to changing work
environments. In the next few years an
organization’s ability to effectively train and
upskill its employees will be essential to
maintain a competitive advantage. Those
organizations that cannot or will not invest in
employee training will become steppingstones
to success for those who commit to effective
employee training programs.
Introduction
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
How should you begin?
One of the most important aspects to assess the impact of training and
development on employee performance as well as assess the impact
of training and development on organizational performance is the
evaluation of its programs.
But training evaluation is often approached in this order - 1) after a
training solution is implemented and 2) from the bottom up –impact
on employee performance and then the organization, and 3) a decision
on what to do with the results of the evaluation.
However, effective evaluation of the impact of training and
development programs is most effective if it is approached in the
opposite order.
So, the first step in assessing the impact of training solutions should
begin before even a Training Needs Analysis (TNA) and leaders should
discuss and agree how to use the findings.
38
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
The most effective use of this data is as the impetus iterate training
solutions, honing them into highly effective programs. Commitment to
action before analysis, development, and implementation is key.
Organizational leaders should ask themselves these questions:
•What will be done with the data?
•Will data interpretation initiate the time and effort into changing
training programs?
•Is the plan to just report results or implement plans based on
those findings?
Action of this type, even asking and answering the questions requires
courage, accountability, and honesty from the leadership team.
It’s also vital to determine the value of evaluation for training
solutions and the amount of effort that should go into it. For small
programs where the intended and admitted impact might be small, it
may not be worth the effort it takes to evaluate training. But for
programs that consume moderate to large resourcing should be
evaluated for effectiveness.
39
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
1. Learning-Transfer Evaluation Model (LTEM): LTEM is an
eight-tiered assessment strategy: (1) Attendance, (2)
Activity, (3) Learner Perceptions, (4) Knowledge, (5)
Decision-Making Competence, (6) Task Competence, (7)
Transfer, and (8) Effects of Transfer. This model
postulates that the first two levels (Attendance and
Activity) do not provide enough data to determine
effectiveness but provide a foundation for each
succeeding level.
2. Kaufman’s five levels of evaluation: This model
includes five levels: (1) Input and Process, (2)
Acquisition, (3) Application, (4) Organizational Output,
and (5) Societal Outcomes. These Training metrics
span measurable results, starting from how effective
the training materials were perceived by the learners
(Input and Process) to the impact the training has at
the organizational level (Societal Outcomes).
3. The success case method: Instead of focusing on the
success levels of specific training programs, this
model measures how well an organization leverages
training. It attempts to consider factors outside a
training program that might influence employee
performance and business results.
Which training evaluation model would work?
There are several training evaluation models that you can pick from to
measure training effectiveness and impact. The list below briefly
summarizes a few well-known models.
40
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
4. Context, Input, Process, and Product Evaluation
model (CIPP): The CIPP model measures (1) the
impact of context, input, process, and product, (2)
sustainability, (3) effectiveness, and (4)
transportability. This model is iterative and is used
throughout a training solution instead of after.
5. Kirkpatrick’s levels of evaluation: This is perhaps the
most ubiquitous model and is the starting point of
many other models. It’s taxonomy measures (1)
learner reaction, (2) knowledge retention, (3)
behavior change, and (4) impact on the business.
3%
4%
8%
19%
20%
44%
63%
Anderson
Kaufman
Brinkerhoff Success Case
Learning Transfer Evaluation
Context, Input, Process, Product
Internally developed model
Kirkpatrick/Phillips Level 5
Source: BHG, State of Learning Practices
Which of the following learning measurement models do you use?
41
Many organizations use one or more of the models - sometimes
modifying them to fit their needs. Using proven models, like those
listed above, are useful because they’ve been tested over the years.
Take a look at the following insights from Brandon Hall Group on how
popular each of these models are. It is interesting to note that 44% of
organizations use a custom, internally developed model.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
EI Design’s unique framework to measure the
effectiveness and impact of training
Gauge learner
reaction
Begin with Training
Needs Analysis (TNA)
and Learner Needs
Analysis (LNA) Identify the employee
Training metrics –
L&D metrics, Learner
Performance Gain,
and Business metrics
Establish measures
to motivate learners Communicate the
relevance and value
of the training to the
learners
Determine the right
training format (Online,
Blended, or VILT) and
support through
Coaching/ Mentoring
Identify the right
learning strategy
Determine the gain by
the learners and
change in behavior
Assess the gain
for business
Close the loop
1
2
3
4
5
6
7
8
9
10
42
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Introducing LITMUS – EI Design’s unique framework to
measure the effectiveness and impact of training
LITMUS uses an adaptation of the popular Kirkpatrick Model of Training
Evaluation and we have established methodologies and frameworks to
capture learner reaction, learning retention, recall, and its application
on the job. We have created unique methodologies to determine the
training effectiveness and impact leading to a high ROE.
Our approach
The measurement of training impact needs an integrated approach that
should begin during the Training Needs Analysis or TNA phase. It then
successively builds right up to the determination of the training’s
impact on the learners and business.
At EI Design, we use the approach shown here. I have highlighted the
key aspects that need to be captured/processed at each stage.
This is the baseline and will be eventually used
to measure the desired gain for the learners
(acquisition of new skill or fixing a gap) and if
this gain resulted in the required impact the
business had sought (against the specified
business KPIs).
Capturing the needs of the learners through
Learner Needs Analysis (LNA) and having clarity
on how their progress will be measured (that is,
the expected gain that learners and business
seek) is a key component of the exercise.
Step 1 –Begin
with Training
Needs Analysis
(TNA) and
Learner Needs
Analysis (LNA)
43
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Step 2 - Identify
the employee
Training metrics
–L&D metrics,
Learner
Performance
Gain, and
Business metrics
Alongside the TNA phase, 3 key metrics need to be
identified –L&D metrics, Learner Performance
Gain, and the Business metrics. The L&D metrics
can be measured easily through an LMS.
It is extremely important to identify the business
KPIs that the training should impact. Then the
approach to measure Business metrics should be
identified. The KPIs should have both Leading
indicators and Lagging indicators for
measurement.
44
Step 4 -
Communicate
the relevance
and value of
the training to
the learners
During TNA as well as LNA phases, motivation
factors must be identified for key learner personas.
Looking at the dynamics, the appropriate support
measures to improve motivation levels must be put
in place.
Step 3 - Establish
measures to
motivate learners
At this stage, the approaches to highlight the
significance of the training and how it would be
relevant to them should be identified.
Additionally, the approach to highlight the specific
value the training would create for the learners
must be established.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Step 5 - Gauge
learner reaction
At this stage, the strategy to ascertain the
learners’ reaction to determine if the learners
found the training to be relevant, engaging, and
easy to internalize and apply is firmed up. This is
typically done through polls and online surveys.
This feedback allows L&D teams to confirm that
the basic goals have been met and if there are any
gaps, they should be addressed through
remediation/additional intervention.
The next step is identifying the right format of
training that aligns best to the TNA and LNA
cues. (Sometimes, training may not be the
answer, and you need to identify supporting
measures like coaching or mentoring).
The selection of the right format is crucial in
encouraging the learners to pursue it and this
also ensures that they connect with it as they
see value, complete it, and apply the learning
on the job.
The right immersive learning strategy combined
with an effective assessment strategy will help
you accomplish learning goals ranging from:
•Learn.
•Practice.
•Apply.
•Change in thinking.
•Change in behavior.
Step 6 -
Determine the
right training
format (Online,
Blended, or
VILT) and
support through
Coaching/
Mentoring
45
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Step 7 - Identify
the right
learning
strategy
The selection of the right learning strategy
format is vital in engaging the learners. As we
know, only an effective learning strategy can
create a sticky learning experience.
However, it is equally important to note that
having Formal Training alone may not be
enough. We also need to provide room for the
application of knowledge and practice or
nudges to mastery. Hence, the learning strategy
must have a combination of Formal Training
and Performance Support intervention to
successfully manage your mandate.
You need the learning journey to have
measures to:
•Refresh.
•Reinforce.
•Practice.
•Challenge.
We also recommend adding measures to
sustain connects including,
•Ongoing check-ins.
•Curated content to explore and learn more.
46
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Part 1 - Checkpoint
An effective assessment strategy is the main
approach to determine this.
However, it is important to ensure that the
measurement of this gain also factors for the
validation of the application of the acquired
learning and is not limited to the validation of
knowledge acquisition.
The right assessment strategy is critical in
determining if the required cognition levels were
met. Hence, the key measure at this stage are the
scores of the summative assessment.
From an evaluation perspective, this feedback
allows L&D teams to measure if the training met
the required learning outcomes. If any gaps are
identified, the required action (reinforcement,
remediation, or providing supplementary learning
aids) can be taken.
Part 2 - Validate
Next, a validation must be done if there is a change
in the learner’s behavior that is directly
attributable to the training.
This is typically done by polling the learners and
their supervisors after 30/60/90 days, and a
validation of the required gain is done. The gain is
measured through a combination of L&D metrics
and Learner Performance Gain metrics.
From an evaluation perspective, this feedback is
the crucial validation of internalization of learning
and its application on the job (performance gain)
as well as its ability to trigger behavioral change.
Step 8 -
Determine the
gain by the
learners and
change in
behavior
47
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
We can look back at the parameters identified
during the TNA stage and assess the required
gain that has occurred and if the business saw
the required impact.
At this stage, a validation is done if the training
created the required value for business and if it
was able to impact the business KPIs. This is
done by tracking the parameters captured in
the Business metrics.
From an evaluation perspective, this feedback
provides the crucial validation that the
business impact goal was met.
Remediation or sustaining the impact and ROI
determination is the final outcome of this exercise.
Once we monetize the gain and compare this with
the costs, we arrive at the Return on Investment
(ROI) on the training spend. Often, this outcome
may need one to go back to the TNA phase and
reassess or tweak the way forward.
Step 9 -
Assess the
gain for
business
Step 10 - Close
the loop
48
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs 49
Note
Increasingly, the term Return on
Expectations (ROE) is emerging
as a more effective approach
rather than Return on
Investment (ROI).
In fact, in their 2010 paper, Kirkpatrick and Kirkpatrick described ROE as
the ultimate indicator of value. This is because the expected outputs
and outcomes were established before the program commenced.
•They point out, “stakeholder expectations define the value that
training professionals are responsible for delivering” (2010: 36).
•They advise that program leaders should ask senior leadership to
“clarify and refine” what expectations they have at all four
Kirkpatrick levels.
•This approach of asking senior leadership what expectations they
have of the programme can be then used to measure and
demonstrate the true value of training.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Here is an instance of how we use LITMUS:
50
Part 1 - To assess the impact of training programs on
employee performance
•Assess if learning outcomes are met.
•Through Surveys, discussions, quizzes, exams,
and case studies.
•Skill-based assessments to understand if talent
development outcomes are met.
•Cues to assess if performance outcomes are met:
•Time-to-productivity.
•Employee performance gain (before and
30/60/90 days after the training program).
•Feedback from managers (before and 30/60/90
days after the training program).
•Review employee productivity (before and
30/60/90 days after the training program).
•Measure employee behavior change (before and
30/60/90 days after the training program).
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs 51
Part 2 - To assess the impact of training programs on
organizational performance
•Pre training
•TNA phase: Map learner outcomes to performance outcomes
and business outcomes.
•Identify L&D metrics and Business metrics and establish a
link between them.
•Post training
•Review improvement in organizational performance by
analyzing L&D metrics, Learner Performance Gain, and
Business metrics.
•Measure the business impact of training programs and
assessing training ROI.
•Other indicators:
•Employee morale and motivation.
•Improved communication and collaboration.
•Talent mobility.
Here is an instance of how we use LITMUS:
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs 52
As mentioned in the beginning of
my eBook, the task of measuring
and sustaining the impact of
training programs is tough and
there is no single approach that can
help you achieve this.
I hope these insights and strategies
provide practical cues that you can
use to measure the business impact
of your employee training programs
and use the recommended
strategies to maximize the impact.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Successful L&D programs align to both employee performance targets as
well as corporate strategy and tactics. A Learning and Development
program may seem effective based on metrics like utilization or reaction
scores, but only legitimate KPIs identify actionable results.
The correct KPIs can ensure that Learning and Development programs
support and drive vital enterprise strategy and tactics. Training metrics
track progress on specific measures - learner reaction to a course,
number of people trained, knowledge assessment scores, or hours utilized
in an LMS.
However, this is not enough. Inherent in this equation is the need to align
employee performance targets with corporate strategy. This requires
effective collaboration between the L&D team and business stakeholders.
So, how should you select the right training KPIs, and what strategies can
help you identify them for your learning and development programs?
Read our article for insights on identifying the right KPIs to drive business
results and ensure alignment with key corporate strategies and initiatives.
54
How to Identify the Right
Training KPIs for Your
Learning and
Development Programs
Read Article
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Analyzing training metrics drives improvement. Often, organizations don’t
factor how to measure the impact of training until after the solutions are
implemented. The correct Training metrics can serve to validate the level
of success of your training programs.
Effective Training metrics:
•Will help you validate if the planned training is improving employee
and business performance.
•Can help you identify where employees struggle, feeding into the
training design loop.
•Can then be used to help drive business results as employee behavior
improves.
So, how can you use employee training metrics to measure training
effectiveness and impact?
Read our article to find out how to map employee training metrics to KPIs
and tips and strategies to use the right employee training metrics for your
training program.
55
How to Use Employee
Training Metrics to
Measure Training
Effectiveness and Impact
Read Article
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Data has become the byword as a key to success in business. Without data
analytics in training and development to drive decisions, business leaders
are operating at a distinct disadvantage to their competitors.
Without the insights provided by data analytics in training and
development, L&D teams cannot successfully support business strategy,
tactics, or drive positive outcomes. It is also important to create a plan
that will help convert the insights mined out of data into actionable steps
to improve learning programs.
So, how can L&D team leverage data and analytics to enhance the impact
of training programs?
Read our article for benefits of leveraging data analytics in L&D and key
questions and action plans to convert them into steps for improvement.
56
How to Leverage Data and
Analytics in L&D to
Enhance the Impact of
Training Programs
Read Article
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Every year, significant investments are made on workforce
development programs. However, assessing their impact on business is
a tough mandate.
Workforce development programs are essential in enhancing employee
performance and, more significantly, driving business results. A
majority of the trainings get delivered as planned and are normally
tracked for registrations, completions, timely completions, and
assessment scores. However, not much data is available on the impact
of these trainings on business –and if they were able to demonstrate a
clear gain that was sought.
How do you maximize the business impact of your workforce
development programs?
Read our article to gain insights on tips and strategies to maximize the
impact of training programs.
57
Tips and Strategies to
Maximize the Impact of
Training Programs
Read Article
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
In professions where the results of poorly trained and underqualified
employees turn into high-stakes negative results, like the military,
various emergency services, and medical professions, the value of
training is implied and recognized. Those organizations train often and
effectively.
But in organizations where the results of training are less stark, it’s
more difficult to demonstrate the value of training and development.
Regardless of how obvious the results, it’s critical to demonstrate the
value of training to an organization.
So, how do L&D teams demonstrate the value of training and
development?
Read our article to get insights on tips and strategies to demonstrate
the value of your training programs.
58
Tips and Strategies to
Demonstrate the Value of
Your Training Programs
Read Article
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
1. Research Brief - Tying Learning Investment to Performance
2. Strategic Measurement –How to Ensure Learning has a Positive
Impact on Performance
3. How to Measure and Maximize Training ROI
4. Creating a Stronger Link between Learning and Performance
5. A Framework to Measure the Business Impact of Training Programs
6. Demonstrating Learning’s Value to the Business
7. Case Studies
8. Demos
9. Insights
Read More
59
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
As a learning experience design company, we partner with customers in
their Digital Transformation journey.
We help our customers deliver high-impact Learning and Performance
Support solutions that offer:
•A high engagement quotient.
•Drive employee performance.
•Create high ROI for the learner, business, and organization.
We are certified as a Smartchoice® Preferred
Provider by Brandon Hall Group. This is a
testament to our expertise of two decades in
helping L&D teams deliver trainings that
boost employee performance, impact
business KPIs, and trigger behavioral change.
About EI Design
60
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
We have a comprehensive framework to help organizations
“futureproof” their learning.
•From strategy to development to delivery to measurement, we work
with customers step-by-step to ensure they create engaging
learning environments.
•Our measurement model ensures learning, application on the job,
performance gain, impact, and the eventual ROI on the business. We
will help you gain more accurate insights on where you should be
making future investments, further maximizing the ROI.”
L&D Advisory and Consulting services - Our comprehensive portfolio
enables organizations to develop and execute strategies that drive
individual, team, and organizational performance.
Our training strategies will ensure that your investments deliver
business results through the following:
•Custom content design and implementation services.
•L&D Advisory services to futureproof your Virtual Training.
•L&D Consulting services to ascertain your training impact and
determine the ROI.
About EI Design
61
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
With our expertise, you can also improve employee performance and
leverage learning strategies that impact your business objectives.
We ensure training investments drive performance gain and
maximize training ROI. We have been recently ranked number one for
Best Content Providers To Help You Prove The Training ROI, by
elearningindustry.com.
A testimony of the impact of our
Consulting and Advisory Services.
About EI Design
62
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Testimonials
63
rankings over the last 2 years
25
Michael Rochelle
Chief Strategy Officer and Principal HCM Analyst
Brandon Hall Group
EI Design offers a unique value proposition that is rarely seen
in the market. EI Design assists organizations in optimizing
the impact of learning on individual, team, and organizational
performance and maximizing the ROI on learning.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
rankings over the last 6 years
41
Testimonials
64
Christopher Pappas
Founder
eLearning Industry
EI Design has a massive impact on how training makes a
business reach success. I trust that their methodologies
and training needs analysis experience can lead
companies to choose the proper training solutions.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
rankings over the last 2 years
2
Testimonials
65
Ken Taylor
President
Training Industry, Inc
EI Design’s focus on the combination of learning and performance
strategy with a specific attention to effective measurement of the
business impact of their programs sets them apart from many of the
vendors in the custom content segment of the market.
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Looking at adopting new ways to enhance your Learning Strategy?
EI Design can help you with that and more!
Training Delivery
•Mobile Learning.
•Microlearning.
•Virtual and Blended
Training.
•Mobile Apps for Learning.
•Personalized Learning.
•Localization.
•eLearning Accessibility
Considerations.
Immersive Learning
•Gamification.
•Scenario Based Learning.
•Video Based Learning.
•Story Based Learning.
•Branching Simulations.
•Virtual Reality.
•Augmented Reality.
•Next Gen Strategies.
Continuous Learning
•Performance Support
Tools.
•Informal Learning.
•Social Learning.
•Self-Directed Learning.
•Content Curation.
EI Design’s comprehensive portfolio of services is not limited to Formal
Training alone but will enable you to drive employee performance and
bring in behavioral change.
Learning Technologies
•LXP Expertise.
•LMS Expertise.
•Learning Portals.
•VR Framework.
•Interactive Video Framework.
•Rapid eLearning.
•Flash to HTML5 Migration.
Key Learning Consulting
Offerings
•Virtual Training
Transformation.
•Measuring Training
Effectiveness and Business
Impact.
•Technology Migration
L&D Advisory Services
•Optimal Virtual Training
Transformation.
•Redesigning existing content
into Next Gen formats –to
drive results.
•Redesigning content for LXPs.
66
Contact us
Prove and Improve - How to Measure and Demonstrate the Impact of Your Training Programs
Training Needs We Address
•Compliance Training.
•Induction and Onboarding
Training.
•Leadership Training.
•Sales Training.
•Application Simulations
Training.
•Product Training.
•Soft Skills Training.
•Professional Skills Training.
•Diversity, Equity, and Inclusion
Training.
•Business Sustainability Training.
L&D Consulting Services
•Ascertaining Training Impact
through our unique framework.
•Training Needs Analysis
(TNA) and Learner Needs
Analysis (LNA).
•L&D metrics and business
metrics identification.
•Measurement of impact on
learners and the business.
•ROI determination.
Contact Asha Pandey
email: apandey@eidesign.net
Contact EI Design
Stay connected on EI Design’s social platforms for
regular updates.
email: solutions@eidesign.net | www.eidesign.net
Contact us
67
