When Donald Kirkpatrick first formed his training evaluation framework, it was 1964 and the world was a very different place. The Beatles were just making it big, training was a classroom event and no one had even dreamt up the idea of the internet – let alone online learning.
Back in these simple times, The Kirkpatrick Model worked well – because learning took place in a classroom it was much easier to measure its success.
Since the internet arrived and online learning exploded, research has found that learning is a social phenomenon, not a classroom session. In this brave new world, we can’t treat online learning like a one-off event.
Even though a lot has changed in half a century, the models we use to evaluate our learning haven’t. They still treat learning like it’s a specific moment in time that you can pause and measure. But, we know that isn’t the case.
A Training Evaluation Framework For the Future…
It’s time we brought our training evaluation frameworks up to date. Which, is why we gathered in our War Room and drew up ‘The GEVALUATION Framework’. We drew inspiration from heroes of old like Donald Kirkpatrick, Roger Kaufman and Jack Phillips and injected these old models with a serum that transformed it into a lean mean evaluating machine.
So, now that the machine’s been powered up let’s see what it can do:
Integrated Feedback
What is it: These aspects of the training evaluation framework are built into the learning content. This includes the learner’s reaction to the content and the degree to which they’ve engaged with the content.
How do you measure it: Learner’s reactions can be measured using rating systems like a thumbs up if you liked it or a thumbs down if you didn’t. You can also use quiz questions to gauge how much information learners can recall and build a picture of learner engagement.
When do you measure it: Everything at this stage is built into the training content and the platform itself. Reporting algorithms and web tracking codes can keep track of when the learner logs in, or how long they spend and their reaction is measured as soon as they submit their feedback.
Case Study: Wayne Enterprise
Batman Bruce Wayne introduced a training scheme for his staff to help them improve their sales of combat equipment. Being a smart billionaire, he used an LMS which gave him instant feedback on how much his staff enjoyed their training and how engaged they were!
Behavioural Change
What is it: Behavioural change is the whole point of online learning. If training doesn’t change the way that learners do things – then what’s the point of doing it?
How do you measure it: Traditionally behaviour change has been very tough to measure. But, with the right LMS it doesn’t have to be so hard. Social functionality and game mechanics help you measure learner engagement which is the first signs of a change in behaviour.
When do you measure it: You can use your LMS to continuously measure behaviour change. After as little as three months of tracking behaviour, you can expect to see the final results!
Case Study: Wayne Enterprise
As Bruce was using a fully gamified LMS it was very easy for him to find out which of his employees had changed their behaviour. Three months after the training had launched he had a look at the league tables and totted up the score. Bruce was pleasantly surprised at how many team members had changed their behaviour and become sales superheroes!
Macro Result
What is it: The Macro Result of the training evaluation framework looks at the bigger picture of how training has benefited the company. Have sales increased? Is company culture improved? Has employee turnover reduced?
How do you measure it: An effective macro evaluation relies on careful pre-planning. You need to begin preparing your training with the end in mind. This means that you need to decide what’s being measured only then will you know if you’ve succeeded.
When do you measure it: Six months to a year after training began.
Case Study: Wayne Enterprise
Before embarking on his training, Bruce had decided that his measure of success would be a 30% increase in combat equipment sold, compared with the previous financial year. This meant that he had to wait until the end of the financial year to find out the results. The results showed that sales had in fact improved by 36.2%!
Financial Return
What is it: How much money did you put in compared with how much you got out?
How do you measure it: This is simple enough – First, you work out how much profit the training programme has generated. Then you subtract the amount you spent on the programme. What you’re left with is the financial return!
When do you measure it: You’ll already know how much you spent on the training programme. Once you’ve worked out the financial impact of the training, you can calculate the ROI. This is typically calculated one year after the launch.
Case Study: Wayne Enterprise
Bruce knew that he had invested $147,000 in the project, and then he worked out that the £36.2% increase in sales led to an extra $1.2m in profit for the year. That’s a return of $1.05m and one chuffed Batman CEO.
The world has changed a lot since 1964. These days The Beatles are bona fide cultural icons and the internet is everywhere. Even training evaluation is catching up thanks to Growth Engineering’s Evaluation Framework!
