- Learning Evaluation
- November 28, 2023
Measuring what learning really changes
If the only metrics you can report to your board are “98% completion rate” and “4.5/5 average satisfaction,” you are measuring the efficiency of the delivery, not the efficacy of the learning.
We have an addiction in the learning industry to easy metrics. It is easy to track who clicked ‘Submit’. It is much harder to track whose behaviour changed on the shop floor, or in the trading room, or on the ward.
What is the flaw of the afterthought?
The primary reason most evaluation fails is that it is bolted onto the end of a project. The course is built, launched, and then someone asks, “How do we measure if this worked?”
By then, it’s too late. Evaluation must be the architectural blueprint of the learning design.
Before a single screen is designed, you must ask: What specific, observable behaviour are we trying to change? And what business metric currently tracks that behaviour?
How do we construct a valid framework?
True evaluation requires moving from correlation to causation. It requires mapping learning objectives directly to operational KPIs.
If you are designing a leadership programme to improve retention, you must track the turnover rates of the specific managers’ teams over the following 12 months. If you are designing technical onboarding, you must measure the reduction in time-to-first-commit.
This requires cross-functional collaboration. Learning designers must become comfortable speaking the language of data analysts and operational directors.
When we shift our focus from what the learner did in the module, to what the learner does in the business, we stop being a cost centre and start being a strategic lever.
Jason Baker, Founder, Snowfish Learning
Board Director