In his blog post, “Stop Evaluating Training!”, Amit Garg summarizes a presentation that Robert O. Brinkerhoff gave at the Australian Institute of Training and Development conference in April. Garg relates Brinkerhoff’s comments to the challenges of measuring the effectiveness of elearning programs. Garg writes:
So how do you evaluate the success of eLearning that you create?" As a learning solutions vendor, I’ve been asked this question countless times and have also encountered it in many an RFP. Proving the effectiveness of training and showing ROI is no walk in the park and still keeps L&D up at night.
Maybe evaluation of the impact of training is not a “walk in the park”, but it is not “rocket science” either. As Brinkerhoff explains in The Success Case Method, there is a simple logic to assessing impact. First we have to understand the alignment between a learning intervention, whether that’s classroom-based, elearning, coaching, on-the-job structured experiences, or something else, and the intended impact (e.g., customer retention, production, sales, revenue, market share). Then it’s a matter of identifying and interviewing learners who are contributing to that impact and those that are not contributing. The purpose of these interviews is to understand the nature of the impact and why it’s happening or not happening.
Three immutable laws drive this method of evaluation. One is the law of learning as a process not an event. This law holds that learning is a process that starts before an instructional event and continues after the event. To study the event only is to fail to include key aspects of what is facilitating learning and what are barriers to learning.
Another is the law of unintended consequences which holds that when there is change there are always outcomes that weren’t anticipated. A useful evaluation of training will examine these unintended consequences (positive and negative) as well as achievement of objectives of the intervention.
And the third law holds that significant, lasting change in complex organizations is always the result of many factors, some that are not in the control of trainers and learners. For example, effectiveness of leaders, change in strategic direction, and the economy could have a profound influence on the impact that learners have on their organizations.
It’s not complicated. Understand how the learning intervention is intended to help the organization achieve its goals. Identify employees who are and are not applying what they learned. Interview these employees to find out what they did or did not do, how that affected the organization, what other factors were intervening in that impact, and what can be done to increase learner impact in the future.