In his article in the current issue of Chief Learning Officer Magazine, Timothy L. Hill argues for IOB over ROI. That is, when it comes to learning interventions, CLOs should measure “Impact on Business” instead of “Return on Investment”. Hill writes:
ROI typically falls short when it comes to determining how engaged learners were, how much they retained or how job performance skyrocketed after learning. Further, the results of programs have to last long after the classroom or online program concludes. For instance, are employees using their new skills to improve customer service? Are sales on the upswing? Has customer repeat business improved? Have inventories declined? Are the improved results maintained for several quarters, or is it just an immediate spike?
While I agree with his basic supposition, he weakens his argument by using a very narrow definition of ROI and then rejecting the measure as insufficient. Hill defines a training ROI as the ratio of cost savings to the amount spent on training. This is a standard financial definition. The problem is that in the employee training field (See ROI Institute.), ROI has come to mean a comparison of all of the costs of employee learning to all of the impacts and benefits of the intervention, which can include employees retained, sales goals achieved, increased revenue, increased market share, job performance improvement, effective teamwork, customer satisfaction, employee engagement, etc. However, even using this broader definition, ROI is only part of the picture. The problem with a formulaic calculation of ROI is that you don’t know how the learning intervention affected outcomes, you don’t know to what extent the learning intervention contributed to the change, and you don’t know what else could have been done to increase impact on business results.
CEOs want to know more about business results. The important thing is to measure attainment of the learning intervention’s specific goals and produce evidence that is convincing to CEOs and other executives, answering their questions about learning and performance. An ROI calculation might be part of this, but the intended outcomes might require observations by team leaders and co-workers or stories of how employees applied what they learned to achieving business results. For example, in leadership coaching, process and outcomes are often not known at the outset. To evaluate this kind of learning intervention, you need to discover the story about what was done in coaching, what the learner learned, how that learning was applied, and what difference it made for the organization. If this is what Hill means by “IOB”, I’m all for it.