The Performance Improvement Blog

ROI vs. IOB

In his article in the current issue of Chief Learning Officer Magazine, Timothy L. Hill argues for IOB over ROI. That is, when it comes to learning interventions, CLOs should measure “Impact on Business” instead of “Return on Investment”. Hill writes:

ROI typically falls short when it comes to determining how engaged learners were, how much they retained or how job performance skyrocketed after learning. Further, the results of programs have to last long after the classroom or online program concludes. For instance, are employees using their new skills to improve customer service? Are sales on the upswing? Has customer repeat business improved? Have inventories declined? Are the improved results maintained for several quarters, or is it just an immediate spike?

While I agree with his basic supposition, he weakens his argument by using a very narrow definition of ROI and then rejecting the measure as insufficient. Hill defines a training ROI as the ratio of cost savings to the amount spent on training. This is a standard financial definition. The problem is that in the employee training field (See ROI Institute.), ROI has come to mean a comparison of all of the costs of employee learning to all of the impacts and benefits of the intervention, which can include employees retained, sales goals achieved, increased revenue, increased market share, job performance improvement, effective teamwork, customer satisfaction, employee engagement, etc. However, even using this broader definition, ROI is only part of the picture. The problem with a formulaic calculation of ROI is that you don’t know how the learning intervention affected outcomes, you don’t know to what extent the learning intervention contributed to the change, and you don’t know what else could have been done to increase impact on business results.

CEOs want to know more about business results. The important thing is to measure attainment of the learning intervention’s specific goals and produce evidence that is convincing to CEOs and other executives, answering their questions about learning and performance. An ROI calculation might be part of this, but the intended outcomes might require observations by team leaders and co-workers or stories of how employees applied what they learned to achieving business results. For example, in leadership coaching, process and outcomes are often not known at the outset. To evaluate this kind of learning intervention, you need to discover the story about what was done in coaching, what the learner learned, how that learning was applied, and what difference it made for the organization. If this is what Hill means by “IOB”, I’m all for it.

Colorado Best Buy Says Bye Bye to Employee Engagement

I’ve posted before about Best Buy’s enlightened attention to employee engagement and learning. However, a recent event demonstrates how fragile this distinction for any company can be, especially store by store. I don’t know all the facts and it’s possible the press accounts are misleading, but going by the news stories it appears that the Broomfield, Colorado Best Buy failed to live up to the corporate image. Two employees that attempted to stop a thief as he ran out the front door and pulled a knife were fired for their heroism. You see, store policy forbids employees from pursuing a suspected shoplifter after that person exits the store.

Giving the two twenty-somethings a reprimand or maybe having an all-employee meeting to once again explain and discuss the policy - yes. But termination just doesn’t seem right. This will set back everything the company has done to create a committed, enthusiastic, going-beyond-the-call-of-duty workforce. In effect, the manager of the Broomfield store has said to employees, “Just do your job; don’t think for yourself; don’t take any risks; and, by all means, don’t care about this store and its customers. Furthermore, be careful not to do anything wrong because we will not help you learn from your mistakes.” I don’t think this is the message Best Buy wants to send to its employees.

CEOs Want to Know Impact of Training

CEOs want more evidence of impact and less evidence of employee reactions (Kirkpatrick’s Level 1) to training. At least, this is one of the findings from a study by Jack and Patti Phillips recently published in T+D. They surveyed CEOs of large companies selected from both the Fortune 500 list of largest public companies and Hoover’s list of the largest privately-held companies. Ninety-six CEOs responded, which is 21.3 percent of the 451 CEOs who were sent surveys. Only six percent were “very satisfied” with their companies’ current measures of learning and development and 96% indicated that their companies should be doing more to measure the impact of learning and development programs on business results.

I have some concerns about the generalizability of the survey data given the way the sample was constructed, but I commend the effort to try to hear from CEOs about their perceptions of learning interventions in their companies. The findings confirm a common belief among evaluators of learning interventions (training, coaching, etc.) that CEOs are not being given the kind of data they want from the learning and development professionals in their organizations. CEOs want to know the impact on business goals but, instead, they are given attendance figures and told how much employees like the programs. The authors of the report write:

Executives intuitively feel that providing learning opportunities is valuable, and they logically anticipate a payoff in important, bottom-line measures, such as productivity improvements, quality enhancements, cost reductions, time savings, and improved customer service. Yet the frustration comes from the lack of evidence to show that programs really work.

If executives want bottom-line measures, there is much they can do to get what they want. For one, they could ask their learning and development leaders to provide this information. Then, when they receive impact data, they could report these findings widely in their organizations. They could ask their managers about the implications of the findings and what they will do to increase impact in the future. And they could give recognition and rewards to learning and development professionals who provide evidence of results and implement solutions to increase results. CEOs should hold themselves and their entire organizations accountable for the impact of employee learning on business results.

Culture, Values, and Empathy

Our national leaders need a little Psychology 101. They seem to be lacking an understanding of basic human behavior. During the recent Senate Judiciary Committee hearings for the appointment of Judge Sonia Sotomayor to the Supreme Court, some of the senators said they wanted judges who “set aside personal biases and prejudices to decide cases in an impartial manner.” Even more stringent, they implied that they wanted judges who were not influenced by their culture and values and were devoid of empathy. Apparently, they don’t want human beings on the Supreme Court. It’s not possible for people to form opinions, legal or otherwise, without being influenced by culture and values, and empathy is an essential characteristic of any competent leader.

President Obama, in responding to questions from Business Week about his leadership, said: 

…my most important job is to get the right people in the right place, give them the freedom to innovate and to think creatively about problems, hold them accountable for results, and make sure that they are cooperating with each other and communicating with each other on an ongoing basis.

So far, so good. But then he goes on to say:

I've always insisted on making sure we are equipping ourselves with the best evidence, the best data, the best information, that it's real-time, and that we don't ignore information just because it's not convenient or doesn't conform with what our working hypothesis might have been. We're constantly trying to make sure we're open to changing our minds about issues.

He, too, seems to want super-human staff. All people tend to ignore information that doesn’t conform to the way they view the world. This is part of being human. So Obama has a huge task ahead of him if he wants himself and his staff to behave differently.

All people form opinions and make decisions within the framework of a personal world view that is shaped by culture and values, and what information they choose to ignore and what information they choose to accept. The challenge for leaders is to embrace this aspect of themselves and to be continually aware of how values and beliefs are affecting their decisions.

Retaining Employees in Good Times and Bad

Retaining employees after the economy improves could be a bigger challenge than right-sizing the workforce. Adecco Group, a global provider of HR services, recently conducted another of their workplace surveys and reported the following:  

…the most serious threat to employers in this recession may be its end. The survey indicates every company's greatest asset, its human capital, might be its most tenuous, as employers could see an unprecedented exodus of talent when the job market rebounds. More than half (54%) of employed adults report they are at least somewhat likely to look for new jobs once the economy turns around.

I can’t speak to the accuracy of Adecco’s findings, given that the company doesn’t provide details about how their surveys are conducted, but I think the findings do draw attention to a problem worth considering as the economy begins to improve (We can only hope!).  That is, employees who are nervous about making a change when the economy is so tenuous will bail when things start to improve. And this might be most true for the youngest employees, the 18 to 29 year-olds, as is pointed out by Roberta Matuson on her blog, Generation Integration. 

Leadership coach and former auto company manager, Bernie Donkerbrook, said to me that the implications of these findings are “huge” for business. Companies have downsized to the point that nearly every person remaining is critical to success. If these employees start leaving just as the economic rebound is occurring, their organizations will have a hard time taking advantage of new opportunities. Recovery will be slowed considerably. And if many of the young seek greener pastures, companies will lose much of their energy and innovative thinking. Of course, as Donkerbrook reminded me, “The grass always looks greener on the other side of the fence.” What happens when they change jobs only to find that their new situation isn’t any better than their last situation?

 All organizations, nonprofits and government agencies as well as businesses, need to do what they can now to keep their best employees. I’ve posted previously about some of these actions: communicate often; build trust by being transparent about the present and future of the company; listen intently to employees; follow through on promises; and recognize and reward performance. Create a workplace that looks just as green from both sides of the fence.