The UK’s Chartered Management Institute offers a seven step “checklist” for evaluating training. Of course, any evaluation checklist is, by definition, an over-simplification of what can be a very complex process. Essentially, the CMI approach consists of clarifying training objectives, comparing trainee knowledge and behavior to objectives, and using this information to improve the training in the future. This is a useful way of thinking about evaluation when the program to be evaluated is highly technical, straightforward, predictable, and knowledge and skills can be learned in a relatively short period of time. But for anything else, such as leadership development, the seven steps fall short. Graham O’Connell, of the UK National School of Government, captures the issues well in his comment about the CMI article:
…I think there are some gaps [in the checklist] which reflect gaps in the wider debate on evaluation. Firstly, let me say unequivocally that evaluation is about securing and demonstrating great value results. But for these to be sustainable results we need to understand why things have worked and, where there have been problems, why those have occurred and what we might do about them. In other words we need to evaluate the process right from concept and the agreement of success criteria, through the design and delivery of the learning interventions, on to the consolidation, application and cascade of learning, and into improved performance and any other intended or unintended outcomes. In a sense evaluation is about learning. Our learning about learners, other stakeholders, organisations, culture, change, business and how learning interacts with all these issues…What we need is an intelligent analysis of the whole process - including satisfaction and results, quality improvement and systemic learning, real business benefits for today and increased capability for the longer term.
I agree with O’Connell; when we think of training evaluation as an opportunity for organizational learning, and not simply a way to determine if the initially intended results were achieved, the approach we must take is very different from CMI’s seven point checklist. We must consider how trainees come to the task of learning, how the training is delivered in the classroom and outside the classroom, what organizational factors (e.g., structure, communication, strategy) are affecting learning, what happens in the workplace over time to facilitate learning or to be a barrier to learning, what are the unintended consequences of the training program, and how are the individual, teams, and the organization affected by what the trainee has learned.