The Kirkpatricks have four levels, the Phillips have ROI, and Brinkerhoff has the Success Case Method. Each approach to evaluation of training has something to contribute to assessing the impact of formal training on employee learning. However, the value of evaluation is not in the data. The real value is in organizational learning from evaluation. That is, from reflecting on the meaning and significance of those findings for improving performance and achieving the goals of the organization.
As I wrote in a previous post titled, Good Evaluation Facilitates Learning:
As organizations seek to be more accountable and measure the impact of learning interventions, they will need facilitation skills to help people think more deeply about what they believe and what they need to do to improve. Measuring change is part of good evaluation, but unless we can get leaders in an organization to learn from that data and apply that knowledge to improving performance, what’s the point?
In a learning culture, employees are continually acquiring knowledge and skills not only from training but also from a myriad of other formal and informal learning interventions. These interventions might include coaching and mentoring, after-action-review, assessment centers, mobile apps, internships, experiments, or any of the many ways in which adults learn outside of the classroom. The combined success of all of these activities depends on other organizational factors, such as executive and manager support, availability of resources, opportunities to apply newly learned knowledge and skills, and incentives.
Therefore, evaluation in a learning culture must examine the impact that the entire organization has on learning…and then use data from that systems-wide assessment to improve the environment for learning and performance improvement. Douglas Thomas and John Seely Brown ask us to imagine this kind of environment:
… Imagine an environment that is constantly changing. Imagine an environment where the participants are building, creating, and participating in a massive network of dozens of databases, hundreds of wikis and websites, and thousands of message forums, literally creating a large-scale knowledge economy. Imagine an environment where participants are constantly measuring and evaluating their own performances, even if that requires them to build new tools to do it. Imagine an environment where user interface dashboards are individually and personally constructed by users to help them make sense of the world and their own performance in it. Imagine an environment where evaluation is based on after-action reviews not to determine rewards but to continually enhance performance. Imagine an environment where learning happens on a continuous basis because the participants are internally motivated to find, share, and filter new information on a near-constant basis.
Learning from evaluation depends on asking the right questions. In a previous blog post, I posed evaluation questions that an organization should be asking itself to assess the extent to which it has a learning culture. In Top 10 Questions to Evaluate a Learning Culture, I wrote:
These are my top ten but not my only questions for a company that wants to evaluate its learning culture. What’s important here is to realize that questions like these are a tool for learning, not blame. I want to help leaders reflect on the culture of their organizations and how well that culture supports learning so that they can make changes that help individuals, teams, and the whole organization be successful.
Another way in which evaluation is essential in a learning culture is in reinforcing learning and making it much more likely that new knowledge and skills are retained over time. As I wrote in the post, Evaluation Reinforces Learning:
Evaluation is not only a way to judge the quality of the learning intervention; it is essential for retaining learning and being able to apply that learning when it is needed. If you want to get the most out of programs intended to improve leadership and management, the evidence suggests that you must follow-up with an assessment of learning and of the application of that learning.
Evaluation in a learning culture can be and should be a tool for learning. The main reason evaluation of learning interventions often faces resistance in organizations is that in the past those efforts haven’t contributed to improving performance. End-of-training surveys and ROI do not tell us how to do a better job of preparing employees to help the organization be successful. We need to be evaluating the totality of learning activities, as well as the beliefs, values, and behaviors of the organization, and use that data to improve employee performance and achieve the strategic goals of the organization.
