The Performance Improvement Blog

Short Course on Evaluation of Training and Learning in Knowledge Economy

LAD Global, in partnership with the Singapore Training and Development Association, has made my short course on evaluation of training and learning available for free online. I hope people involved in talent development will find this course to be a helpful introduction to measuring the impact of all types of learning interventions, not only formal training.  

My emphasis in this course is on using measurement and evaluation for learning. Much of evaluation in organizations today is still focused on formal training programs and limited to Kirkpatrick’s “level one”. In other words, L&D professionals are using “smile sheets” that measure immediate reaction to classroom instruction, collected at the end of training. Of course, we all are curious about what participants think of our programs and us as trainers. Great to know for marketing purposes.

But this information is not particularly helpful to the organization. It doesn’t tell us why the program was the right solution in the first place, what was learned, why that learning is helpful or not helpful to participants and other stakeholders, what happens when participants apply the content in their organizations, what are the intended and unintended consequences, what can be done to ensure that the content is applied in a positive way in the future, what organizational factors beyond the training are affecting impact, and what difference, positive and negative, the training has contributed to achieving organizational goals. This is the kind of information we need if we want to increase the impact of our learning interventions.

Given this purpose, the course covers methods that can be used to measure and evaluate the process of learning in organizations. I summarize three major approaches to evaluation: Kirkpatrick’s four levels; Phillips’ ROI; and Brinkerhoff’s Success Case Method. And then I explain how to select the best method for the situation and how data (quantitative and qualitative) from any of these methods can be used to improve learning across an organization. If you take a look at the course, I welcome your feedback.

For more on this topic, see our new book, Minds at Work: Managing for Success in the Knowledge Economy, published by ATD Press, available now on Amazon.

Assessing Your Organizational Learning Culture

To what extent does your organization have a learning culture? Before you take your first step on your journey, know where you are starting. Look around your organization. What do you see? What do you hear? What do you feel? What is your current culture?

Using Edgar Schein’s definition of organizational culture, you’ll want to know to what extent:

• Underlying beliefs and assumptions support learning in your organization
• Values and principles drive learning in your organization
• Employees and other stakeholders (suppliers, Board of Directors, customers) see the symbols and artifacts of learning and performance improvement.

Do employees, their teams, and the organization as-a-whole know what they need to learn to be successful? Do they know how to develop these competencies? Do they know how to sustain this learning over time? Do they know how to ensure that learning is applied and makes a difference for the organization?

Tools currently exist that might be helpful in this analysis. Each of these tools is designed according to the authors’ definition of a learning culture. For example, David Garvin and Amy Edmondson created an assessment tool to look for: a supportive learning environment, concrete learning processes and practices, and leadership behavior that provides reinforcement. I wrote this in a previous blog post about the tool:

As the authors say, the tool should be used for learning, not to judge the quality of an organization. The survey provides feedback for organizational reflection. By collecting the data and then discussing the findings, people must confront critical questions: Does our culture support learning? Do we have every day processes and procedures in place to ensure that learning and change are embedded in the way we work together? Do our top leaders make continuous learning a priority and communicate this throughout the organization? And then, hopefully, leaders will use the answers to these questions to motivate further learning and performance improvement and contribute to developing a learning culture in the process.

Another assessment tool is one created by Marcia Conner which she calls a “learning culture audit”. This tool is used to determine the extent to which an organization is oriented towards learning. She writes:

One way to begin the process of creating a learning culture and to enroll others in the effort is to conduct a learning culture audit. A simple diagnostic can help you assess your organization and your management team’s orientation to learning. An assessment describes the characteristics of cultures that encourage learning and those that block learning.

The dichotomous variables that Conner presents, comparing a “pro-learning culture” to an “anti-learning culture”, are certainly grist for discussion within your organization. For example, one set contrasts the item, “People at all levels ask questions and share stories about successes, failures, and what they have learned”, with the item, “Managers share information on a need-to-know basis…People keep secrets and don’t describe how events really happened.” Responses should stimulate a rich discussion about information sharing in an organization.

Learning to be Great™ has also created a learning-culture assessment tool. This tool is based on a framework that consists of five key elements of learning in organizations: 1) Alignment; 2) Anticipation; 3) Alliance; 4) Application; and 5) Accountability. These are the 5As of a learning culture. Each element is measured by four items in the survey. To fill out the survey and find out how your organization is doing on each “A”, go to the Learning to be Great website.

Whichever assessment tool you use, what’s important is to follow a stakeholder-focused process that results in organizational learning. Jim Stilwell outlines this process in his post titled, “How to Use Feedback from Employee Surveys to Change Organizations.” He writes that the organization survey process..

…works best if senior leadership communicates a compelling need for the survey in advance of sending questionnaires to all or a sample of employees.  As a part of this communication, leaders should clearly define the entire data gathering and feedback process emphasizing the role of employees.  Secondly, the findings from the survey would benefit from validation by key stakeholders – namely those who have provided the survey responses. The data from a survey can always be interpreted in many different ways given organizational circumstances, response rates, respondent demographics, the wording of questions and design of the survey. Thirdly, discussing the findings with groups throughout the organization and at all levels for their interpretation and recommendations leads to a deeper understanding, greater alignment, and vastly better solutions. A fourth important step in the process is for leaders to communicate back to the entire organization what it is that will be done as a result of the insights gained through the survey and all of the group discussions. This communication should clearly define what leaders know now that they didn't know before? What will be changed that will improve the work environment and increase organizational success? Who will be responsible and when will it happen? How will follow-up happen to ensure the success of the changes? 

The data from a survey is only meaningful if it generates a conversation in your organization that contributes to discovering of where you are starting your journey. Any survey of your organization should have as its purpose to learn from each other, to learn about your organization’s strengths and weaknesses, and to learn what will improve performance.

Organizational assessment is the GPS of your journey towards a learning culture. It tells you where you are at a point in time. Then you’ll be able to select the best route to your destination: a work environment that supports and encourages the continuous and collective discovery, sharing, and application of knowledge and skills at the individual, team, and whole organization levels in order to achieve the goals of the organization. 

Leaders Learning about Learning

Recently, I conducted a workshop for the leadership team of a company that wants to increase the impact of its training programs. I explained the limitations of formal training and the need for taking an organizational learning perspective. I argued that in order for any kind of learning intervention (training, coaching, mentoring, action learning, etc.) to have a positive impact on achieving the organization’s goals, managers had to take an active role in supporting learning.

This message was well received by these senior leaders. They immediately understood the vital role they play in developing individuals, teams, and the organization as a whole. They wanted to know specifically what they could do to facilitate learning.

I suggested the 5As Framework as a place to start. This is a useful model for ensuring that, regardless of the learning intervention, they will achieve performance improvement and organizational success. The 5As are: 1) Alignment – align learning with strategic business goals; 2) Anticipation – expect success; 3) Alliance – form a learning alliance between learner and boss; 4) Application – apply new learning immediately; and 5) Accountability – hold learner and organization accountable for business results.

We discussed what leaders can do to ensure that these elements are addressed. Here are their suggestions:

• Alignment: explain to employees the importance of a particular training program and how that learning will help them help the company be successful; make this conversation part of employees’ performance reviews.
• Anticipation: clarify expectations for what you want employees to learn and communicate high but reasonable expectations for their learning; make this part of informal conversations they have about performance on a frequent basis (not only at annual performance review time).
• Alliance: meet with employees before training to identify learning goals, meet with employees during training (if possible) to discuss progress, and meet with employees after training to identify what was learned, how employees will apply that learning in the workplace, and what additional learning they need
• Application: provide opportunities for employees to immediately apply learning in their work in a way that will make a difference
• Accountability: give employees feedback on their application of learning and measure the impact of that learning on their teams and the organization

I was encouraged by the enthusiasm and understanding that these company leaders had for their role in learning. I can only hope that higher level executives and the company's CEO will support this role and recognize and reward what my workshop participants are doing to facilitate continuous learning throughout the organization.

Learning to Learn from Evaluation of Learning

The Kirkpatricks have four levels, the Phillips have ROI, and Brinkerhoff has the Success Case Method. Each approach to evaluation of training has something to contribute to assessing the impact of formal training on employee learning. However, the value of evaluation is not in the data. The real value is in organizational learning from evaluation. That is, from reflecting on the meaning and significance of those findings for improving performance and achieving the goals of the organization.

As I wrote in a previous post titled, Good Evaluation Facilitates Learning:

As organizations seek to be more accountable and measure the impact of learning interventions, they will need facilitation skills to help people think more deeply about what they believe and what they need to do to improve. Measuring change is part of good evaluation, but unless we can get leaders in an organization to learn from that data and apply that knowledge to improving performance, what’s the point?  

In a learning culture, employees are continually acquiring knowledge and skills not only from training but also from a myriad of other formal and informal learning interventions. These interventions might include coaching and mentoring, after-action-review, assessment centers, mobile apps, internships, experiments, or any of the many ways in which adults learn outside of the classroom. The combined success of all of these activities depends on other organizational factors, such as executive and manager support, availability of resources, opportunities to apply newly learned knowledge and skills, and incentives.

Therefore, evaluation in a learning culture must examine the impact that the entire organization has on learning…and then use data from that systems-wide assessment to improve the environment for learning and performance improvement. Douglas Thomas and John Seely Brown ask us to imagine this kind of environment:

… Imagine an environment that is constantly changing. Imagine an environment where the participants are building, creating, and participating in a massive network of dozens of databases, hundreds of wikis and websites, and thousands of message forums, literally creating a large-scale knowledge economy. Imagine an environment where participants are constantly measuring and evaluating their own performances, even if that requires them to build new tools to do it. Imagine an environment where user interface dashboards are individually and personally constructed by users to help them make sense of the world and their own performance in it. Imagine an environment where evaluation is based on after-action reviews not to determine rewards but to continually enhance performance. Imagine an environment where learning happens on a continuous basis because the participants are internally motivated to find, share, and filter new information on a near-constant basis.

Learning from evaluation depends on asking the right questions. In a previous blog post, I posed evaluation questions that an organization should be asking itself to assess the extent to which it has a learning culture. In Top 10 Questions to Evaluate a Learning Culture, I wrote:

These are my top ten but not my only questions for a company that wants to evaluate its learning culture. What’s important here is to realize that questions like these are a tool for learning, not blame. I want to help leaders reflect on the culture of their organizations and how well that culture supports learning so that they can make changes that help individuals, teams, and the whole organization be successful.

Another way in which evaluation is essential in a learning culture is in reinforcing learning and making it much more likely that new knowledge and skills are retained over time. As I wrote in the post, Evaluation Reinforces Learning:

Evaluation is not only a way to judge the quality of the learning intervention; it is essential for retaining learning and being able to apply that learning when it is needed. If you want to get the most out of programs intended to improve leadership and management, the evidence suggests that you must follow-up with an assessment of learning and of the application of that learning.

Evaluation in a learning culture can be and should be a tool for learning. The main reason evaluation of learning interventions often faces resistance in organizations is that in the past those efforts haven’t contributed to improving performance. End-of-training surveys and ROI do not tell us how to do a better job of preparing employees to help the organization be successful. We need to be evaluating the totality of learning activities, as well as the beliefs, values, and behaviors of the organization, and use that data to improve employee performance and achieve the strategic goals of the organization.

2015 - Year of the Learning Culture

The theme that cuts across most of my blog posts from last year is creating and sustaining a learning culture in organizations.

As a way of review, I’ve selected five blog posts about a learning culture from 2015 that have the most interest for readers. Here is the title of each post with a short excerpt. Click on the title to go to the full post.

1. Developing a Learning Culture Infographic This infographic explains briefly and concisely the need for a learning culture, barriers to a learning culture, actions of a learning culture, and results from a learning culture.
2. Training Culture vs. Learning Culture What’s the difference between a “training culture” and a “learning culture”? The answer is, “A great deal.” As the chart shows, in a training culture, responsibility for employee learning resides with instructors and training managers. In that kind of culture the assumption is that trainers (under the direction of a CLO) drive learning. Whereas in a learning culture, responsibility for learning resides with each employee and each team.
3. Pull, Don’t Push, Employee Learning As the digital revolution continues to fuel the faster rate of change, transforming all aspects of business, from supply chain management to communication, the highest-performing corporations are abandoning traditional “push” training for the “pull” learning model.
4. PwC Canada Strives for a Learning Culture If you’re looking for examples of companies that are striving to create and sustain a learning culture, PwC (PricewaterhouseCoopers LLP) of Canada should be on your list. I recently had the pleasure of speaking about the importance of a learning culture to the Edmonton meeting of The Conference Board of Canada’s Council for Learning and Leadership Development. Another speaker at the event, Karin Muchall, Development Leader for PwC Canada, impressed me with her description of how PwC is implementing the principles of a learning culture through a program they call Enhanced Working Practices^TM.
5. 50 Ways to Lever Learning In a learning culture, formal training is just one of many methods used to facilitate employee learning. In a learning culture, we start with the performance goal and then select the mix of methods that will help employees acquire and retain the knowledge, skills, attitudes, and beliefs they need in order to achieve those goals. This is a list of 50 of those methods.

50 Ways to Lever Learning

(My apologies to Paul Simon.)

In a learning culture, formal training is just one of many methods used to facilitate employee learning. In a learning culture, we start with the performance goal and then select the mix of methods that will help employees acquire and retain the knowledge, skills, attitudes, and beliefs they need in order to achieve those goals. This is a list of 50 of those methods. The first 25 are primarily instructor-directed; the second 25 are primarily learner-directed.

1. Instructor-centered class (fact to face) – traditional classroom in which instructor controls the content and learning process
2. Instructor-centered class (virtual) – similar to classroom except instructor delivers instruction via the Web and class can be synchronous or asynchronous
3. Instructor-facilitated seminar – meeting convened by an instructor; learners discuss a topic relevant to their work and chosen by instructor
4. Instructor-facilitated workshop – meeting convened by an instructor; participants learn from experience of working together on solving a problem or creating something new
5. eLearning  – content delivered to learner via computer; usually desktop computer
6. Mobile learning – a form of elearning that is accessed by a mobile device such as smart phone or tablet; can be anywhere, anytime
7. Coaching – a relationship in which a trained coach helps an employee develop the knowledge and skills to be a more effective manager by addressing real situations that manager faces in workplace
8. Mentoring – a relationship in which senior leaders impart their knowledge and wisdom on employees who are learning to be leaders
9. Learning alliance – a relationship between managers and their direct reports that focuses on employee learning and how managers can support that learning
10. Game –engaging employees in learning by applying principles of gaming (scoring, competition, rules of play, etc.) to create an experience that is interactive and fun
11. Simulation – replicating real-life problem solving within a safe environment; for example, learning business acumen by working with a team to solve a typical business problem and receiving immediate feedback on their performance
12. On-campus college courses – attending for-credit courses or non-credit courses that are relevant to one’s job
13. External online courses (e.g., MOOCs) – taking relevant courses online from leading institutions and from renowned faculty
14. Webinars – participating in a Web-based program using video conferencing software; usually a one-session offering by an expert on a specific topic
15. Internship – working in a temporary position for the purpose of learning about a job, the work environment of that job, and the organizational culture
16. Apprenticeship – working under the guidance of experienced employees for the purpose of learning specific skills
17. Business case-study – drawing lessons from discussing the documented story of actual events in another organization
18. Performance measurement – learning from measures of performance such as sales figures, production numbers, and customer service feedback
19. Success Case Evaluation Method – a method of evaluating training (or any learning intervention) by identifying those participants who successfully applied learning in the organization and telling their stories; learning comes from analyzing those stories and drawing useful conclusions from successes and failures
20. Assessment center – a dedicated space where employees participate in exercises designed to simulate the conditions of their jobs; observers look for specific behaviors that indicate the employee’s suitability for the work; learning comes from receiving performance feedback and planning how to improve
21. Department meetings – often a lost opportunity for learning, these gatherings can be designed so that participants learn about processes such as planning, project management, innovation, and evaluation
22. Testing knowledge – using results of knowledge tests to facilitate more learning
23. Testing performance – using results of behavioral demonstrations of learning to facilitate more learning
24. Training evaluation – learning from evidence (quantitative and qualitative) collected to show the impact that particular training programs have on individuals, teams, and the organization as a whole
25. Learning Management System (LMS) – using the data from training program tracking software as a focus for discussing employee learning goals and progress toward those goals
26. Roleplay – people (usually two or three) acting out roles to learn about themselves and others by putting themselves in somebody else’s shoes
27. Reflection-in-action – learning from reflecting on an activity while doing it
28. Reflection-on-action – learning from reflecting on an activity by looking back on what happened
29. Reflection-for-action – learning by applying what was learned to a new situation
30. Daily log – individual employees writing or recording learning from each day of work and then discussing their observations with co-workers
31. Survey debrief – meeting with co-workers and other stakeholders to discuss what can be learned from the results of company surveys (such as pulse, employee satisfaction, and climate)
32. Experiments – gathering evidence in a controlled environment to support or refute a particular change that is being proposed (for example, testing an innovation in the product development process)
33. Prototyping – testing a new design of a product or process by constructing an example or model and then trying it out and learning from what happens
34. Content apps – using a software application on a mobile device to provide information and instruction on-the-job, just-in-time
35. Job share – splitting the hours required for a job with someone else and sharing experiences with each other for the purpose of learning
36. Job rotation – trying out different jobs in an organization for the purpose of discovering the best fit and, at the same time, learning about the organization and its culture
37. Community of practice – people who share the same interests or responsibilities in an organization come together to learn from each other
38. Social media – using computer-mediated platforms for sharing information, such as Facebook, LinkedIn, Twitter, and blogs
39. Benchmarking – learning by comparing the structure, policies, practices, products, and programs of one’s organization to the best in the industry
40. Structured observation – learning about some aspect of individual, team, or organizational behavior by observing what occurs according to a set of questions and criteria
41. Books and articles – reading books and articles that have information and expert advice about an area of needed performance improvement
42. Video – learning as-needed from high-quality, relevant videos on sites such as TED and YouTube
43. Recordings – learning as-needed from high-quality, relevant recordings of presentations by business and organizational experts
44. Team reflection – working with team members to find useful meaning in data about team performance
45. Enterprise-wide reflection – working with co-workers across the organization to find useful meaning in data about organization performance
46. Performance support tools - print or electronic tools, such as checklists, micro-lessons, and video demonstrations, used post-training to ensure on-going performance improvement
47. Interactive performance support system (IPSS) – performance support tools that allow the user to interact with Web-based materials for the purpose of shaping the support being provided so that it’s just-in-time and just-enough
48. Professional conferences – attending local, regional, national, and international meetings that have presentations of content relevant to one’s work; learning comes from reflecting on that content and discussing application with others
49. Book groups – groups of employees meeting during the workday to discuss a book they have all read; they learn from the discussion about how it applies to their work
50. Internal wiki – like Wikipedia, this is a company Web site where any employee can contribute content that might be helpful to every employee’s learning; this could include relevant books and videos, tools such as checklists, best practices, survey results, project results, etc.

I’m sure you can think of more ways to lever learning. What would you add to this list?

SCORM and xAPI LMS in a Learning Culture

A learning management system (LMS) is considered to be a necessary tool in most complex organizations today, especially those industries, like nuclear energy, airlines, and healthcare, where monitoring compliance with government-imposed training requirements is essential.

However, I have had my doubts about the value of an LMS for enhancing learning and contributing to a learning culture. Knowing who has taken which course and even knowing how they did on a test, is not evidence of learning and certainly doesn’t indicate one’s ability to apply that learning on the job. And more importantly, an LMS does not tell us the impact of that learning on the organization.

I recently attended an “xAPI Party” at Torrance Learning where the conversation was about the shift from a SCORM-based LMS to an xAPI (experience application programming interface) LMS. xAPI appears to be a huge leap forward in tracking the whole gamut of learning experiences from formal training programs to informal, social, and mobile activities. There was a lot of excitement about this change and the benefits seemed clear.

Steve Foreman, in an article for Learning Solutions Magazine, describes XAPI this way:

The xAPI operates based on activity streams, a model that uses software to track things people do. The idea of tracking activity streams emerged from social networking and is used by sites such as Facebook, Twitter, and Google Plus.

The Experience API makes it possible to track activities that people do using computers such as performing work tasks, producing work outputs, interacting with others using social media, achieving milestones in games and simulations, and just about any other activity that one can observe or record.

Imagine that your LMS can communicate, not just with eLearning courses, but also with knowledge bases, collaboration platforms, document management systems, enterprise resource planning systems, helpdesk systems, portals, talent management, performance management, and other types of systems used in the workplace. The learning management system may track not only attendance, completions, and test scores, but actual work inputs, outputs, deliverables, tasks, and more.

However, I have to ask, “Does xAPI do a better job of facilitating learning?” Any tool is only as good as the person who’s using it and this is true for xAPI, as well. Clearly, xAPI, because it includes a wide range of workplace learning experiences, is superior to SCORM for tracking activity, but it is still about activity, not learning.

To make xAPI effective for enhancing learning, employees will have to consistently collect evidence of experience, constantly input the right data, use that data, through feedback and reflection, to help individuals and teams develop their competencies, continually assess the impact of learning on the organization as a whole, and have conversations about the meaning of those findings. These are behaviors of a learning culture.

Employee learning requires feedback, reflection, and practice. Experiences, whether formal, informal, social, or mobile, do not result in learning unless these other elements are present. If an organization is committed to a learning culture, those elements will be the way the organization functions on a day-to-day basis. Without a learning culture, I’m skeptical that any LMS (SCORM or xAPI) can enhance learning and improve performance. However, in the context of a learning culture, xAPI offers great promise for contributing to organizational learning.

[image from Pixabay]

Industry Report: Too Much Training; Not Enough Learning

Training magazine has published its 34^th annual Industry Report that is based on a survey of its database of organizations with 100 or more employees, asking them questions about training expenditures and activities. The 2015 study has 777 respondents which is typically a good size sample from a statistical standpoint but I would question the reliability of this data given that it’s not a random sample. I would expect the margin of error to be high.

In any case, assuming that the findings, in general, describe the state of the industry in 2015, it is disappointing to see the continuing substantial investment in “push” training. “Instructor-led classroom only” is still reported to be the primary method of employee learning and development. If we include “blended learning”, “virtual classroom/webcast only”, and “online or computer-based methods” in the mix, we see that companies are continuing to invest most in methods of learning that have the least payoff.

We know from previous studies that only about 10% to 50% of learners (depending on the nature of the training and workplace environment) actually apply what they learn in training programs back on the job. If Training magazine’s numbers are accurate and approximately $70 billion was spent on training programs last year, we can estimate that the industry wasted roughly between $35 billion and $63 billion on training.

Organizations might be increasing their investment in “pull” learning through social media, on-the-job training, and action learning. However, we don’t hear about these learning interventions because they don’t fit the more easily observed and measured formal methods.

In any case, I think we need to look at all of the ways employees learn in their organizations and to what extent employees apply that learning on the job. Currently, we have a good picture of the state of formal training. If we are to truly understand learning in organizations, we need a better picture of the informal and "pull" ways in which employees learn.

I welcome a different interpretation of Training magazine’s data. Please comment to either support or to challenge my view of the industry.

Top 10 Questions to Evaluate a Learning Culture

In a discussion started by Camilla Keen on LinkedIn, she asks, “In order to find out and evaluate if a company has an effective learning culture, what would be your top ten questions?” I commented:

1. Are leaders communicating the importance of learning (acquiring new knowledge, skills, and capabilities) and holding managers accountable for learning and applying that learning to making a difference for the organization?
2. Are managers helping their direct reports create an individualized learning plan linked to strategic goals of organization?
3. Are managers monitoring learning progress and providing feedback; are they structuring opportunities to apply learning on the job; and holding direct reports accountable for results?
4. Are managers coaching; are they partnering with direct reports to develop their capacity to achieve organizational goals?
5. Are leaders mentoring; are they using their experience to advise new and less experienced employees on how to fulfill the functions of their jobs?
6. In recruitment, hiring, and onboarding of new employees, is learning part of the conversation; is learning conveyed as a value of the organization; are expectations for employee learning and development discussed during recruitment phase?
7. Can employees see how their learning is aligned with the strategic goals of the organization; do they understand how acquiring certain knowledge and skills will help the organization be successful; is a direct link from learning to results made clear?
8. Are a wide range of formal and informal, hi-tech and hi-touch methods being used to facilitate learning; are the methods used determined by the intended outcomes for the organization?
9. Are expectations for employee learning discussed with employees; do employees know what they need to learn and why they need to learn it and what criteria will be used to monitor progress and assess results; are high expectations communicated?
10. Is learning applied throughout the organization to continuously improve performance and achieve strategic goals?

These are my top ten but not my only questions for a company that wants to evaluate its learning culture. What’s important here is to realize that questions like these are a tool for learning, not blame. I want to help leaders reflect on the culture of their organizations and how well that culture supports learning so that they can make changes that help individuals, teams, and the whole organization be successful.

Performance Management at Deloitte

Kudos to Deloitte for making a concerted effort to develop a fair, efficient, and useful performance management system for its 65,000 employees. In a blog post for Harvard Business Review titled Reinventing Performance Management, Marcus Buckingham and Ashley Goodall describe Deloitte’s evolving system. They write:

This is where we are today: We’ve defined three objectives at the root of performance management—to recognize, see, and fuel performance. We have three interlocking rituals to support them—the annual compensation decision, the quarterly or per-project performance snapshot, and the weekly check-in. And we’ve shifted from a batched focus on the past to a continual focus on the future, through regular evaluations and frequent check-ins. 

To me, the key to their performance improvement will be the weekly check-in. This is where team leaders talk to team members about past performance, goals for future performance, and what can be done to be successful. This conversation, as I have written in a previous post, is an essential aspect of any leader’s role. As the authors say, “…check-ins are not in addition to the work of a team leader; they are the work of a team leader.”

Also, Deloitte has designed a potentially efficient way to measure performance over time. They intend to ask team leaders four questions at the end of each project or at least once a quarter. The questions are:

1. Given what I know of this person’s performance, and if it were my money, I would award this person the highest possible compensation increase and bonus [measures overall performance and unique value to the organization on a five-point scale from “strongly agree” to “strongly disagree”].

2. Given what I know of this person’s performance, I would always want him or her on my team [measures ability to work well with others on the same five-point scale].

3. This person is at risk for low performance [identifies problems that might harm the customer or the team on a yes-or-no basis].

4. This person is ready for promotion today [measures potential on a yes-or-no basis].

It remains to be seen whether these questions will “fuel” performance in the long-run. Given research that shows manager ratings of employees are more about the person doing the rating than the person being rated, Deloitte is trying to make the ratings more objective. However, it still requires rating by managers who are affected by their own attitudes and biases. If this process results in more conversation between team leaders and team members about learning and improving performance, then it will have served its purpose.

What concerns me most about Deloitte’s new performance management system is the apparent lack of alignment between wanting to be strength-based and using variable compensation. A system that rewards the performance of some and not others can be a disincentive for learning and performance improvement. In a strength-based system, everyone is recognized for what they contribute and for what they can become. A variable compensation system is a good way to control behavior but not necessarily a good way to encourage learning and improvement.

I want to be clear. I admire Deloitte’s intent and the willingness to experiment and use evidence to improve their performance management system and I am very curious to hear about results and impact on the organization. My concern is that the recognition and reward aspects of the system will negate the learning that can be achieved from leader feedback aspects of the system. We’ll see.