The Performance Improvement Blog

Learning and Performance Improvement in 2013

The Internet Time Alliance principals have made predictions for 2013. Charles Jennings offers this picture for the year:

A confluence of technology and improved connectivity, increasing pressures for rapid solutions and better customer service, and demands for higher performance, will force the hands of many HRDs and CLOs to refocus from models of ‘extended formal training’ to place technology-enabled, workplace-focused and leader-led development approaches at the core of their provision.

Clark Quinn predicts that “…we’ll see more examples of the notion of a ‘performance ecosystem’ of resources aligned around individual needs and responsibilities, instead of organized around the providing silos.” While he expects greater use of mobile devices to support individual needs and responsibilities, he thinks many organizations will fail to use these methods effectively.

Harold Jarche reminds us that “…work is learning and learning is the work in the network era…” However, he also believes that most organizations will continue to deliver formal courses and instructor-led training programs.

Jane Hart says that no matter what IT and T&D departments continue to do, individuals and teams will find ways to bypass these units and use technology to access what they need when they need it.

Jay Cross, similar to Jarche, argues that because of the speed and complexity of change, “Learning will continue to converge with work. Increasingly, workers will learn their jobs by doing their jobs.” He believes that “smart companies” will recognize the need for a management style that supports this kind of learning.

This group of thought-leaders seems to envision a world that is making strides away from classroom-based, formal learning and towards informal, social, mobile learning. My prediction is that 2013 will be a time of sorting out instructional delivery methods, with mobile devices taking center stage. But I don’t think there will be much overall gain in support of learning. The economic and legal pressures on organizations, perceived and imagined, will prevent most from adapting to the technological, social, and demographic changes that are going on around them.

To make learners accountable for their own learning, to push down responsibility for learning to customer-facing employees and project teams, and to engage workers in continuous performance improvement, means admitting that the way managers have been doing things might not be best for employees and it means giving up centralized control. This is difficult stuff for CEOs and other managers to accept.

So, while I hope the ITA prognosticators are right (Full disclosure: I have co-founded a business to support individual, team, whole organization, and community learning.), I’m afraid that 2013 is not going to be a year of substantive change in our organizations. Executives will talk a good game, and may even look like they are adapting to the new reality, but when faced with a choice most will opt for the familiar and safe path. 

A Year of “The Performance Improvement Blog” in Review

It’s time to do a little year-end reflecting on my blog posts from 2012.

Throughout last year I used this blog to illuminate important leadership and management issues. The topics ranged from creating a learning culture to increasing employee engagement to improving organizational communication to evaluating executive coaching, and more.

Out of all of these blog posts I’ve selected five that seem to have had the most interest for readers. Here are the links with a short excerpt from each post:

How to Create a Learning Culture in Organizations - Organizational learning is not about training. Rather, it's about a community of workers sharing in a process of constantly seeking improvement through new knowledge, new skills, and new applications of knowledge and skills to achieving the goals of the organization. They examine what they do, compare that to what needs to be done, reflect on what they have learned, and make the needed change in the organization.

 

Leaders Make or Break Employee Engagement - A leader’s attitude and behavior determine whether an organization has an engaged workforce or not. In other words, what leaders think and feel and say and do has a profound impact on employee satisfaction, enthusiasm for their work, desire to do their best, and commitment to go beyond their job descriptions every day. In an article for The Business Thinker, my colleague Jim Stilwell and I describe 13 ways in which leaders affect employee engagement. 

 

 

What Gets Measured Gets Done...Revisited - Measurement is not enough. An organization has to do something with those metrics in order to “get done” what needs to get done, i.e., organizational learning, performance improvement, and change. The act of measuring things like sales, customer service, product quality, teamwork, employee engagement, or learning, does not necessarily mean that anything will be done differently.

 

 

Why Evaluate Executive Coaching - CEOs and Boards are beginning to ask if the substantial investment they are making in executive coaching is paying off.  Unfortunately, we don’t know much about the impact of coaching. Studies tend to rely on surveys of executives who are being coached. But this method of studying coaching is fraught with problems.

 

 

What Do You Mean? - Dan Pallotta, president of Advertising for Humanity and HBR blogger, in a post titled “I Don't Understand What Anyone Is Saying Anymore”, writes about business jargon and how it affects communication in organizations. The blog post must have hit a nerve because it has broken the record for most comments to a single HBR Blog Network post. 

Wishing you a healthy and prosperous 2013!

Managing to Outcomes in Nonprofit Sector

This is the traditional time to reflect on trends observed during the past year. One major trend I have observed is performance management of nonprofit organizations. Actually, this is an accelerating trend that got its impetus from Peter Drucker decades ago. Nonprofits are becoming more and more focused on what McKinsey & Company calls “managing to outcomes”.  

I know, you thought all organizations managed to outcomes. Not so, particularly in the social sector. Nonprofits have been more like manage-to-what-makes-you-feel-good or manage-to-spend-the-money. It has been less about results and more about the process of doing the work and providing services. Now a shift is occurring towards greater recognition of the importance of measuring outcomes and using that information for learning. That is, learning how to be a more effective organization and how to increase impact of programs and services.

If your organization is involved in philanthropic activity, if you serve on the board of a nonprofit organization (or NGO), or if you work for a nonprofit or a philanthropic foundation, or if you donate to nonprofits, then you should pay special attention to this trend. It will affect what you do and how your money is spent.

This intensified focus on outcomes and learning is a result of a confluence of factors in the growing and increasingly important U.S. nonprofit sector (According to the Foundation Center, “In 2010, nonprofits contributed $804.8 billion to the gross domestic product (GDP); this equates to 5.5 percent of GDP… the nonprofit sector employed 13.7 million people.”). The factors that are contributing to the change include:

• Increasing demand for services for the poor, the disadvantaged, the elderly, and the marginalized
• Less public money for social programs (Mario Morino calls this the “Age of Scarcity”.)
• More people, and a wider spectrum of people, interested in “doing good”
• Greater demand by social investors for evidence of impact
• Greater demand from policy-makers for evidence of impact
• More leaders on nonprofit and foundation boards who want to see results
• Increasing professionalization of the sector (e.g., MBAs leading large nonprofits)

So the need for services is growing while resources are dwindling and, at the same time, funders and boards of directors want more accountability and more impact from their investment. Because of all of these factors, there is pressure on the sector to develop more efficient and effective organizations. These are organizations that embrace a culture of measurement and performance improvement. Mary Winkler of the Urban Institute emphasized this point when she said:

…organizational culture and a predisposition to measurement and managing toward results is perhaps the single most important ingredient to success. A culture of continuous improvement needs to be evidenced at the top. Equally important, however, is the extent to which the culture of continuous improvement is integrated at every level of the organization. 

That means having leaders who are willing to learn from what they do and apply that learning to improving organizations, as well as their programs and services. This is not an easy transition for many of these leaders who have been accountable for only process and output in the past. They will need the support of funders, staff, constituents, consultants, and the public at large, to make this change.

Learning to be Great Organizations

In a previous post, I wrote about the importance of democratizing organizational learning. I argued that organizations need to be continuously learning for the purpose of improving performance and achieving business goals. And in another post I wrote that for this to happen, organizational leaders need tools. That is, they need methods for learning from the organization and for learning how to learn from the organization. I wrote, “If we are going to democratize learning in organizations, we need to teach everyone how to learn and how to use tools to discover useful information. We need to make learning tools accessible to all employees when and where they need them.”

Developers of organizational learning tools, whether they be faculty, internal leaders, or free agents, have very few outlets for getting those tools to the people who need them in a fast and cost-effective way. Major publishing houses take too long, cost too much, and have limited distribution channels. Penelope Trunk addresses this problem in her blog post, “From PR to profits: the problems with publishing.” One could challenge her motivation for writing the piece, but her criticism of traditional book publishing, from my experience, is accurate.

Many tools, such as employee surveys, graphical representations of models, or steps to solving organizational problems, do not require a book. These tools could be made available in shorter forms and without all the trappings of a best seller. However, simply posting materials on the author’s Web site is not the answer either.

Another avenue for distribution is professional associations. These guilds develop and sell their own materials as a major source of revenue. They often work with a select cadre of member-authors and the products can be quite pricey. The American Evaluation Association is an exception to this with their AEA365 blog, a vehicle for any member to post an article and links to related resources. It is intended to be a free exchange of information and tools among members. The Employee Engagement Network is another example of a place to share information about tools for improving performance, specifically by increasing employee engagement. Although both sites serve an important purpose, neither is designed for authors to publish tools or for managers to find what they need when they need it.

It is wonderful that many free resources are available within professional communities. However, an alternative is needed where creators of organizational improvement tools can distribute those tools quickly and easily to any leader when and where those tools are needed.  And to make high quality tools available, creators must be compensated, even in a small way, for their expertise and effort. This is the space that my business partners and I hope to fill with Learning2BGreat.com.  Anyone who has created materials that can be used for organizational improvement may sell their wares from the site. The author decides what it is worth and sets the price. Anyone who needs a tool to help them help their organization may purchase and download the appropriate tool from the site. The tools are organized in topics and linked to information about the authors. L2BG is a virtual marketplace for connecting professional resources with the people who need them, on-demand. This ecommerce site is an experiment that we hope will continue to grow and have a profound effect on improving organizations of all types. 

Democratization of Organizational Learning

Organizational learning has, for too long, been owned by consultants and chief training and learning officers. Any effort to learn how to improve leadership, management, team building, communication, planning, or other organizational skills, has started with these external and internal gatekeepers. This has made learning generally inaccessible to the people with the greatest need. 

This is why Jim Stilwell and I have created Learning to be Great^tm. This Web site is an online
marketplace where internal leaders and managers, as well as external consultants and authors, can self-publish their organizational improvement products and where they can find practical, inexpensive learning tools when and where they need them. 

This rapidly-growing store has materials that can be used to:

• Enhance leadership
• Facilitate coaching
• Increase employee engagement
• Assess culture
• Improve decision-making
• Conduct strategic planning
• Evaluate programs and services
• Maximize organizational learning

Our intent is to de-mystify organizational learning and give all managers the tools they need to be successful. Learning is everybody's business. Everyone should be able to quickly find tools that will help them understand a situation, improve performance, and develop a new capability along the way. And people who have designed useful tools should have a way to get them to the people who need them. This is our hope for Learning to be Great^tm. Please let me know what you think of the site. 

Leader Learning Library

Online resources for leaders and wannabe leaders are quite extensive these days. We are witnessing the democratization of leadership, making the "servant leader" even more possible. Everyone can become a leader (as well as an effective follower) by learning about leadership when and where it is most convenient and applicable.

But it's not easy to find Web links that have been screened for quality and topics. Masters in Leadership has put together 100 Exemplary Sites for Future Leaders. And I have the honor of having my blog included in that list.

One of the very useful features of the list is the categorization of topics. These include:

• Leadership Coaching
• Team Building
• Business Leadership
• Women in Leadership
• General Leadership

While Masters in Leadership has created an excellent resource, that has something for everybody, as with any best-of list, each of us can think of people and organizations that we would have included. For example, I would have selected the sites of Wally Bock and John Baldoni for a hundred-best list. I hope Masters in Leadership continues to monitor the sites on their list and remove and add links as these Web-based sources of information change over time.

 

Community of Practice and the TorranceLearning Download

One of many methods of organizational learning is a “community of practice”. This is a term coined by Etienne Wenger to describe “…groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly.” TorranceLearning has created a kind of community of practice that they call the Torrance Download. The Download is a time and place for people to come together to share experiences and ideas, with the intention of everyone learning something new. 

I had the pleasure recently of attending another session of the Torrance Download. Once again the TorranceLearning team created a “…casual, fast-paced and friendly exchange of ideas with our friends, colleagues and clients in learning.” Between noon and 3:00 on a Friday, five concurrent, 30-minute discussions were held for a total of 20 topics. All of the topics were about learning and development and ranged from video production to elearning design to mobile learning to onboarding new employees. The tone, set by Megan Torrance, was one of fun, high energy, and engagement. This is a terrific model for a community of practice, either for people from different organizations or for people internal to one organization.

One type of Download discussion is called “Flash Project”. This is a leaderless conversation for the purpose of creating a list of answers to a specific question. I participated in one of these in which the
question was, “What is learning evaluation?” In 30 minutes, our group came up with an excellent list of ideas and questions that we shared with everyone in attendance at the end of the day.

We said that evaluation of learning is:

• Measuring business outcomes
• Establishing metrics for behavioral change
• Deciding how we want a learner to behave differently
• Thinking about individual, team, whole organization, and community learning
• Asking the learner, “What do you want to be known for?”

We also said that evaluation of learning is answering these questions:

• Did you like it?
• Did you learn it?
• Did you use it?
• Did it make a difference?
• Did you retain it?

And that evaluation of learning is measuring:

• How well organizational communications are aligned with pre, during, and post learning
• What managers are doing to facilitate and reinforce learning
• What resources (people, technology, money) are available to facilitate learning
• Where the gaps are between resources and learning
• Where people are starting from pre-learning
• What we know about learners' situations pre-learning
• What the organizational constraints are to learning and applying learning
• Why learning isn’t happening
• How to provide more opportunities for applying learning on the job
• What learning is still being applied months and years later
• Agile learning - Are learners able to extract useful information in any environment? What can be done to help learners become agile?
• What is being done to facilitate learning before, during, and after training

The striking thing about our Flash Project definition of “learning evaluation” is that we said that evaluation is much more than finding out what learners have in their heads at the end of a training program. We came to the conclusion that evaluation of learning should empasize achievement of business goals and it should measure the learner and learner’s situation before training, during training, and after training. And we agreed that the culture of the organization should also be evaluated because it is a critical factor in learning and application of learning.

Philanthropy, Learning, and Change-Making

The recent U.S. election reminded us on a daily basis of all the serious problems we face as a nation. Political campaigns made the solutions sound simple, as if one person or one policy could put people back to work, improve our educational system, deliver better health care, clean up the environment, or prevent war. It’s just not that simple. As Baldoni writes in his Forbes blog post, “Post-Election: Now What?”:

Now is the time to put the interests of the whole society first. That will require a commitment – indeed a partnership – of community, business, nonprofit and yes government.

Philanthropic foundations, although not specifically called out in Baldoni’s post, are also part of the solution mix. Steve McCormick, President of the Gordon and Betty Moore Foundation, writes that private foundations should be “game-changers”; they should be “…creating solutions to our most critical social problems.” He writes that philanthropists must do some things differently. One of these is measurement. He admonishes philanthropists to:

Use meaningful measures. It’s difficult to chart progress in the not-for-profit sector. Measurement requires creativity, coupled with discipline, to establish the right indicators—qualitative as well as quantitative. But, as the saying goes:  “if you don’t know where you’re going, you’ll never get there.”

Jeff Raikes, CEO of the Bill & Melinda Gates Foundation, in launching MarketsForGood.org (see below), said that foundations should be continually learning from doing. Conversations about learning do not happen as much as they should in foundations and their grantee organizations. They should be sharing information. Raikes says that the answers exist but it’s too hard to find and apply the information. Bigger, faster impact requires finding ways to get better data to social investors and nonprofits that are doing the work on the ground. He said, “Our work relies on the free flow of quality, accessible data and information…that lead to greater impact and a better world.”  

 

I agree with McCormick that private foundations can be “game-changers” and that creative, disciplined measures will help make that happen. And I agree with Raikes that much of the data and information already exists to increase impact on society’s most serious problems; that it is more a problem of access to information than it is one of collecting the data.

However, beyond applying measures and accessing data is the challenge of actually using the information to bring about change. This is where many foundations and nonprofits get stuck. They have the information in reports on their shelves or in computer files, but that’s where it stays. That information needs to be liberated, discussed, and translated into action. If they are going to be change-makers, they need to get their organizations engaged in conversations about the implications of the information, plan the action that must be taken to implement what has been learned, and hold themselves accountable for making change.

Opening the Black-Box to Evaluation

Black-box evaluation is measuring the effects of a program or service without examining the nature of that program or service. It is evaluating what people think and do after the intervention (e.g., training, classroom education, e-learning, coaching, policy change) and then attributing that behavior change or lack of change to what happened in the “black box”. Sometimes a comparison is made post-intervention to what people knew and did before the intervention.  However, nobody opens the black box to see what’s inside. Because of this, we don’t know what aspects of the program had the effect and why they had that effect. So, while we can be accountable for results, black-box evaluation doesn’t help us understand how to replicate those results in the future, we don’t know what should be continued and what should be eliminated, and we don’t know what should be changed about the program to get even better results.

My last two blog posts are about my recent work with Agahozo-Shalom Youth Village, a residential community for orphaned youth in Rwanda. That work provides a good example of the problem with a black-box approach to evaluation. If we only examine the output (e.g., number of graduates, national exam scores) and outcomes (e.g., enhanced self-confidence, enrolling in college or technical school, starting a business), staff would not have the information they need to replicate ASYV, to improve programs, and to possibly figure out how to get the same results at lower cost.  

                   

                              

What ASYV needs is a way to link the contents and process of their programs and activities with outcomes. In addition, they need to know how circumstances of each child before coming to the Village interact with Village activities to affect results. And they need to look for the causes of those pesky unintended consequences that might become a predictable aspect of learning and change.

One of the methods we propose for this evaluation is in-depth interviews of a small sample of students at their time of graduation, then 18 months later, and then six years after that. Much like Brinkerhoff’s Success Case Evaluation Method, the Village staff will use these interviews to tell the stories of the youth. The interviewers will listen for links between what the students experienced at ASYV and how they’ve developed as self-confident, independent adults. They will listen for how the change has occurred from the time the youth came to the Village to how they are today. They will look into the “black box”, which is the Village experience of informal and formal education programs as well as the support of house mothers, counselors, social workers, and other staff, and examine how those experiences contribute to intended and unintended results. 

Continuous Improvement in Agahozo-Shalom Youth Village

In my last blog post I mentioned my evaluation work with Agahozo-Shalom Youth Village in Rwanda. One of the tools we used to help the Village staff think about indicators of success was a model of the residents’ total Village experience. See below.

We asked the staff, “What indicates that you have been successful at each point in this model?” At "baseline", they said that it’s knowing students’ abilities and expectations when they first arrive at the Village. It’s understanding the circumstances they came from and how the Village is different? It’s knowing their hopes and dreams...if at that point they are able to hope and dream.

During the “process” stage, indicators of success include the level of participation of residents in their school and non-school programs. Measures of learning, examples of leadership, examples of volunteering to help others in the Village and outside, and examples of showing initiative were all considered evidence that youth are learning and developing into knowledgeable, self-sufficient, and productive adults.

Some of the indicators of “output” success are the number of students who stay in the Village for four years, the number who graduate from the High School, and the number who pass Rwanda’s national exams. A low incidence of health clinic visits and emotional problems are also indicators of success at this point.

“Outcomes” that staff think are indicators of success include attending the university or a technical institute, starting a business, getting a job, raising a family in a responsible way, and maintaining contact with other graduates from their cohort. Also, taking a leadership position in the community and “giving back” to their communities in other ways are indicators of success.

“Impact” success is apparent in graduates who complete a college education, who are running a successful business, who have an adequate income that is supporting a family, and who are giving back to their communities through teaching, service, and economic development projects.  

What the ASYV staff and I explored together is that, given their theory of change, success is measured at a number of points in the learner’s experience. If they waited, as so often happens in organizations, until after a program (i.e., learning intervention) has ended for learners, it’s too late for continuous improvement.