Informal learning in organizations is finally being recognized as a key aspect of employee development and performance improvement. But how do you evaluate something that is as unpredictable and serendipitous as informal learning? The following post first appeared on this blog in November 2010.
Learning occurs in "informal" (another term that I don’t like but is gaining traction) settings and online, as well as in the classroom. How can we know what is learned informally and what difference that learning makes to the success of an organization? Some of the methods being used in informal science education could be applied more broadly to informal and online learning in business, nonprofit, and government settings. Kirsten Ellenbogen describes some of these methods on the American Evaluation Association blog. She writes about “embedded evaluation”:
Embedded evaluation integrates “invisible” tools for evaluation into the existing program activities. For example, in a youth program that has a focus on interactive media, the projects produced by youth are posted to online environment. http://info.scratch.mit.edu/ScratchR:_The_Online_Community The projects can be downloaded by others, modified, and reported. All activity in the online community can be tracked, and the ongoing development of the youth’s projects can be analyzed for more detail.
And she writes about videotaping users:
For example, videotaping a museum visitor using an exhibition (using IRB-approved informed consent as appropriate). In the post interview, after the initial set of questions, show the visitor a video segment of his or her interactions with the exhibition that was taped just moments before. Use a semi-structured interview approach and ask the visitor to narrate their video and tell you more about what they were doing. This approach can become somewhat automated using technologies like Video Traces. http://depts.washington.edu/pettt/projects/videotraces.html
She also suggests apps that can be used to log and document experiences and reactions to experiences on a mobile device (smart phone, iPad, etc.).
These evaluation tools could easily be applied in work settings. Employee teams could post the progress and results of their projects on a Web site (e.g., internal wiki). Managers could follow these results for the purpose of assessing impact of team learning. Employees could be videotaped while applying a newly learned skill and then those tapes could be reviewed with the employee to assess learning and impact. Workers could be asked to use an app to log their reactions to a benchmarking activity and then discuss that experience with their supervisors. New technologies are offering many useful ways of monitoring and assessing the impact of “informal” learning.
How do you measure the impact of informal learning experiences?