A Digital Storytelling Approach to Participatory Evaluation of a Community-Based New Media Project
Abstract submitted to IAMCR for Human Rights Communication Conference, July 21-24, 2009 Mexico City
This paper presents the design and preliminary results of a visual storytelling approach to the participatory evaluation of a community-based new media project. Mobile Voices is a Communication For Social Change project that develops low cost, open-source mobile phone and internet-based storytelling platforms to enable the participation of low wage immigrant workers in the digital public sphere. Through a series of popular education workshops, workers have learned to use their cell phones to upload multimedia stories onto a designated open-source internet site.
The Mobile Voices project is grounded in the popular education methods of Freire that emphasize reflection and action-learning, drawing on participants’ own lived experiences, to empower them to affect change. The evaluation design presented here is the result of efforts to develop monitoring and evaluation tools that both reflect these popular education goals and are appropriate to the digital storytelling context of this project. The design is informed by participatory evaluation methodologies that have arisen from recent critiques of evaluation research. Conventional evaluation methodologies have been criticized for lacking sensitivity to the complex and unexpected directions of social change. Alternative international development theorists and practitioners have also criticized the top-down nature of evaluation techniques that privilege expert knowledge (and thus dismiss local knowledge). This paper will discuss the theoretical and epistemological backdrop to participatory evaluation and how it has influenced the methodology developed by Mobile Voices.
The Mobile Voices evaluation design was inspired by Davies’ Most Significant Change (MSC) technique. MSC is a qualitative, participatory approach to evaluation that uses storytelling to help stakeholders identify and understand project impacts. It involves the production of stories of significant change by program participants, followed by a participatory interpretation process during which stakeholders discuss the value and implications of the changes captured by the stories. In contrast to conventional evaluation, MSC is not driven by pre-determined indicators and is sensitive to unexpected outcomes, thus facilitating the involvement of all stakeholders in creating a shared understanding of program impacts. While MSC has been gaining popularity in international development, most applications have relied on written stories, despite MSC’s apparent compatibility with other forms of storytelling including drawings, folk theater, video, and digital storytelling platforms such as that provided by Mobile Voices.
The Mobile Voices evaluation design puts a digital spin on MSC, thus streamlining evaluation methods with popular education and storytelling activities that are already part of Mobile Voices workshops. This paper presents the evaluation design for Mobile Voices as a case study. The study is situated within a theoretical framework that analyzes and critiques epistemological assumptions that underlie both conventional and participatory evaluation approaches. Assumptions within the international development discourse about how “legitimate” programmatic knowledge is created and acted upon – and by whom- are also assessed. Preliminary results and experiences in implementing the Mobile Voices digital storytelling MSC approach to evaluation will be presented. Implications for future applications of MSC and participatory evaluation approaches for Communication for Social Change interventions will also be discussed.