Spy Hop, a youth media organization, needed an evaluation framework tailored to their unique Youth in Care programs. These programs serve vulnerable youth in foster care, residential treatment centers, and secure facilities—contexts distinct from their afterschool and community programs. The existing logic model did not align with the pedagogies, practices, and outcomes of Youth in Care.
Spy Hop asked Convergence Design Lab to help create a framework that would support program design, evaluation, and grant-writing. This task came with a broader challenge: overcoming skepticism about evaluation, which is often seen as a tedious or disconnected exercise by program staff.
Convergence Design Lab aimed to make evaluation engaging, participatory, and reflective of Spy Hop’s culture, a culture that is uniquely playful and fun. Knowing that we would be designing a virtual interactive workshop on Mural, a digital whiteboard, we used principles of game design to drive participation and engagement. We drew inspiration from MadLibs, the classic silly story-building activity, in order to build micro theories of action. MadLibs is low-stakes, and invites players to come up with words first, before putting them into any sort of order or story.
During the workshop, participants were provided ample space and time to share reflections and ideas. We presented the first activity in the form of a challenge —build a success profile. Our prompts:
The stories unearthed during this process were moving, rich and detailed. Participants easily shared success in their own words, shored up through personal experiences and everyday observations. After capturing these Outcomes, we then introduced the concept of Inputs. How did these outcomes occur? What were the essential ingredients that created success?
The use of the word “ingredients” invited people to conceptualize the learning ecology as a signature dish comprised of many components working together to create a lasting and flavor-filled experience.
Sharing stories in this way presents an all-too-rare opportunity for practitioners to hear from one another as make new connections across their individual practices and experiences. Drawing from these word lists (Success stories, outcomes and ingredients) that teaching artists developed in the workshop, we then led them through the Mad Libs process to self-author their own theory of action statements using this simple template:
The teaching artists jumped right in, and in lock step, focused on a series of theory of action statements that were descriptive, thoughtful and written in their own words. Convergence was able to synthesize their Mad Libs into one cohesive theory of action after the workshop.
Overall, our participatory evaluation planning process drew on three of our guiding design principles:
Our collaboration produced a comprehensive framework for Youth in Care programs, delivered through a slidedeck and analytic memo. Key elements included:
The framework resonated with Spy Hop staff. One mentor noted, “Being a new mentor, I learn a lot from meetings like this, and I am able to take this back into the classes that I am teaching.” In 2021, Spy Hop secured a prestigious National Endowment for the Arts research grant, leveraging the Youth in Care framework. This research has since been published in journals and presented at conferences, elevating Spy Hop’s work to national attention.
This participatory approach to evaluation strengthened Spy Hop’s community of practice and provided an actionable, authentic framework for Youth in Care programs. It demonstrated how evaluation, when done collaboratively, can align with an organization’s mission and inspire meaningful insights.
Our flexible and client-centered approach ensures that the final outcome is perfectly tailored to meet your organization’s unique vision of success.
