Opening of Artpark

Measuring Project Results

Most creative placemaking projects have results that are both tangible and intangible. While some of these results are easily observed and measured, many require more sophisticated evaluation techniques. Read on to learn how Our Town Project managers measured success.

Lessons Learned: 
Project Evaluation Most Often Happens In-House, So Be Prepared in Advance

Creative placemaking practitioners often rely on a combination of formal and informal tools to evaluate project outcomes.

Informal tools can range from collecting responses from project participants during events, to logging press coverage or media traffic.

Formal approaches often require additional resources to develop and implement measurement tools.  Ideally, projects create evaluation frameworks within their initial project planning phases.  Working with an evaluator from the beginning of the project helps you better understand your intended outcomes. It also allows you to build and integrate the appropriate data-collection tools into the project execution phase.

In Stone Mountain, GA, the ARTStation project worked to provide space for artists to create, exhibit, sell and share their artistic work with the community. Project managers developed a system whereby the team evaluated the artists – but also the reverse, with artists evaluating the project itself. “Staff of ART Station developed several evaluation tools," the NEA grantee organization reported. "The Artistic Director, the Gallery Manager, the Education Director formally evaluated each artist each month with an evaluation system that included quality of the work, sales, [ability to be a] team player, participation in educational activities, serving as an ambassador to the community, etc.  We also asked that each artist evaluate the program itself quarterly.”

More often than not, in Our Town projects the evaluations were handled by project team members.  If this is the approach used, then managers should identify early on who will be taking on that role and what resources they will be able to access. It is also important to identify not only what information needs to be collected, but how that information will eventually be used.  While evaluation data often are required for reports to funders, the same information can also be used strategically to tweak implementation of a project. Such data also can be used to garner wider community support.  Policy-makers often need to provide concrete reasons for why or why not to support an initiative. Evaluation data can be critical in helping to communicate the project’s vision in terms that elected officials can champion.

How To: Identify a Theory of Change

  • Project team members can get a head start in designing and implementing an evaluation strategy if they can identify a Theory of Change at the root of their creative placemaking project.  A theory of change establishes a clear line of reasoning for how a particular strategy or action serves the overall vision for a project.   It is a logic model that maps project activities to specific outcomes.  Deciding why particular actions are important, why certain neighborhoods should be involved, which types of partnerships should be fostered, and what impacts should be expected--these are all essential questions in developing a logic model that can serve as the foundation for a measurement strategy.  A list of resources for developing a specific theory of change can be found on the Animating Democracy website.
Measure What You Value and People Will Value What You Measure

Finding good measurement tools for a project evaluation can be difficult--especially when the intended outcomes are likely not to be realized for a long period after the project commences. For project managers not trained in the social sciences, the task may seem overwhelming.  Nevertheless, creative placemaking project managers have used some relatively simple, replicable strategies to evaluate their work. They often have chosen small, discrete variables that can be measured through surveys, direct observations, or other data-collection methods that capture general feedback from stakeholders and community members. 

In some cases, merely tracking unsolicited, informal feedback about their projects have given fresh insights to creative placemaking practitioners. In Portland, ME, for example, Marty Pottenger, who leads the Meeting Place project, created a specific folder for all emails she received that mentioned a specific reaction or idea related to her project.  She used this information to monitor the project’s progress and to take a retrospective view of the community’s understanding about the project as it evolved.

 

How To: Evaluate Community Meetings

  • Evaluation not only occurs at the level of a full-blown study; it can also take place during or after community meetings that are held as part of the project. For Meeting Place, Pottenger had specific techniques for evaluating each community meeting. “Our team set up an evaluative practice, where at the end of every event or meeting, we had an evaluative go-around. We called it a Final Go-Around. We asked each participant to name either a delight, a highlight, an insight, or a question. You get to hear back from every person, and it gives them a chance to think about their experience in an evaluative frame. It’s tremendously valuable – both for us and for them.”  Another technique she used was to “have sign-in sheets at the beginning of every workshop, where they would ask people to add a sentence, along with their name and contact information, about what had stuck with them from the past workshop. Was there any place in their life that – in the last month – something that came to mind or that they did differently because of the other workshops they attended? That was really helpful as well.” 
Be Open to Experimentation, and Let Evaluation Inform Your Processes

Reaching project goals can require taking unexpected turns.

In the Los Angeles Willowbrook project, the team converted this challenge to an advantage, embracing what its managers called “an iterative process in which outcomes influence next steps organically.”  Working creatively to assist an inner-city, low-income neighborhood in identifying cultural assets, the project team found that “it was both challenging and rewarding to adopt the iterative process in which outcomes were unpredictable. But this approach encouraged [us] to be more nimble and responsive.” Through constant evaluation and adjustment, project artist Rosten Woo was able to arrive at specific techniques for reaching out to community members that provided meaningful input -  including a "Home, Garden and Vehicle" tour, which Woo used to draw out the varied and complex stories of Willowbrook residents. As the team reports, “Rosten went door-to-door to invite residents to participate in a Willowbrook Home, Garden and Vehicle tour. That format enabled him to strike conversations with residents, learn about their hopes, dreams, anxieties, their impressions of their community and its evolution. The actual tour was a safe and approachable platform for residents to meet neighbors, share in all that is positive about Willowbrook and ultimately build mutual understanding.The stories and photography from Woo’s research for the tour became a tool to convey the community's distinct identity to planners and policy makers. It also became an important leave-behind at the local libraries and for residents, reinforcing the community’s awareness of its assets.”

How To: Set and Achieve Project Objectives

  •  It is an iterative process. As in the case of Willowbrook, other project managers can set clear objectives at the beginning of a task but then allow the specific method of achieving a result evolve as the team gains feedback from stakeholders and the larger community.  These practices help to more quickly identify success factors and to increase the potential impact of a project.  For information on how funders view this approach, see: Funder Exchange on Evaluating Arts & Social Impact.