Accounting for Audience Impact—What Were They Thinking (and Feeling)!?
Recently, I attended one of the workshops associated with the rollout of a new book, Counting New Beans: Intrinsic Impact and the Value of Art. The book includes results of “intrinsic impact” research commissioned by Theatre Bay Area and conducted over the last two years. It also features interviews with artistic leaders as well as patrons and essays from Arlene Goldbard, Clayton Lord, Rebecca Novick, and Diane Ragsdale.
At the event, Clayton Lord (Theatre Bay Area) and Alan Brown (WolfBrown) discussed results from 19,000 surveys conducted in 18 theaters nationwide to measure the effects of theatrical productions on audiences. The workshop included perspectives from local theater reps who participated in the surveys.
Applying subjective values to rigorous research about the arts is never easy. But quantifying intrinsic impact presents special challenges: How do we assess impact? How do we interpret the results? How can we develop standardized measures to cover the many different types of art and its intended effects?
In this study, metrics for intrinsic impact included five constructs: captivation; intellectual stimulation; emotional resonance; aesthetic enrichment; and social bridging and bonding. In addition, there were questions related to audience members’ “readiness to receive” the performance, including advance preparation (e.g., reading a review), anticipation of the event, and the degree to which the arts activity is “relevant” to them (e.g., their likelihood of attending other professional theater performances).
The conceptual framework was translated into survey questions, such as the following:
- Frequency of Attendance: In a typical year, approximately how many times do you attend [the theater company’s] productions?
- Anticipation: Overall, how much were you looking forward to this performance?
- Captivation: Overall, to what degree were you absorbed in the performance?
- Emotional Resonance: Overall, how strong was your emotional response to the performance?
- Motivations: Respondents were given a list of 11 reasons why people attend theater performances and asked to choose three. (Ultimately, the top three reasons cited were: “to relax and escape,” “to be emotionally moved,” and “to discover something new.”)
The speakers stressed that the study was “not a contest”; higher levels of impact, as recorded for one construct, are not necessarily better than higher levels recorded for another. Indeed, one would expect the impact levels to vary by type of performance—a trend borne by the data. For example, audience members for plays reported higher levels of intellectual stimulation and social bridging, while audience members for musicals reported high levels of feeling “emotionally charged.”
The opportunity to learn more about the survey results and its implications were motivation enough to attend to the workshop, but I had an additional reason: at the NEA, we are undertaking efforts of our own to measure intrinsic impact. In 2010, the NEA developed a strategic plan that includes development of new metrics of the value and impact of the arts on communities and individuals. As part of this goal, we are working with WolfBrown’s team to develop questionnaires and protocols to gauge the extent and nature of audience responses to NEA-funded events. Currently in the pilot phase, the resulting surveys and methodologies will be implemented for a larger, nationwide study to be conducted in 2013. The surveys will include audiences of most types of grantees we fund, including: music; theater; opera; dance; media arts; folk and traditional arts; visual arts; literature; and museums.
There are a number of ways the data from these surveys will be beneficial. First, the data will help fill an existing research gap about the extent to which we are engaging audiences through events the NEA funds. This will lead to a fuller understanding of the arts’ impact, complementing our current and future data-collection efforts.
Still, throughout the planning process for these surveys, we have always kept broader goals in mind for maximizing their value. Our hope is that the surveys will not only yield the valuable information we need, but will serve others outside the agency. Providing survey results to the grantees will enable them to learn more about the impact of their work, as well as the characteristics, perceptions, and attitudes of audiences experiencing such art. In addition, we will make templates for the questionnaires publicly available so other organizations can adapt them for their own uses.
For people interested in learning more about intrinsic impact and related research, we have made a literature review available on the NEA website. Late this summer, we plan to hold a public webinar to discuss our findings from the pilot and the next steps for the larger study. We look to forward to learning the results and to sharing them with you.