Measuring Cultural Engagement Amid Confounding Variables: A Reality Check

A Joint Research Symposium of the National Endowment for the Arts and the Arts & Humanities Research Council's Cultural Value Project, June 2-3, 2014

For many governments tasked with setting national policies or funding programs in arts and culture, surveying the public on participation has become a way of life. Yet it remains a costly proposition. In addition to the technical and logistical challenges attending any large-scale survey, the enterprise is beset by a wave of disruptive factors. Problems arise, for example, from competing definitions of arts and culture and, indeed, of participation itself. Other issues stem from overt or hidden assumptions about which types of activities, art forms, or cultural assets are privileged in survey questionnaires, and which populations or subgroups are envisioned as users of the survey data, and for what purpose or agenda.

For what purpose. The organizers of this research symposium believe that the original, policy-based motives for undertaking such data collections often go unexamined long after the systems are set in motion. Shifting policy imperatives are often articulated in pragmatic changes to survey questions and the presentation of results. A vexing question for the future of measuring cultural participation is whether current instruments and methodologies are flexible enough to accommodate rapidly evolving art forms or genres, changes in national and regional demographics, and emergent technology platforms. There is a further dilemma: how can the descriptive statistics culled from such data be linked compellingly with research (including other data sources) about the value and impact of arts and culture? For that matter, do periodic, cross-sectional surveys remain effective tools for learning about public participation?

The U.S.A.'s National Endowment for the Arts (NEA) and the Cultural Value Project of the U.K.'s Arts & Humanities Research Council (AHRC) convened a two-day symposium to bring together leading researchers and practitioners from both countries--as well as from other parts of Europe, Australia, and Canada--to conduct a "reality check" on the landscape of cultural participation metrics. The goals of the event were to probe our assumptions about how and why we measure public involvement in arts and culture, to confront any orthodoxies in how cultural participation is reported, and to chart a path toward more durable and meaningful measurement. Finally, the symposium identified pressing research questions and opportunities for standardizing certain data fields across international boundaries. 

The co-hosts of the event were Geoffrey Crossick, director of the AHRC Cultural Value Project, and Sunil Iyengar, director of the NEA's Office of Research & Analysis.

Speaker Biographies


DAY ONE: June 2, 2014

8:30 am


Welcome  and statement of purpose [VIDEO]

Jon Clifton, Managing Director, Gallup World Poll

Geoffrey Crossick, Director, Cultural Value Project, AHRC

Sunil Iyengar, Director, Office of Research & Analysis, NEA

9:00 am


Keynote speech: Bob Groves, Provost, Georgetown University [VIDEO]

9:45 am


Session 1: Why measure cultural participation, and for and by whom? [VIDEO]


Tom Knight, Deputy Head of Analysis, The Department for Culture, Media & Sport, UK

Diane Ragsdale, Faculty, Erasmus University Rotterdam

E. Andrew Taylor, Assistant Professor, Department of Performing Arts, American University

Moderator: Josephine Ramirez, Program Director, The James Irvine Foundation

  • Potential questions for discussion: What are the different motivations behind conducting surveys of arts participation at the national level? What sort of information would be most useful for arts practitioners versus policy-makers and the general public? To what extent does a focus on attendance habits pertaining to arts "institutions," typically those supported by public funds,  potentially distort our understanding of participation? Are there any common patterns or trends that demand changes to the ways that arts participation is measured and reported?

11:15 am



11:30 am


Session 2: What do we mean by cultural participation? Scrutinizing activities and genres [VIDEO]


Abigail Gilmore, Senior Lecturer, School of Arts, Languages and Cultures, The University of Manchester

Bridget Jones, Director, Research & Strategic Analysis, Australian Council for the Arts

Alaka Wali, Curator in Anthropology, The Field Museum

Moderator: Alan Brown, Principal, WolfBrown

  • Potential questions for discussion: How does one select or prioritize among various visual, literary, performing arts events, including museum-going and other cultural activities (such as creating art or learning an art), for the purpose of constructing a survey about arts participation? What role can arts communities have in helping cultural researchers and policy-makers decide which art forms and activities are worth tracking regularly? How do we account adequately for activities related to "cultural heritage" or "culturally specific art forms"?  Have policy imperatives in driving participation surveys reinforced an unhelpful hierarchy: subsidized professional, then commercial, and finally amateur arts? How important is it to distinguish between people participating as audience members and as producers?

1:00 pm



2:00 pm


Session 3: The challenge of encompassing new media-and technology-driven forms of participation [VIDEO]


Johanna Blakley, Managing Director & Director of Research, The Norman Lear Center, University of Southern California

Alan Freeman, Special Advisor, World Cities Culture Forum

Doug Noonan, Director of Research, Indiana University Public Policy Institute

Moderator: Kristen Purcell, Associate Director of Research, Pew Research Center's Internet & American Life Project

  • Potential questions for discussion: What unique measurement issues arise with regard to capturing rates of arts participation via digital media and technologies? Does the ecology of participation change in a more complex world of platforms and new types of engagement?  Is there a danger that the genuinely participatory dimension of cultural engagement in a digital cultural world, blurring the conventional distinction between producers and audiences, is not captured by surveys? What relatively new art forms or genres must be accounted for by any comprehensive measurement? How can we ensure that such survey instruments stay current with these media and technologies?

3:30 pm



3:45 pm


Session 4: New ways of knowing: alternative data sources, methodologies, and units of analysis [VIDEO]


Hasan Bakhshi, Director, Creative Economy in Policy & Research, Nesta

Anthony Lilley, CEO and Chief Creative Officer, Magic Lantern

Mark Stern, Kenneth L. M. Pray Professor of Social Policy and History and Co-Director of the Urban Studies Program, University of Pennsylvania

Moderator: Joan Jeffri, Director and Founder, Research Center for the Arts & Culture, National Center for Creative Aging

  • Potential questions for discussion: What methods beyond household and individual surveys currently exist for capturing arts participation rates? How can both public and private (i.e. commercial) data sources be brought together to inform a fuller view of individuals' arts participation habits? Is there a place for ‘big data’ and ‘open data’ in our methods? What is the value of more geographical flexibility in defining the appropriate units (e.g., nation, city, neighborhood or locality) for the objectives of both policy and greater public understanding? Which units of time are optimal for taking such measurements, and what periodicity should such data collection methods serve? What potential do time diaries and/or longitudinal study designs extend to such research? How might we introduce arts participation variables into other longitudinal and cohort studies (e.g., epidemiological, health, household expenditures)?

DAY TWO, June 3, 2014

8:45 am


Welcome and recap, Geoffrey Crossick, Director, Cultural Value Project, AHRC [VIDEO]

9:00 am


Keynote speech and discussion, Jon Clifton, Managing Director, Gallup World Poll [VIDEO]

9:45 am


Session 5: Beyond participation rates: understanding motivations, barriers, and outcomes [VIDEO]


Kelly Hill, Founder and President, Hill Strategies

Maria Rosario Jackson, Senior Advisor, The Kresge Foundation

Nanna Kann-Rasmussen, Royal School of Library & Information Science Copenhagen, University of Copenhagen

Moderator: Peter Linett, Partner, Slover Linett Audience Research

  • Potential questions for discussion: How can measurements of arts participation also address changes in resultant outcomes for audiences and communities? How important is the question of motivations, attitudes and barriers regarding arts participation and how can these variables be captured effectively in a single instrument? Can audience marketing survey practices and methodologies help us here? Is there potential for using social media as another way of capturing "revealed preferences"? What role can subjective well-being studies play in this research?

11:15 am



11:30 am


Guided brainstorm exercise: making comparisons across countries [VIDEO]

Moderator: Lydia Deloumeaux, Assistant Programme Specialist for Culture Statistics, UNESCO Institute for Statistics

1:00 pm


Closing remarks and next steps, Sunil Iyengar and Geoffrey Crossick