Taking Note: New Report Elevates Rigor of Arts-Integration Studies in Education Settings

Since 2015, the National Endowment for the Arts has incentivized researchers and arts practitioners to conduct rigorous studies comparing the arts with other types of activities, programs, and interventions—or with none at all—in a bid to establish causal evidence of the arts' effects. Whether achieved through a randomized, controlled trial (an RCT) or a retrospective data analysis involving matched-comparison groups, such research is relatively scarce within the arts sector. Even when they have been undertaken, the studies sometimes are not statistically powered to permit generalizations of the findings to different subgroups. In other cases, little to no qualitative research has gone into developing a theory about the underlying mechanism of change resulting from the arts practices; accordingly, the proposed outcome measures might be inappropriate for the study. Notwithstanding these obstacles, more and better experimental and quasi-experimental studies of the arts are of the essence. Federal clearinghouses of evidence-based programs to bring about social change (e.g., in education delivery, positive youth development, or crime prevention) all rely on research or evaluation models using these study designs. A robust comparison- or control-group study (ideally with random assignment of the “treatment” and “non-treatment” groups) can win government agencies’ endorsement of a "best practice," allowing a particular delivery model to vault over other programs and services and become adopted by the broader community of funders and policy-makers. The latest authoritative document to capture this dynamic, at least from an arts-related perspective, has been produced by Meredith J. Ludwig, Andrea Boyle, and Jim Lindsay of American Institutes of Research (AIR). Commissioned by the Wallace Foundation, the November 2017 report is titled Review of Evidence: Arts Integration Research through the Lens of the Every Student Succeeds Act (ESSA). Examining reports and articles (including “gray” materials) that have appeared since 2000, Ludwig et al. conclude that 44 arts-integration programs covering different student populations (from pre-K to 12th grade) have generated results that warrant each program’s inclusion in one of four “tiers of evidence.” The four tiers are stipulated by ESSA (Title VIII, Sec. 8101). The first and second tiers, respectively, rely on “strong evidence from at least one well-designed and well-implemented experimental study” and “moderate evidence from at least one well-designed and well-implemented quasi-experimental study.” Tier III programs must show “promising evidence” through correlational data, including statistical controls for selection bias; and Tier IV requires at least a theory-based rationale for the program. According to the AIR report, the majority (34 out of 44) of arts-integration programs fall into Tier IV. These include, for example, the Kaleidoscope Preschool Arts Enrichment Program, which has been evaluated by West Chester University’s Eleanor Brown, the lead author of a paper that resulted from an NEA research grant. Tier IV also includes Arts Integration with Science Lessons, a program and study led by Johns Hopkins University’s Mariale Hardiman, who had shared her findings in an NEA public webinar. Among the nine arts-integration programs falling into Tiers II and III are offerings by Global Writes, which previously has received NEA research grants to evaluate its work. This leaves only one arts-integration program in the coveted Tier I bracket. It is the Crystal Bridges Museum Field Trip in Bentonville, Arkansas. Jay Greene of the University of Arkansas is the lead investigator on studies associated with the program. Greene now directs an NEA Research Lab, in which capacity he recently blogged about challenges and opportunities for arts and cultural organizations seeking to do experimental studies. Among other projects, the NEA Research Lab is currently investigating “the effect of multiple arts experiences on the long-term social/emotional outcomes of urban elementary school students,” as the Lab website explains. Former colleagues of Greene’s—Brian Kisida at the University of Missouri and Daniel Bowen at Texas A&M University—are now collaborating on a separate NEA-funded research project. Texas A&M is running a randomized, controlled study “to determine the casual impacts of arts enrichment, facilitated by partnerships with schools and community cultural organizations, on students’ academic and social outcomes,” according to the NEA grant description. That study will be conducted with elementary and middle schools in the Houston Independent School District. Beyond contemplating more opportunities for such research, arts educators who read the AIR report will learn about “at least 12” distinct funding opportunities through ESSA for the implementation of arts integration in pre-K through Grade 12 learning environments. Another highlight from the report—at least for this reader—is the nifty diagram on p. 9. Can you say logic model?