Taking Note: On the Value of a Negative Finding

Last month saw the peer-reviewed journal Early Childhood Research Quarterly publish a “special section” devoted to the arts. The section was co-edited by my colleague Melissa Menzer and Adam Winsler, a developmental psychologist who is also a principal investigator of the National Endowment for the Arts Research Lab at George Mason University. (Go here to register for an October 29th public webinar about these arts-in-early-childhood studies.) When Melissa undertook this responsibility, I know she didn’t expect all the research findings chosen for publication to be uniformly positive. She already has authored a widely cited NEA report describing numerous positive relationships between the arts and social and emotional development in early childhood. But it would have been too much to expect all the studies that landed in this journal issue to favor a tidy or singular narrative about the arts’ benefits for children. First, the Good News… Across the eight published articles, positive findings are reported from two out of the four studies that tested a relationship between younger children’s engagement with arts education and some outcome of interest. In a randomized, controlled study, Brian Kisida, University of Missouri, et al., examined 2,253 students from kindergarten through second grade who did or did not take part in an educational program offered by the Crystal Bridges Museum of American Art in Bentonville, Arkansas—the site of other notable experiments by Kisida and his colleagues. (See this recent interview with co-author Jay Greene, University of Arkansas, a principal investigator of a NEA Research Lab that is extending this line of research.) Compared with students who did not partake of this experience, boys and girls who visited the museum as part of a field trip were “more likely to think trips to art museums were fun,” to feel “happy about art museums,” and to indicate “they would like to visit art museums with their family.” Weighing those results, the authors conclude that “if exposure to the arts in early childhood increases the likelihood of students seeking arts-rich experiences in later childhood, then early childhood exposure may lead to benefits that compound over subsequent years of schooling and adulthood.” Previous research also suggests this gateway effect. Analyses based on the NEA’s Survey of Public Participation in the Arts, for example, have revealed childhood visits to art museums or performing arts events as a key predictor of arts participation in adulthood. More recently, Kenneth Elpus, University of Maryland, has shown how school-based music education is positively correlated with adults’ levels of arts engagement—as creators and consumers. In the journal issue, a separate study involves 265 children, aged three to five years old, from economically disadvantaged backgrounds. One group of children attended an “arts-enriched” Head Start preschool (at Settlement Music School in Philadelphia) while another group attended a Head Start preschool in which the arts were not fully integrated. By the end of the study, children at the arts preschool showed greater growth in school readiness and greater self- and social awareness than did students at the non-arts-enriched preschool. According to the authors, the results “suggest that intensive arts integration can add to the value of Head Start.” Eleanor Brown, West Chester University, led the study, which was supported in part by a NEA research grant. The study builds on her team’s prior investigations of arts enrichment in Head Start preschool settings. Those earlier efforts have borne out the positive relationships between intensive arts integration and school readiness (e.g., gains in vocabulary and emotional expression and regulation). In another NEA-funded research project, Brown and colleagues have studied the potential of this program to reduce poverty-related stress. Comfort in Ambiguity So far, so good. Now we turn to two other studies in the journal issue. Like the previous studies already referenced, they were rigorously designed to examine the prospective benefits of arts education in early childhood. Their findings, however, were null. In research supported by the Administration for Children and Families within the U.S. Department of Health and Human Services, Wendy Mages of Mercy College studied the impact of a theater-in-education program on 155 children enrolled in a Head Start preschool in New York City. She learned that “all children in the study showed significant improvement on their language, perspective-taking, and imaginative abilities,” but that there was no measurable effect from the intervention itself. Nevertheless, Mages reasons, “in the current academic climate, in which an increasing focus on academics in preschool curricula can lead to the elimination of arts programming, it is worth noting that the inclusion of a [theater-in-education] program did not detract from the children’s acquisition of skills that contribute to school readiness.” That’s one way of putting it. Another article in the journal issue offers “the first randomized controlled study investigating whether more time in general music in kindergarten results in better executive functioning, self-perception, and attitudes toward school.” In the NEA-funded study, Jillian Hogan of Boston College, et al., compare two groups totaling 203 kindergarten children from primarily low-income backgrounds. One group received a weekly average of 45 minutes of a general music class, while another received two to seven times more minutes of the class per week. Hogan’s conclusion is stark. “Results fail to show an association between increased time spent in general learning spent and stronger extra-musical outcomes.” Admittedly, the study lacks a control group that would have received no music education whatsoever. (Citing previous reports that 94 percent of U.S. elementary schools offer some form of music instruction, the researchers claim that such a comparison would have been “very difficult” to achieve in a real-world setting.) There were also no baseline measures of the outcome areas under study; thus, one is unsure of any pre-existing differences in “extra-music” abilities, as far as the two groups are concerned. Still, the results are important for what they convey about the issue of “dosing” relative to the specific type of music education on offer. For example, questioning whether general music classes truly challenge executive functioning, the researchers observe that “while young instrumentalists enjoy the challenge of continually learning the fingerings for new notes, and adding to their repertoire of pitches, no such parallel exists in singing, the predominant activity in general music.” Beyond this aspect of the study’s relevance, a report of a null finding is always useful when confronting the strengths and limitations of specific arts programs or interventions. As Hogan and her team note, null findings are too often suppressed by journal editors, with the consequence that “results that are surprising (hence less likely to generalize) may overpopulate the research literature.” Also known as “the file-drawer problem,” the issue of publication bias is well-known in social sciences and behavioral research circles. Experts agree that more incentives are needed for academic researchers to replicate studies that indicate positive results. One of Hogan’s co-authors is Ellen Winner, who, with Lois Hetland, several years ago reported critically on the lack of robust evidence suggesting cognitive transfer from arts education into non-arts domains of learning. The results helped to debunk popular press accounts of a so-called “Mozart Effect.” Retrospectively, the research by Winner and Hetland (and, more recently, by the late Michael Foster of University of Alabama at Birmingham and by Jade Marcus Jenkins of University of California at Irvine—in another NEA-funded project) has strengthened our knowledge of arts-related benefits that are indeed more supportable. To quote the preface of Early Childhood Research Quarterly’s special section on the arts, “we need to learn from negative or null findings as we make decisions not only about research but also design and implementation of arts programs for children.” At the same time, there is a distinct problem of “limited assessment tools being available to get at the particular arts and extra-arts child outcomes and psychological constructs that might be affected by different types of arts participation,” the preface adds. Fortunately, one of the articles in the journal issue aims to present just such a measurement tool, the “Preschool Theatre Arts Rubric (PTAR)” designed to assess preschool children’s theater arts skills. According to research in the article, authored by Amy Susman-Stillman, University of Minnesota, et al., the tool is reliable and relatively easy to use. Further, “psychometrically sound measures, like the PTAR, can help early childhood researchers, teachers, program developers, and theater artists explore the potential impact of early childhood on young children’s school readiness and gauge program effectiveness.” Our office funded Susman-Stillman’s research, and we hope to spur more development of sound measurement strategies that can test other theorized benefits associated with arts education.