To Measure or Not to Measure? That is the Question
How do we measure student achievement? How do we quantify learning? This story is part of a series revealing how schools address these questions.
BY MAURA WALZ
Can good acting be measured? Can the emotional impact of a piece of music be quantified? Is it possible to assign a concrete score to an ephemeral art form?
One teacher at a visual and performing arts high school in Queens says it can. He and other educators believe that quantifying the arts may help rescue them from the curriculum cutting room floor.
Drama teacher Robert Frisch has developed several rubrics for measuring students’ performance and writing abilities during his six years of teaching at the Frank Sinatra High School for the Arts in Long Island City. His tables and score cards may suggest the future for arts education in New York City.
“The public perception of drama has always been that it’s something sort of mysterious,” said Frisch. “How do you assess success in an art form that has a bizarre, non-conventional, subjective association to it?”
The answer, he added, is found by breaking down its complex parts into component elements, skills and professional standards. There is nothing mysterious about it. Discipline and free expression have always existed side by side in the arts. Frisch has incorporated that pairing into the way he teaches and evaluates his drama students.
On one afternoon in February, the tone of Frisch’s ninth-grade drama class musical rehearsal seesawed between focus and freewheeling chaos.
“It’s like screaming banshees here,” exclaimed one student, his voice punctuating the rowdy crescendos of the class’ vocal warm-ups. Moments later, the group harnessed its energy and delivered a lively and nearly note-perfect round robin version of the gospel tune, “This Little Light of Mine.”
After their February performance of this Black History Month revue at the Queens Public Library, students used one of Frisch’s rubrics to evaluate themselves, filling in a table and ranking each element of their performance:
How well did you memorize your lines? (Rate 1 to 4, with 4 being the highest level of success.)
How well did you inhabit your character?
How consistent was your delivery over each performance?
Frisch scored each student as well. He then added up the ratings for an overall score. This final number is added to other ratings each student has accrued over the course of the semester.
“By the end of the marking period, you’ve accumulated enough rubrics to have a very thorough evaluation of each student,” he said.
The nearly 650 high school actors, musicians, dancers and visual artists at Frank Sinatra choose this public school because its curriculum is designed around a rigorous arts program coupled with a more traditional academic instruction. Arts education is its raison d’etre.
But for students in schools with less entrenched arts programs, increased standardization of arts instruction through rubrics like Frisch’s may hold the key to solidify the place of music, art and drama in their curriculum. In a public school system that increasingly values only those skills that can be tested, many believe educators must find a reliable way to measure student artistic success or risk marginalizing arts education.
Arts educators sometimes struggle to convince principals of schools without an arts focus to strengthen their programs when art does not fit into the standardized test regime that measures a school’s performance, said Sarah Calderon, executive director of the Casita Maria Center for Arts and Education, who worked as an advisor to the New York City Department of Education in 2006 to create a plan for accountability in the public schools.
“To say that art is special and ephemeral—it doesn’t help the situation,” she said.
Calderon observed that in an environment where schools are held accountable for students’ performance, the subjects that are graded and tested in the citywide assessments attract the most attention from principals and teachers, and other subjects fall by the wayside. She said that creating data-based ways to evaluate teaching and learning in those other subjects—and then holding schools accountable for those outcomes—might be the best way to ensure that subjects like the arts are not ignored.
“I think that because something’s hard to measure, people decide that it’s different and impossible to measure,” Calderon said. “I think that you can figure out how to measure it if that’s what you want to do.”
In 2004, the education department’s Arts Education Task Force developed a plan for holding schools accountable for arts education in stages called “Blueprints for Teaching and Learning in the Arts” for dance, music, theater, and the visual arts. The blueprints set standards for what students should know and be able to do in these fields, according to the education department’s guide for principals.
In 2007, the department created ArtsCount, its program to evaluate the quality of arts programs in New York City schools. The program evaluates schools on an array of factors, including the number of students who graduate with New York State Regents diplomas in the arts and metrics for school spending on arts teachers, supplies and partnerships with outside cultural organizations.
So far the program does not incorporate student performance on any rubrics outside of the Regents diploma program, though Calderon said that she hoped one day the program would incorporate quantitative measurements of student learning.
“In many ways, it’s not ideal because it doesn’t look at those outcomes,” she said. “But at the same time, it provides us with data we’ve never had before. It gives us an idea of what’s happening.”
Doug Israel, the director of Research and Policy for the Center for Arts Education in New York, said that the ArtsCount program does not yet go far enough in holding schools accountable for arts education primarily because arts metrics still have only a small weight in schools’ overall scores. Since arts education is still relatively unimportant to the overall evaluation of schools, principals will continue to divert money away from arts to other subjects such as math and reading.
“If the goal is to develop a real rubric, why not report that as part of the progress report?” Israel said. “Don’t bury the arts.”
Calderon said that the education department is unlikely to rank arts higher in their report cards until measurement of quality arts education is more standardized. For that, she pointed to more rigorous development of rubrics.
“Right now the rubrics are mostly teacher-developed,” Calderon said. “So you don’t know if teacher A’s rubric is really good and rigorous and teacher B’s is not.
“My goal is always equitable access to arts education,” she continued. “How do we know that student A is getting the same education that student B is getting?”
Standardized scoring does not necessarily contradict the goals and content of school arts programs, said Steven Seidel, director of the Arts in Education Program at the Harvard Graduate School of Education, but evaluators should proceed with caution.
“A rubric is just an attempt to break down a process or activity into different dimensions, and so in and of itself it’s not the antithesis to the creative aspect of arts education,” Seidel said.
Seidel said that he first encountered rubrics as an educational tool in the late 1980s, when educators began using them as a tool to assess writing exercises. Use of rubrics increased as school reformers began to hold schools accountable for measurable outcomes in student performance across schools and programs, he said.
A consistent rubric means you can point to what students have learned, and how well they absorbed it, Seidel said. “From an accountability perspective, that’s necessary,” he said. “From the perspective of the authenticity and integrity of their art, it gets trickier.”
Complications arise depending on how the rubrics are used. If evaluators, students, teachers and schools all recognize the subjectivity involved, then rubrics are useful tools for student feedback, Seidel said. If they don’t, the rubrics are less useful.
“I need to know as a young actor what my teachers’ opinion of me—whether that’s brilliant or terrible—is just one person’s opinion,” he said. “Part of the trick in all of that is to understand subjectivity and to try to look for the inter-subjective truths.”
Back at the Frank Sinatra School of the Arts, drama teacher Frisch said that rubrics are a fair way of judging the impact of an artistic work or performance in a way that anyone, not just other artists, could understand. He recalled the evaluations of last year’s senior class testing for their arts-focused Regent’s diplomas, which used rubrics that were scored by himself and by the school’s principal, who does not have a background in drama. His scores and the scores assigned by the principal were never more than five to seven points apart, he said.
“There are things that an audience member instinctively knows,” he said. “You know when you feel engaged. You forget that you’re watching a performance. That’s success at a very high level.”