Case Study

As the math department chair at Elkhart Memorial High School, Alex Holtz didn’t need to be convinced of the power of numbers. So he was pleased at the commitment and leadership district administrators demonstrated when they launched a data-driven decision initiative for teachers in 2004.

“Across the country there is a lot of pressure on schools to improve student achievement. What worked two generations ago in terms of instruction of 75 percent of the students isn’t good enough any more,” says Stephen White, a professional development associate at the Littleton, Colo.-based Center for Performance Assessment. The attention means school districts of all sizes are jumping on the data-decision bandwagon seeking answers to what works best and proof that a specific method is effective.

But what makes Elkhart Community Schools in northern Indiana stand out, White says, is its long-term commitment to get it right, even when that translates to still more changes.

Finding the Numbers

Fundamentally, the district has built a SEQUEL database to sit beside its student management system — a repository to store student demographics, test scores, grades, attendance information and disciplinary information. It’s updated daily and “our database manager spends a lot of time error-checking to make sure it’s clean,” says Daniel Rice, director of technology.

Teachers log in to the data via the secure Web portal — which requires two servers, one facing in, the other facing out — that checks their clearance credentials and admits them to the section of the student management system that corresponds with their class. The screen initially presents top-level information to tease the instructor into digging deeper — for instance, she might see a child’s standardized math score sitting next to that student’s math grade and notice a discrepancy.

“What we are trying to do is get data formatted in such a way that people see patterns,” Rice says. “If you’re looking at your class and see that one child has been absent 10 percent of the school days, that needs to be flashing in your face before it’s too late to do something about it.” Ditto the student who has six different school enrollment entries by the time he enters junior high.

When a teacher digs into a curious top-level category, a clearer picture should emerge. Say a student boasts a 190 summary score on math assignments, compared with a second child at 180. Drilling into that 180 score may reveal that his assignment scores were actually 200, 200, 200, 50, 200, and 200 — but for that anomaly score, that student would be on par with the first student. That’s a horse of a different color from a child who is struggling.

“The teacher’s goal now is to ask, ‘What did that child trip over and if I throw my coat over that mud puddle, can I help?’ ” says Rice.

Of course, none of this student information is new to Elkhart schools, but previously teachers and administrators searched through multiple filing cabinets to extract it.

Elkhart Community Schools chose to build its database in-house despite a cadre of vendors willing to tackle the job. “They’d say, ‘We can do A, B, C and D, and it will look really great.’ But E and F would cost more,” says Rice. Since he couldn’t look into a crystal ball and correctly anticipate the types of data the district may want in the future, it became more cost- effective to hire another programmer.

It also gave him the freedom to integrate data into standard spreadsheet formats. Now when teachers log in to view their data, they can click on the “export to Excel” button to work with the information in a familiar context.

Although such a system could track thousands of fields per student, Elkhart administrators are shooting to keep their records to fewer than 15 fields each. “Absolutely no more than that,” says John R. Hill, director of curriculum and instruction. “We are satisfied with what we have — the idea is to focus in tighter and tighter on smaller amounts rather than increasing the number of categories. That is exactly the wrong way to go. We can flood a system like this and incapacitate it.”

The Hard Part

But data for data’s sake adds up to just numbers. “The secret to data-driven decision-making that I think really energizes schools and helps teachers find themselves in terms of their craft is this: When teachers work together on collecting data and then get around the table to analyze it and talk about the results, that’s where the magic happens,” White says.

So although step two in Elkhart’s journey isn’t as clear-cut at the rank-and-file level, leaders like Hill assuredly have a road map. His role early on was to design instructional workshops and insist every teacher and administrator attend. “This is not an if or when kind of thing,” he notes.

And Rice recognized that he’d have to sell the idea, too. “Because the reality is that schools are data rich as a matter of fact — flooded with data and always have been,” he explains. “But using it to make instructional decisions, that’s the new part. And actually refining out of all of the literally thousands of pieces of data that could be put into use which ones really make the most difference in terms of instructional decisions — that’s the part everybody has to learn.” With just under two years’ worth of practice at this, Rice admits some data teams are still more proficient than others, but he has succeeded in one sense: 980 out of 1,000 teachers have been through the data training.

He’s patient. He continues to ask teachers to meet in collaborative groups of four to seven members at least once a month — by grade at the elementary level; by subject area in the upper grades — and follow a five-step structured script:

• Obtain the data around a particular standard or subject the group needs to know, e.g., how many students are proficient at that time?
• Look at student work to derive the traits of proficient work versus not-yet-there performances.
• Set goals that determine what the teachers are working on with this age group, what they want them to do, how they will measure results and when. For instance, the group may want students to be proficient at word math problems by Oct. 31.
• Assign structural strategies to help students master the topic.
• Determine how the group will recognize when students are proficient by setting exact expectations. “This is the most critical step,” says Hill. “This is how we know when students have arrived.”

He says the richness of such a collaborative dialogue outstrips any gains a teacher can make by poring over the data individually. Holtz agrees, which is why his advice boils down to stay committed to communication and collaboration. “I’ve been a teacher for 11 years and have seen a lot of fly-by-night ideas. This isn’t one of them,” he says.

The fourth-grade teachers at Bristol Elementary School used the data to determine standards they wanted to see their students clear in math and science. Jeff Antioho, one of those fourth-grade instructors, posts his student’s aggregate test scores on a chart inside the classroom and issues each child his own folder to keep track of data showing improvement or a standstill. “We tried to keep individual scores private, but the kids get so excited, they’ll start yelling, ‘That’s my score! That’s my score!’ when I put them up,” he says. The group also repeats the numbers on a chart in the hallway so parents and other administrators can see where the grade stands.

“The program has focused teachers’ attention on specific skill developments,” Antioho adds. “Obviously we want student achievement to be important, but this also focuses on what it is we as teachers are doing, too.”

When it comes to proof, Hill is cautious, saying only that he knows as the system becomes more well-oiled and consistent, teachers will know if their students are working at proficiency. Meanwhile, 11 of the district’s 19 schools showed enough academic growth on the 2005 state tests to be placed on Indiana’s “exemplary progress” list.

“We are not going to be satisfied until this data-driven decision model is in effect in every classroom,” Hill sums up. “We are going for consistent programming for students as they progress through Elkhart Community Schools.”