How To: Add IT Up

Ten masterpieces were stolen and placed in museums around the world, and it’s up to students at Washington Township High School in Sewell, N.J., to find them. Students, posing as art historians, complete crime reports on the artworks, while “detectives” dig for clues and “travel agents” map detailed itineraries to museums. After the students finish their investigations, they impart their findings using Microsoft PowerPoint presentations.

The students are studying the usual subjects — geography, art history, math, English and technology — but their enthusiasm for these lessons is far greater than it is for a typical lesson plan. “It becomes very relevant and hands-on to them,” says Steve Gregor, social studies department supervisor at Washington Township High.

When businesses measure the returns on their technology investments, they look at dollars and cents: increased sales, decreased staffing, greater productivity. Schools, however, don’t measure their return in dollars and cents, but in terms of their students’ success. For instance, Washington Township Public Schools’ ROI for its new notebook PC carts is increased student engagement in learning and the improved quality of their work.

“I think it helps them synthesize the material better,” Gregor explains. “To me, the whole project has been very successful so far.”

The return-on-investment concept in education technology is controversial. Many educators insist that so many factors affect student achievement that any ROI formula measuring it would be ineffectual. Even if achievement could be tied directly to technology, how would you assign a number to it?

Others argue that a rigorous ROI process can help budget-starved schools make more informed technology purchases and can help convince communities that technology is a good investment.

“Understanding value is essential if we’re going to keep on asking for and using money to deploy technology,” says Bill Rust, a research director at Gartner, a Stamford, Conn.-based research firm, where he covers K-12 education. “You have to understand how [technology] is going to be used and what it adds to the educational environment.”

MEASURABLE SUCCESS

In the fast-growing Dallas suburb of Forney, Texas, Technology Director Roger Geiger may have found a way to keep pace with the Forney Independent School District’s ever-expanding demand for textbooks. Instead of ordering new print editions, he’s piloting a project in the fifth and sixth grades to purchase notebook PCs and electronic textbooks.

So far, the project has proven valuable. “I think you can measure a couple of things that are pretty significant if you’re trying to measure your return on investment,” Geiger says, pointing out that effective technology can boost both student engagement levels and their attendance rates, which are tied directly to school funding.

More important, he adds, the test scores and pass-fail rates of classes that have technology can be compared with the scores and rates of classes that don’t have it. “That’s our measure of success,” Geiger says. “That’s our profit.”

Since the pilot’s launch in August 2004, attendance rates and standardized test scores have gone up, and Geiger has won praise from teachers, parents and administrators. “[Teachers] said it was the best, most interesting year they had teaching,” he says.

A major indication of the success of the notebook PC and electronic textbook project is the unsolicited praise from the parents, according to Geiger. They believe that their children are more interested in school since they got the new computers. Geiger adds that children who are engaged in schoolwork cause fewer disciplinary problems.

The students are clearly enthusiastic about the district’s new technology. Accustomed to researching on the Web while chatting via instant messaging, talking on cell phones and listening to MP3s, today’s students love e-books, Geiger says. Technology engages them and leads to the development of higher-order thinking skills, which can be measured, he adds. “They’re used to all this multisensory stuff,” Geiger points out. “It’s the world they live in.”

WHAT’S I.T. WORTH?

In the past, the secretaries in human resources at Mesa County Valley School District 51 in Grand Junction, Colo., would spend all of June pulling data and compiling spreadsheets, which included student demographics, free and reduced lunch figures, and other statistics for state reports. But last June was different, because the school district used a new homegrown tool built with Visual Basic to automate the entire process.

“Some secretaries were taking three to four weeks to get these reports out,” recalls Ben Startzer, the district’s executive director of technology services. “We were able to cut that down to four or five days.”

The ROI for such a tool was clear, Startzer says. The investment equaled the cost of the equipment and staff development time. The return was the cost of staff hours that would have been spent compiling the reports, minus the actual time spent. The investment minus the cost clearly showed a positive return, he reports.

Calculating the return on investment for school administrative and productivity tools is often straightforward, Startzer says. However, he points out that classroom technology represents a very different challenge.

It’s not as though a formula can show that a $50,000 tech investment will be returned within three years. “That’s the holy grail that everyone’s looking for,” Gartner’s Rust says.

Although classroom ROI formulas may not look much like their business-world counterparts, they do exist and can be extremely beneficial, he adds. Rust advises schools to analyze value through a portfolio of frameworks and categories. (See “Five Steps to Success ” on this page.)

School districts can measure several factors before and after a technology deployment to help them determine ROI cost, benefits and the resulting value of investment, says Rich Kaestner, a consultant and the TCO project director for the Washington, D.C.-based Consortium for School Networking (CoSN), who helped develop a tool to enable schools to calculate their technology infrastructure’s total cost of ownership. (See “Before ROI, Evaluate TCO ” on page 24.) The tool lets schools look at figures such as graduation rates, the percentage of students entering college and standardized test scores.

“The hard thing to do is to turn that into dollars,” Kaestner says. It’s difficult, but not impossible. If test scores rise by 5 percent after installing new software, the return is 5 percent of the per-pupil spending costs multiplied by the number of students.

But that’s not the whole story. “You’re taking something that’s qualitative and trying to make it quantitative,” Kaestner cautions.

“Schools should put together a matrix that lists what’s important to them, then look at how each project affects those goals. You can’t just look at absolute numbers.”

KNOW YOUR LIMITS

Each year, Startzer and his team poll the Mesa County school community to determine the district’s technology needs. Then they weigh that feedback with factors such as the district’s limited budget, the proven effectiveness of the technology and the actual need versus the perceived need.

Startzer says he can’t think of an instance in which he’s implemented technology that didn’t produce value. That’s because, with such tight funding, he sticks with best practices where the returns are well documented.

Startzer does scrutinize direct and indirect costs to ensure that he knows the total cost of ownership for the district’s technology. But taking that to the next level to determine ROI is another story, he says.

“In business, you can say ‘We instituted this, and it produced these profits,’” he says. “In education, when the product is student learning and that’s subjective, it’s hard.”

“As soon as we find out that a student isn’t learning, there’s so much we can pull from,” Startzer adds. It could be the teacher, the home environment, the school leadership or the textbooks. The student may be more of an audio learner, or he may not do well on tests.

“You can do ROI, but I believe there are too many variables to pinpoint when you’re talking about educational technology,” he says.

However, Forney’s Geiger suggests that ROI measures can prove valuable as long as they don’t distract schools from their primary mission: educating their students.

“Don’t get too bogged down in the details,” he advises. “You’re not looking for scientific certainty, but rather for common sense that’s based on scientific principles. You want indications that you’re going in the right direction.”