The Importance of Measurable Benefits

Eighteen months ago I visited three districts that had recently rolled out one-to-one notebook programs. They each had a different implementation approach, means of financing, grade levels and usage guidelines. But upon walking in the door of any of these schools or their district offices, one could feel the enthusiasm — something good was happening. One thing in common among these districts was a dynamic, charismatic, tech-savvy superintendent who wanted to make a difference and pulled in the entire community. These projects all were promoted as a means of developing 21st-century skills, enhancing student achievement, providing student equity and attracting good teachers, among other things. You can read about these projects at www.classroomtco.org/gartner_intro.html#case.

Project Sustainability

As exciting as these one-to-one programs are, I am concerned about their sustainability. What will happen when these successful superintendents move on from the district to pursue other challenges (as has already happened in one of these districts), and the enthusiasm has died down, as it is bound to do? How will the school board or community react when the bond issue or lease is up for renewal or the budget comes under scrutiny? What evidence is there to show constituents that the project is successful and should be continued? The problem here is that while the community was caught up in the enthusiasm of the initial project, there is nothing to validate its success; that is, few if any of the one-to-one project benefits were stated in measurable terms. It is impossible to go back and evaluate project success if the anticipated benefits have not been previously stated in measurable terms.

Whereas businesses use financial impact as a measure of success, schools need to be able to point to the measured impact of technology and other projects on their mission, goals and mandates when it comes time to refresh or otherwise reinvest in the project.

Selling a Proposed Project

The ability to articulate measurable benefits for proposed projects and relating these benefits to school or district mission and goals can be critical to selling these projects to constituents in the first place. Most initiatives that are focused on improving teaching and learning are sold and implemented based on what feels right — what seems to be a good approach to respond to a need. The one-to-one implementations mentioned above are examples of projects that are most likely worthy, but are sold to the community based on personal conviction and minimal articulation of real anticipated benefits. For many districts, the days of acquiring technology for major projects without quantifiable justification are gone.

Fountain Hills Unified School District in Arizona has been pursuing approaches to one-to-one student computing but suffered a reality check when voters turned down a trial referendum. “To a large extent, the community doesn’t buy in to the district’s earnest efforts in this as a project that drives academic excellence,” says Tim Leedy, assistant superintendent for business and support services. “In essence, we have been asked to explain just what we plan to accomplish beyond providing a computer for every high school student.” Major qualitative benefits uncovered by Leedy and his team include specific student achievement measurements as a key to maintaining a high academic ranking within the state, 21st- century life skills in preparation for their next level of learning, enhanced curricula through online learning, enhancing teaching and retaining good teachers, and continued community involvement, all of which are directly related to district mission and goals. With these benefits stated in measurable terms — in terms of specific anticipated achievements in these areas and related to district mission and goals — the district hopes to have a more convincing story to bring forward to the voters.

Another example involves Des Moines Public Schools, which has an objective to raise student literacy at the elementary school level, and has hopes for use of formative assessment as an aid that will allow teachers to fill in student reading and interpretation gaps on a timely basis. The concern is knowing how effective formative assessment can be before the district completely overhauls the current pedagogy. Using the Consortium for School Networking’s Value of Investment methodology, the district set specific measurable goals for a pilot project. With measurable goals, they will be able to specifically identify whether or not this approach is working for the pilot classrooms and use data to determine whether to fully implement this project.

Projects Competing for Funding

With budgets that are always too tight to do what we want or to truly implement projects properly, it is difficult to determine which project has the most merit. These selection decisions are generally made by district executives based on their personal interpretation of value and frequently with precious little impact information. Once again, where financial savings is not the issue, measuring qualitative project benefits for each proposed project and relating these to district or school mission, goals and mandates is key to making a good decision.

It helps to break down proposed projects into specific measurable benefits and relate them to school or district mission, goals and mandates, and in this case, comparing the relative merits of one project versus the other. For example, a school may be forced to decide between implementing an online learning project or refurbishing and upgrading the auditorium and stage. These two very different projects also have totally different benefits, but each can be defined by its specific anticipated benefits, and the effect of the benefits on what is important to the district can be estimated in relative terms.

Benefits of the auditorium and stage refurbishment project might include results such as these:

• Increase in seating capacity from 350 to 385;
• Improvement in community relationships by providing a safe and comfortable space for plays and community meetings as measured by an increase of three points in parent satisfaction in the annual survey.

These benefits can be scored based on their effect on school goals that might include statements about community relations and providing a safe environment.

Benefits of online learning alternatives could include aspects such as the following:

• Provision of a teaching/learning style alternative, increasing the percentage of students passing eighth grade state assessment in math from 52 percent to 58 percent by 2009;
• Increase in the graduation rate from 92 percent to 94 percent by capturing the interest of students who do not identify well with classic face-to-face teaching methods.

Likewise, these benefits can be scored based on their effect on school goals and mandates that might include providing learning opportunities for all students and overall student achievement goals.

By scoring both projects, a more objective decision can be made concerning the relative merits of both projects and focusing on what is most important to the school or district.

Getting Help

This whole process of evaluating major proposed projects might seem complex and confusing, but there is help in the form of a methodology and tools from CoSN’s Value of Investment Leadership Initiative at www.edtechvoi.org. This initiative is focused on helping you determine the costs and benefits of proposed projects.

Businesses have been using Return on Investment and Net Present Value tools for many years to determine anticipated costs and benefits, where benefits can be stated in terms of top-line income or bottom-line profits. When it comes to K–12 investments to enhance student learning or to accomplish district mission, goals and mandates, we need to think in terms other than revenue and savings. It is not easy for most school leaders to think of student achievement, 21st-century skills, student behavior or community relationships in discrete or measurable terms. However, when project benefits are stated in measurable terms and related to school goals, the project can better be evaluated, sold to constituents and, as measurements are met, sustained over time.