CoSN 2019: How to Design an Effective K–12 Technology Evaluation Program

Schools must have robust evaluation if they want to continue to find success in technology integration, said Steven Baule, assistant professor at the University of Wisconsin-Superior, and former superintendent for Muncie (Ind.) Community Schools.

Proper technology evaluation is a crucial element to digital innovation, said Baule, speaking at Consortium for School Networking’s 2019 annual conference.

Assessments can be used to make programs more cost-effective, improve implementation, enable replication elsewhere in the district and justify more funding to administrators. 

However, many schools are not properly assessing technology programs, Baule told attendees. For example, when Baule conducted a survey during his time as a superintendent in Indiana, he found only one-third of schools were assessing their one-to-one device programs. 

To help administrators create effective, long-term programs, Baule outlined some best practices for technology program evaluation.

JOIN THE CONVERSATION: Follow @EdTech_K12 on Twitter for continued COSN 2019 coverage.

4 Key Evaluation Questions for K–12 Schools

When designing a technology evaluation system, here are four key questions schools should be asking:

1. Active use: First and foremost, it is important to ask whether users are actually taking advantage of the technology that is being evaluated. “I have done evaluations for other school districts and colleagues, and they have a lot of technology, but it is just sitting in boxes in a closet,” said Baule. If schools have technology sitting idle in a storage space, it may be good to restructure their professional development programs to give teachers the agency they need.
2. Sustainability: While a program may start off successfully, it will not matter if it cannot be maintained over the long term. To achieve sustainability, programs must have the appropriate support systems in place. In one school district Baule previously worked in, administrators bought personal tablets to integrate into the curriculum. However, technology leaders did not install a learning management system, did not upgrade the Wi-Fi and did not have a replacement plan. Without these things, the new device program was doomed to fail. 
3. Systemwide outcomes: Look into whether a program is successful across the board or if there are only specific pockets of success, said Baule. Siloed examples of success could come from a program that is too dependent on teachers’ willingness to participate. 
4. Standards alignment: Schools need to know how well their programs align with national and state privacy standards, educational frameworks and credentialing programs, said Baule. 

MORE FROM EDTECH: Check out where K–12 schools should direct their education technology efforts.

3 Metrics K–12 Schools Can Use to Measure Success

Schools have the capability to collect a lot of information; however, not every data point will give an accurate picture of success. Baule points to three areas that schools can focus on to get a better idea of how technology is impacting a school system:

1. Student engagement and motivation: “Student engagement and motivation is, in most cases, the most important thing you are evaluating,” said Baule. Some helpful measures within this category include homework completion, detentions and suspensions, and absenteeism. Schools can also send out surveys to teachers and students to get helpful qualitative data. 
2. Instructional cost: “All three of my school districts as a superintendent saved significant money by using technology,” said Baule. “Technology not used effectively will cost more.” During his time as superintendent in Indiana, Baule found his schools’ one-to-one programs reduced the paper budget, decreased staff absences and significantly cut spending on textbooks over the long term. 
3. Student achievement: Increases in standardized reading and math scores are some of the tried and true methods to measure increased student success, said Baule. Schools can go one step further when measuring student success and identify specific areas that caused greater student outcomes. For example, when Baule assessed his district’s program, he found test scores improved because there was a spike in formative assessments, and teachers could identify struggling students more quickly. 

Regardless of what measurements schools decide to use, it is important to create a robust evaluation plan that fits the school’s specific needs and culture, said Baule.

Keep this page bookmarked for articles and videos from the event. Follow us on Twitter at @EdTech_K12 or the official conference Twitter account, @CoSN, and join the conversation using the hashtag #CoSN2019.