When a ransomware attack hit Oberlin City Schools in 2016, IT Coordinator Steve Nielsen thought the district was ready. Its cybersecurity vendor had assured him that the Ohio district’s cloud backups were stored in the United States. However, when Nielsen went to recover the affected systems, he found that the backups had actually been moved to China.
The problem? The state’s education department wouldn’t allow school networks to connect to that region due to its high number of cyberthreats. Although Oberlin’s systems were backed up, the district couldn’t retrieve its data.
For Nielsen, the incident brought to mind his father-in-law’s advice about motorcycles. “He told me that there are two types of bikers: those who have fallen off the bike and those who will fall off the bike,” Nielsen says. “That’s the approach people should take with cybersecurity. If you haven’t had an incident yet, then it’s just a matter of time. You need to take every avenue to harden your systems and make sure that you’re able to recover.”
The rise in ransomware attacks and the threat of natural disasters have spurred districts such as Oberlin to level up their backup and disaster recovery activities in recent years. Often, it’s an adverse event that motivates the change. By implementing the right tools and practices, districts can transform a serious outage into a relatively minor incident.
“The K–12 sector is a high-value target for ransomware attacks,” says Amy McLaughlin, cybersecurity project director at the Consortium for School Networking. “If something happens, a district will struggle with everything from basic operations to the door locks and the clocks on the wall. That’s how important technology has become to school districts today. Without a solid backup solution, all of those systems are at risk.”
Click the banner to start teaching your K–12 team to flag phishing attacks.
How Oberlin Schools Improved Systems After A Ransomware Attack
Unable to connect to the backups in China, Nielsen took an unconventional approach to recovering the district’s IT environment. “I actually had to take the server to my house to restore it from there,” he recalls. “It caused a significant delay. We have a 1 gigabit connection to the internet here at the schools, but I only have only got a 30-megabyte connection at home. It took a lot of time to download all of that backup data. It took us almost 70 hours before we were able to get those systems running again.”
The ransomware attack affected the district’s systems for managing transportation, facilities and substitute teachers. While these were being restored, the district had to temporarily assign someone to manage substitutes’ schedules, and district officials pored over email archives to try to keep buses on time. Despite those efforts, some field trips had to be canceled.
DISCOVER: Learn how school districts can successfully shift to the cloud.
After the incident, the district adopted Veeam for onsite backups and 11:11 Systems for offsite backups. The Veeam solution enables the district to rapidly restore its environment, as long as its onsite backups are unaffected and available. If the local backups are corrupted, the district can restore from the 11:11 Systems solution.
“Our data is currently stored in Virginia, which is a big, big deal,” Nielsen says. “One rule of thumb is to always verify your backups. I had been verifying our local backups in the past, but I was not verifying our cloud backups.”
Schools Build Reliable Lines Of Defense Against Ransomware Attacks
Don Ringelestein, executive director of technology for Yorkville Community Unit School District 115 in Illinois, says K–12 has undergone a philosophical shift on backup and recovery in recent years. While basic local and cloud backups were once considered enough to bring an organization back from virtually any disaster, districts now must ensure that their backups are immutable or air-gapped so attackers cannot hold them for ransom along with primary environments.
“You have to make sure that you’ve got a clean copy somewhere,” Ringelestein says.
Yorkville relies on Veeam for local backups and Wasabi for immutable backups that are stored in the cloud with Amazon Web Services.
To successfully protect against ransomware, Ringelestein says districts need to take a defense-in-depth approach. This means that backup and recovery solutions, while important, should be the last line of defense in a robust cybersecurity environment. Solutions such as endpoint protection and properly configured firewalls are also critical to the district’s efforts to keep ransomware attackers at bay, Ringelestein says.
“Educational institutions are being more targeted by more by bad actors,” he says, “but attackers typically don’t waste time if it’s going to take a lot of effort to penetrate a district.”
Ringelestein also notes that annual security assessments and regular testing are crucial for evaluating potential vulnerabilities. “It’s all well and good to have these systems in place, but if you don’t test them, you won’t know if they’re working or not,” he says.
Houston School Recovers From An Unexpected Network Outage
Before Christopher Hodge became technology director for St. Thomas High School in Houston, the school suffered a huge outage to its network-attached storage. At the time, the school was backing up its data to tape, a time-consuming, inefficient and — at least in this case — unreliable process.
“The tapes needed two backups a week, and maybe one of them was good,” Hodge says. “We had a fireproof safe where we kept our backups, but with the amount of data the school was generating, it was becoming quite expensive to have all of those tapes. And then, every week, someone would need to take an hour to drive the backups to an offsite location.”
When the network-attached storage failed, it took IT officials almost two weeks to recover, and the school still lost what Hodge calls a significant amount of data. Teachers suffered the greatest impact, with some losing a decade’s worth of lesson plans. Along with their lost files, Hodge says, many teachers lost trust in the school’s IT systems.
To modernize its backup and recovery processes, St. Thomas High School adopted Commvault. The solution performs daily backups that are stored both on a virtual machine on-premises and in the public cloud with AWS. Commvault backs up about half a dozen on-premises servers and some virtual machines. To minimize data costs, the solution pushes backups to the cloud during off-peak hours. If the school ever needs to rapidly recover its backups, Hodge says, it could do so by paying a higher data rate. He tests the solution every week.
“We were looking for a solution that would not only allow us to back up our data to a local server or appliance but that would also push those backups to the public cloud,” Hodge explains, "so that if we had a catastrophic failure within the server room, we could go out and pull that data back down.”
“You can always buy new hardware or install a new operating system, but you can never get back employee data after it’s lost,” he adds. “Backups are the most critical system we have.”
RELATED: K–12 IT leaders should ask these five questions about disaster recovery configurations.
