Dec 13 2013
Data Center

Broken Pipe Floods Washington State University’s Data Center

Outages affected websites, email and online students.

Earlier this week, a broken pipe led to significant water damage in Washington State University’s data center. The flood affected student email, the university’s website and other systems. Some students were actually taking final exams when the outages occurred; without a website, the administration was forced to use social media to communicate with faculty members and staff.

The college has two Tumblr blogs: WSU News and a more general WSU blog. Both sites, in addition to Twitter, Facebook and text messaging, were used to spread news about outages. This message was posted on both sites on Wednesday:

Beginning at approximately 7 a.m. today, flooding resulting from a burst water pipe in the IT building on the main Washington State University campus in Pullman disrupted the operations of a significant number of computer servers housed in the lower level of that building. The flooding occurred because recent freezing temperatures in the region damaged the building’s water pipes, which then released a torrent of water as temperatures moderated.

WSU is in the midst of finals week, and it is important to note that the server outages are not affecting the final examination schedules of the large majority of WSU’s on-campus students. It now appears that a limited number of on-campus students and 98 Global Campus students scheduled to take online examinations today may encounter difficulties in gaining access to online exams. Impacted students are working with faculty members to develop alternative plans to submit final assignments or take their exams.

On Thursday, the WSU team posted this update:

Information Services staff worked throughout the night restoring most of the critical university enterprise systems. Although many systems are operational it should be noted that there are still pockets that are not operational and work is ongoing to get these systems restored as well. Of those systems restored many may be slow to respond and there is still much work to be done to gain a fully restored status.

While every effort is being made to restore services, it should be understood that water is being discovered in new places at every turn. So while systems are up and running now, there is still the possibility of future system failures due to residual water.

It’s an IT nightmare. Our hats are off to the WSU IT team that is working so diligently to restore the affected systems.

Here is a look at the reaction from students and the media:

Check Tumblr for updates.

<p>Credit: WSU Facebook</p>

Become an Insider

Unlock white papers, personalized recommendations and other premium content for an in-depth look at evolving IT