Email / Voicemail Server Failure
July 18, 2008 07:05 PM
Sometime on July 11th, the Corvallis School District's email and voicemail server failed. We think the failure was caused by overheating in the server room that week during the high outside temperatures, or a power outage or surge, but we are still investigating the cause of the failure. Our technicians worked with a Dell technician for two days to bring the server back online, and then worked with a couple of Microsoft technicians all day on July 17th trying to restore our system, but were unable to recover the database.
During this process, they discovered that both backup systems that were in place had not been functioning as originally thought. The newest database version we have is 6 months old; therefore technology services staff will need to retrieve the cached Outlook data off of each person’s computer and save the cached information into a file that can then be copied back into Outlook as soon as the database is turned on. We can't turn on the system until the last 6 months of cached data is saved, or the data will be lost. After the data is saved, the database can be restarted and the saved data can be moved back into each user's mailbox to bring it up to date. It will take approximately all of the week of July 21 in order to do that and we've hired additional staff to assist in the endeavor.
We're taking measures to add an additional server to be mirroring the server that we have and we're getting outside vendor assistance to help us with our backup systems. It is our desire to do what we can to keep this from happening again, even with a server failure.
We are hopeful that the email/voicemail system will be up and running by the close of business on July 25. However, any email and voicemail messages sent to staff since July 11 may be lost; community members are encouraged to call or email again if they have not received a response to a voicemail or an email message from staff who work over the summer. Most emails have been queued up and are awaiting delivery. We sincerely regret the inconvenience this situation has caused.
