Preparing for outages: the major incident process
“By failing to prepare, you are preparing to fail.” said Benjamin Franklin – and he was a smart guy. In this post, I’ll outline the preparations we’ve undertaken in case the worst should happen and one of our priority services fails.
Top priority services
All our services are important to someone (I hope) but here I’m really talking about ‘Top Priority Services’: these services have been agreed as core to the University’s business:
- The University website
But I’m also taking into consideration services we provide that would underpin these, and it would be disastrous if they were not working for extended time periods:
- EdWeb CMS
- University search
What is a major incident?
Regrettably, technology hasn’t reached the point where it works 100% of the time (work harder, tech people!). So with that in mind, it’s worth establishing processes for our staff to undertake so that we can calmly respond to outages, and ensure we always take appropriate action. To help with this, Information Services has established a major incident process that helps us understand what to do in an emergency.
But I wanted to create some processes to help our staff do the initial service analysis work before progressing to that major incident stage. All of our staff go through an IT induction, and will be taken through these processes so that all our staff can help when something goes wrong – we don’t want to rely on single points of failure.
So to prepare for when our services go wrong, we’ve:
- Developed service outage notifications: if our sites go off, our monitor picks it up immediately
- Provided step-by-step instructions on what to test if we suspect a service is down
- Updated contact lists of technical support (internal and external) once we understand there is an issue
- Standardised emails and comms plans for our user groups
Communications from Website and Communications
Please consider signing up for our mailing lists if want to hear from us on general topics:
- compliance and statutory updates
- notifications of changes in the University web estate
- invites to our communities of practice
or targeted topics, based on your needs and interests:
- web analytics
- social media
Website editor mailing lists (EASE login required)