The outage will end today. The workarounds will not
- Technology
- IT consultancy
- Knutsford
Microsoft 365 went over on Thursday and a good many firms have now gone more than twelve hours without email. Outlook, Teams, OneDrive, all of it, across a long list of countries. Microsoft's own status page is carrying the notice on the public web rather than in the admin centre, for the reason it states plainly: administrators may not be able to reach the admin centre either.
So the first thing to say is the unhelpful thing. You cannot fix this. Nothing in your building is broken, there is no setting to change, and the only honest instruction is to wait.
The second thing is less obvious, and it is the one worth your attention this morning. While you are waiting, your staff are not. They are getting on with the job, inventively, and some of what they are inventing will still be in your business long after Microsoft has published its incident report.
The advice doing the rounds today is mostly for somebody else
Every outage produces the same article. Switch to your secondary communications channel. Invoke your business continuity plan. Fail over to the backup tenant.
For a firm of ten people, none of that exists, and pretending otherwise wastes the morning. There is no secondary channel. The continuity plan, if there is one, was written for a fire. A backup tenant is a thing large organisations buy and small ones have never heard of, reasonably, because the cost of one would buy a member of staff.
What a small firm actually has is a dozen people with their own phones, their own email addresses, and a strong desire to get today's work done.
That is not a weakness. It is just the thing that needs managing, and it is the only part of today you have any say over.
What is being invented in your office right now
None of this requires a bad actor. Every one of them is a conscientious person trying to do their job.
Someone has given a client their personal mobile number, because the client was waiting and the office number goes to a system that is also down. Someone has set up a Gmail address to send a document that had to go out today. Someone has pulled a client file out of OneDrive from a laptop that still had it cached, and put it through a free file transfer site because it was too big to attach. Two people have moved a live job onto a WhatsApp group, and a third has been added who is not an employee.
And somebody, somewhere in the building, has pasted a paragraph of a client's letter into whatever chatbot is still working, to get a reply drafted.
Microsoft's problem lasts a day. Yours lasts longer
Work through what each of those leaves behind once the service comes back.
The personal mobile number is now the number that client rings. Not today. For the next four years, including after that member of staff leaves, and nobody else will ever see those calls or know what was agreed on them.
The Gmail address sent a document on your behalf, from an account you do not own and cannot search. If somebody asks next spring what was sent and when, the answer is in a mailbox you have no right of access to.
The file transfer site has a copy of a client's data and a public link with no expiry anybody checked.
The WhatsApp group contains a record of a job, including the bit where the price changed, and it is on personal handsets. We have written before about what happens to those messages when the member of staff leaves, because it is the question that arrives late and badly.
The forwarding rule is still forwarding. That one is the genuine sleeper. It survives the outage, it survives the next password change, and it does not announce itself.
Not one of these is a disaster on its own. Together they are a week of unpicking, done months later by somebody who was not there, and some of it cannot be unpicked at all.
What to do before lunchtime
Short, and none of it technical.
- 01Say out loud which fallback is sanctioned, and which are not
- 02Give people one place to write down what they promised anyone today
- 03Tell everyone that nothing gets forwarded to a personal mailbox
- 04Put a note in the diary for next week to go looking for what was set up anyway
The first one is the one that works. People improvise because nobody told them what to do, and they improvise individually, in twelve different directions. A single message naming the channel you are happy with, and saying plainly that client files do not go through personal accounts or free transfer sites, will prevent most of it in the time it takes to type.
The second matters more than it looks. The commitments made today exist only in people's heads and on their phones. When email comes back, the inbox will tell you what arrived. Nothing will tell you what was said on a mobile at half past eleven.
The fourth is the one everybody skips. Some of this will happen regardless of what you say this morning, and the person who set up the forwarding rule will not volunteer it, not out of dishonesty but because by Tuesday it will not feel like a thing that happened. Go and look. The forwarding report in the admin centre takes a couple of minutes and answers it directly.
A workaround nobody records is indistinguishable from a workaround nobody undoes.
And the part you will be asked on Monday
Somebody is going to suggest leaving Microsoft over this.
The honest answer is that the alternative has outages too, and that a migration decided in temper during a bad week is how firms end up with two half-finished mail systems and a worse story than the one they started with. Google has gone down. AWS has gone down and taken a quarter of the internet with it. The lesson of an outage is almost never that you picked wrong. It is usually that you had no idea how much of the business was sitting in one place, which is a different and more useful thing to learn.
There is a version of this worth doing properly once the dust settles: work out which handful of things genuinely cannot wait half a day, and arrange for those, specifically, rather than buying resilience for everything. That is an afternoon's work and it is the same exercise as establishing what still runs when the broadband goes.
Today is not that day. Today the useful work is telling people what not to improvise, and writing down what you promised while you could not send an email.
If you want a hand going through what got set up this week once it is over, get in touch.
If any of this sounds like your business, we will tell you plainly whether we can help.
