Techie assumed boss knew what they were talking about and killed Exchange by deleting data

Friday afternoon is not the time for snap decisions, but weekends are useful to cover your tracks

Monday mornings can be filled with trepidation and tension, which is why The Register kicks them off with a little light-hearted fun in the form of a new instalment of “Who, Me?" It’s the reader-contributed column in which you admit to your errors, and reveal the consequences.

This week, meet a reader who asked to be Regomized as “Phil” and told us that many years ago he started a new job that, among other things, involved maintaining Microsoft’s magnificent Exchange server.

“Exchange servers, especially onsite ones, were nothing new to me or to be worried about,” Phil told The Register. “The clients I worked with at my previous job were still hosting some local Exchange servers on Windows NT systems with 15 GB drives in 2014!”

REG AD

Phil therefore felt confident when, quite late on a Friday, a user reported that email was running slowly, and messages weren’t making it into or out of the building.

REG AD

“This organization had a centralized exchange system for the entire company and on quick inspection I determined that it had run out of space. Further inspection showed that the vast majority of this was due to a rather full log file directory.”

Phil’s confidence then ebbed a little.

“I hadn’t dealt with a system this large or for this many users,” he admitted. “The usual fix that I knew from previous experience was to ask users to delete emails from their Outlook clients.”

That wasn’t going to work late on a Friday when people were racing to wind up their weeks.

While Phil pondered the situation, one of the company’s IT directors came to watch him work. Noticing that Phil was looking at log files, the director told him to delete them.

“I assumed the director knew a bit more about the dark arts of Exchange management than I, so I proceeded to say goodbye to 300GB of log files.”

Phil and the director then logged off, shut the office, and went home for the weekend.

“A few hours later my internal alarm bells started ringing and my spidey sense started tingling,” Phil wrote. “Sure enough all hell was breaking loose: deleting those logs had crashed the entire system, and with this being a 24/7 business with sites all over the country not having email was a major problem.”

REG AD

The poor chap who was on call for the weekend was instantly swamped but found time to send Phil a message to ask what was going on.

“My reply was suitably vague and I said I'd take another look at things to help out,” Phil wrote.

Despite having deleted the files after being told to by the IT director, Phil spent the weekend doing two things: learning how to repair and rebuild an exchange database with the command line, then doing the job.

“By Sunday afternoon I'd restored the services, got everything working and now knew how to properly truncate the logs,” he told Who, Me?

What he didn’t know was if this could be a career-limiting incident.

“I trudged into the office on Monday expecting to be hauled over the coals, and sure enough was called into the CEO's office … where I was praised for my dedication and given a huge pat on the back,” he wrote. “I refrained from dropping the director in the mire and took the praise with good grace.”

Have you landed in trouble after following orders? If so, The Register instructs you to click here to send your story to “Who, Me?” so we can share it on some distant Monday. ®