Not too long ago, my friend Andy said “I think that’s how i convince so many people I’m decent at being a sysadmin, I just look at logs.” It was a statement that really struck a chord with me. On the one hand, I feel like I’ve had a pretty successful career so far, and if I haven’t been excellent, I’ve at least been sufficiently competent. On the other hand, I don’t feel like I have a lot of the experience that “everyone” has. I’ve never run email servers, I’ve only done trivial DNS and DHCP setups, and so on.

In my current job, I work with some really smart people, few of whom have much if any sysadmin experience. Andy and I, because of our sysadmin background, have become the go-to guys for sysadmin questions. It’s a role I enjoy, particularly when I’m able to solve a problem. Sometimes it’s because I have direct experience with the issue, but more often than not, it’s because I know how to poke around until I find the answer.

There are many skills required for successful systems administration. Troubleshooting is high on the list. The ability to troubleshoot new and unusual problems is particularly helpful. I’ve found my own troubleshooting abilities to depend on the ability to build off of previous (if unrelated) experience and being able to piece together disjointed clues. It’s sort of like being a detective and also a human “big data” system.

Fishing for clues in log files is great, too, if you can find the needle in the haystack. But what seems to separate the good troubleshooters from the bad that I’ve known is intellectual curiosity. Not just asking what happened, but finding out why. Being willing to ask questions and learn about unfamiliar areas instead of immediately deferring to those more knowledgeable builds a strong skill base.

Of course, without Google, we’d all be out of luck.