While we at OpenDNS dedicate an entire month to showing our SysAdmin love, today is the official System Administrator Appreciation Day. Cheers to all the SysAdmins out there - be sure to enjoy your day!
We asked you all to nominate the SysAdmin in your life who stands out as exceptional. We had so many fantastic nominations to choose from, but after an intense review session our judging team of David Ulevitch, Jesse Davidson, Bill Fumerola and George Patterson came to an agreement. Winners in each category will received a $50 AmEx gift card and the SysAdmin of the year gets $200 and a gift package of OpenDNS schwag. Without further ado, the winners are:
Best Disaster Response Award
Winner: Sean Harrington, SysAdmin for Telford Group
Nominated by: Justin Mecham
Justin says: “Our company has two facilities located at an International Airport. There is an inconvenient runway in between the facilities. To communicate between the buildings a Point to Point Wireless bridge was put in place. The furthest facility’s LAN and WAN connections were routed over the bridge. During one thunderstorm a lighting strike hit a nearby utility pole and leapt to our antenna in its search for ground. This completely incapacitated the PTP Wireless bridge since the antenna wire now had large holes burnt in it.
When this happened our Systems Administrator was in his second week and the boss (i.e. the rest of the IT department) was on a business trip and out of communication. Within the first hour the extent of damage was determined and a vendor was brought in to make a repair. Unfortunately, the equipment was rather old and parts could not be acquired to repair it, nor would he have wanted to since it was only 3Mbps. It was going to take a week to get the parts in and the devices reinstalled.
Within the second hour management was notified, a budget was created, and a communication plan was drafted. All information was to be routed through a few key individuals to keep himself free to find solutions. A makeshift call center was created in the IT office and conference rooms. A representative from each department was setup with a computer. The sales guy would create quotes and process orders, the shipping guy would search for locations in inventory and process pick tickets, and so on and so forth. Phone lines were cleared and designated for this inter site communication. A typical part being sold would take three phone calls to three different people, but within two hours parts and maintenance were moving again. Sean was using these people as a Quasi-Modem TM – Patent Pending.
Sean made it a point to update management twice a day on progress and ensured the vendors were making progress in getting the new equipment. The event happened on a Tuesday and by Friday afternoon everything was up and running. Not only that but the network was running faster at 11Mpbs. While business for the 50 employees in the remote facility certainly was a little slower and a lot more tedious things continued to operate. The IT Manger returned to find his network running better than it ever had.
Since the incident Sean has become the IT Manager for our company. One of the biggest changes he has made since the incident was negotiating a deal with a local carrier for Dark Fiber so that facilities would not have to worry about a similar event. The wireless still sits on the roof just in case a dump truck takes out a utility pole.”
Unbelievable Uptime Award
Winner: Tom Scholl, Jack of All Trades on a major global backbone
Nominated by: His friends
TSERV2.XXXXXX uptime is 8 years, 36 weeks, 1 day, 31 minutes
System restarted by power-on at 20:56:30 UTC Wed Nov 17 1999
System image file is "flash:XXXXXXXXXXXXXXXX", booted via flash
Tom says: “The above device is a Cisco router that is acting as a terminal server. Its role is to provide out-of-band CLI connectivity to servers and routers. Over time, at least two ports on its console cable have gone bad. Obviously, we have a good power infrastructure where that device is located. Yes, it is running older code but it hasn’t ran into any problems for awhile. At least two other terminal servers are exhibiting uptimes of at least 8 years.”
Tom’s friends say that Tom is a fantastic engineer who prides himself on his ability to design and deploy stable and reliable infrastructure in all corners of the planet under all kinds of harsh environments. He spends most of his time online, which is part of the reason he makes sure that the infrastructure on the Internet is really stable. Tom has also previously given technical presentations in major industry forums such as NANOG and #IX.
Shoestring Budget Award
Winner: Luke Popejoy, SysAdmin for Integrity Computer Services
Nominated by: Mark Sanger
Mark says: “Luke Popejoy took our ‘little’ radio stations with about 21 computers and cleaned up our network and IT needs, literally overnight. Knowing we are a very small company, but have a giant need for computers to run our radio stations, Luke literally started working on our machines before he was even hired or even considered all our needs. He said, ‘you guys seem like good people so let me just take care of a few simple things for you right now while I’m here.’ UNHEARD OF!! Then when we got his quote we quickly negotiated and Luke is making the computers work for us. He is always monitoring and suggesting things to improve our network and our uses of our machines.
We have two industry specific pieces of software that run two different pieces of software, then it all integrates with windows servers, novell, Linux, a few machines with the first version of windows running on it and then it all talks using DOS, audio files, FTP, streaming, VNC, and whole bunch of other ‘protocols.’ So how much would it have cost, estimates were from $20,000-$25,000 for a new system and then networking costs and some hourly rates… way too much and not even affordable for us. Luke negotiated us into an extremely reasonable monthly fee for any support and help we need and at any time we need it, no hidden costs… If we need a new ‘drive’ or ‘memory’ we pay for it but there is no additional fee for things. It is truly an amazing thing!!! Luke says he doesn’t understand why I’m so amazed at his help… I guess he is humble, but all the parts he’s worked on help us create, produce, broadcast, switch satellites, maintains all our records, accounting, clients, and pretty much allows us to turn the lights on everyday. For pete’s sake if we had lost one of our machines a month ago, we’d still be recovering and running the entire radio station by hand and having people doing everything manually. Can you imagine running a radio station without computers?? We can’t and we pray we never have to learn how.
What does all of Luke’s ‘work’ power? It powers two radio stations in North Carolina, it powers the only local stations in our area, it powers our communities ability to talk about issues and ideas and even buy or sell something daily on the Home Ad show. It powers a whole bunch of community organizations that rely on our stations to inform the public and help others in our communities.
We purchased the radio stations from a company that had giant computer budgets so we more things on the network than we needed or even knew what they were or how to control them. So, Luke has maximized our network, local machines, backups, security, vpn and several server things that we don’t even know what they are… and much much more… It is almost unbelievable what he has done within a couple of months of working every now and then. He has done more in a couple months than we did in our first year of owning this place. Before Luke arrived we were on a domain controller and we were slowly loosing connections to machines and in a few cases we lost access to the actual machines. It was truly a nightmare!! We could not get access to, nor figure out the domain controller work, nor could we afford to hire someone, full, part, or anytime, let alone pay for any parts that most of the IT people we interviewed said that we had to have. NOT Luke, he has taken what we have and makes it work for us… yes, he’s recommended some upgrades as we get the money, but he took what we had, merged and converged it, and now we have a great network! I can actually sleep good at night knowing that we have access to all our machines if the power goes out.”
Flying Solo Award
Winner: Adam Merritt, SysAdmin for MaxWiFi
Nominated by: Richard Hughes
Richard says: “[Adam] joined us 2 years ago as our first employee and was thrown into the deep-end, configuring and managing networks including VoIP, WiFi & Fixed across major events sites such as the Open Golf, The Stella Artois Tennis Tournament, The World Rally GB and many many more.
Perhaps he should also be in the shoestring budget category as we are a struggling company so most of Cisco gear is rented in and arrives 2 or 3 days before we install a network.
Adam is in sole control of designing the network, configuring the equipment and managing the deployment of all services on a temporary basis.
This year we started on-site at the Stella Artois Tennis with 17 ADSL’s, 2km of ethernet, 5 routers, 19 POE switches, 68 VoIP handsets, 20 APs, (http://www.lta.org.uk/Watch/Individual-Tournaments/The-British-Tennis-Championships/) had two days break after that before deploying at the European Open Golf Tournament (http://www.europeantour.com/) 13 ADSL, 1 Router, 4 POE Switches, 52 VoIP 12 AP’s. That finishes as I am writing this and then Adam has to break it all down, drive from London to Scotland (8 hours) and reinstall all of it, plus a lot more, at the Scottish Open Golf tournament (25 ADSL, 8 Routers, 12 switches, 69 VoIP, 14 AP’s)
He is also remotely managing the Clipper Round the World Yacht Race Finish in Liverpool, The Tall Ships Race also in Liverpool, an event for the Ministry of Defence and a last minute event for Nokia in London! Somehow the networks hold together. At the European Open he had his radio taken off him half way through the event as nobody had called him on it as there were no issues!
Please remember that the users of the networks come from all over the world as they are journalists and camera crews, sportsmen and various others so we have no idea who is going to do what with our networks. In amongst all that our 24-year-old Sys Admin passed his CCNA and driving test in the same week.”
SysAdmin of the Year
Winner: Dan Kaminsky
Nominated by: The Internet community
There is little question in our minds that there is nobody who has done more to keep the Internet up and running this year than Dan Kaminsky, the man to thank for identifying a major vulnerability in the Domain Name System. While not technically a SysAdmin at the moment (though he’s worn that hat many times), we feel strongly that Dan deserves the SysAdmin of the Year honor because of the countless hours he’s spent working with SysAdmins all over the world, guiding them through the process of securing their DNS so their networks are not exploited. Dan has spent the last four months working non-stop to ensure that major backbones, ISPs, vendors and others were patched from a DNS vulnerability that would have had disastrous consequences if not mitigated as much as it already has been. For that, we have Dan to thank. And if you think you weren’t going to be impacted by his work, think of all the support requests and trouble tickets you aren’t receiving because he was able to get another network to patch the vulnerability.
———————-
Thanks so much to everyone who attended our SysAdmin Appreciation party in San Francisco on Wednesday night. It was a huge success, with upward of 200 guests mingling with counterparts and sipping DNStinis. We can’t wait to celebrate with you all again next year.
Winner Patrick Farley is the SysAdmin for 
