Routing Routine?

I’ve been very cagey about writing anything to do with my job on here. Mainly because my employer says that I’m not allowed to write about them. Well that’s fine – I totally understand that. Doesn’t stop me writing about the stuff I see or experiences I’ve had through my career, or what I do as long as it’s anonymous.

Today, my job pretty much centres around managing a brilliant team of people who manage a large computer network (the bits that link all the machines together), but also being hands on and technical myself too – I find it works best if I at least know what I’m talking about. But also knowing and respecting that my team know as much as I do, if not more sometimes, so it’s not a dictatorship – more a consensus.

So we all make a bunch of routers, switches and firewalls talk to each other. And hassle people who stop them working. But I’ve not always done that – I started my career in IT back in the early 90′s filling up toner in Oki laser printers, and mending 386SX desktop machines running DOS 3.3 and Wordstar. Hand coding NDIS drivers and fixing Novell servers were standard practice, and I used UNIX SVR4 and Solaris/XWindows before Microsoft Windows was mainstream, and Linux was a pipe dream of some bloke called Linus. (If you wanted Intel Unix you used Solaris or Microsoft’s Xenix – remember that?)
DEC MicroVAX machines were also a staple diet back then, and I used to have an EGA monitor (8 or 16 colours at the flick of a switch) on my Apricot Xen-I 386SX with 4Mb RAM and 40Mb Hard Drive. VGA was a luxury with it’s 256 colour display and higher resolutions. Gem Desktop anyone?

So what are the things I’ve learnt thanks to working in IT for just short of 20 years?

• BT Openreach are a law unto themselves. Any time period you can put on a comms link installation, then always add an extra 30 days for ‘blocked ducts’. And if you have an office anywhere near the new Media City in Manchester, be aware that the fibre installation engineers are unaware that the BBC have requested that all the ducting covers are welded shut!
• There is no Service Level Agreement on an ADSL line. But businesses will always use ADSL lines instead of proper leased lines (or even now EFM), because they’re cheap and cheerful, usually work OK and are in the main easy to install and maintain.
  However, people usually get very upset when their ADSL line goes down because Openreach are meddling in the exchange or they’ve managed to break their authentication system, and you tell them sadly there’s no quick fix. That’s usually followed by a quote for a link with a proper SLA attached (and a heftier price tag). Service Level Agreements are exactly what they sound like – a commitment from a comms provider for a guaranteed fix time, with financial penalties attached if they don’t succeed in fixing the link in the time they say they will. That costs money, and you get what you pay for.
• Companies that outsource their IT usually regret it fairly quickly after they get their first bill and realise that to keep the same level of service to the business it’s costing them more and they’re getting a much lower quality service. I’ve seen it time and time again where the Outsourcing company generally works to rule with no flexibility, with staff that bugger off at 4pm and leave you high and dry.
  I’ve been all sides of the coin – working as a contractor, working for an outsourcer and being in a company looking at outsourcing. Usually companies that outsource their IT are on the slippery slope to a takeover.
  And also, whatever an IT outsourcing company tells you it will cost to provide a job, double the price or fix them on very strong contracts (and watch out for work to rule again here). They’re only quoting you the lowest price to get their foot in the door. Added extras will put the price up, as will you changing the scope every 5 minutes.
• The amount of bandwidth you need is usually inversely proportional to the amount of bandwidth you think you need.
• No matter how much bandwidth you have on your internet connection, someone working from home will always have more and complain that your link is ‘too slow’.
• VMWare is great, but beware older virtual infrastructure that has very weird side effects to some applications (manifests itself as strange delays or blips) – even though a VMWare guru will swear blind they don’t know it happens. Those of us who have used everything since 386 servers will know something isn’t *quite* right. Although to it’s defence I have to say it’s less prominent now in the latest version of VSphere.
•  Users/Clients will think Virtual Infrastructure = Free + Full DR.
  It’s not. Usually it’s more costly than filling a rack full of 1U servers each running an individual app. But you don’t necessarily get the flexibility with a rack of 1U servers. Strangely the whole thing has come full circle. I remember using Mainframes with Virtual Machines and Hypervisors. They’ve been around since the 60′s (and most big corporates still use them).
• 10Gb switches make no difference if you’re using Microsoft Shared Folders (SMB) to copy files between hosts. You’ll struggle to max out a 100Mb LAN port using Windows Shared Folders. You’ll even struggle to max out a 1Gb port with FTP. Try a groovy UDP file transfer protocol instead that hoofs the data at wire speed (and at the same time frustrate fellow WAN users by absorbing all the WAN bandwidth).
•  No matter how much redundancy you think you’ve built into a solution, the whole solution will at some stage fail on it’s arse due to an unknown /  missed single point of failure.
• Scopes of projects are always dynamic. You should never feel like you’ve completed a project as something else will crawl out of the woodwork and bite you.
• Mitigating against power is all very well, but if you have a massive heating boiler in the plant room above your main office floor that bursts (or something sets off the sprinklers you haven’t capped in your comms room) you end up with alot of wet kit.
• Having security guards and commissionaires is no guarantee of security – especially when they let the burglars in the comms room to rob all your RAM from your servers and steal your routers.
• Don’t complain that the LAN is slow when your machine is riddled with Viruses.
•  Documentation is for wimps. Well, that’s what people keep telling me anyway. I don’t believe them…