Today I’m sitting in the office in NJ, doing work as normal. What I’d prefer to be doing is going to the IT Roadmap Conference & Expo in NYC. According to the website, it’s “designed for IT professionals who want to cover multiple industry topics in one day”. That sounds like something I’d be interested in!

Essentially, it’s a sales pitch, or a series of sales pitches. I don’t know if I’m in the market for what they’re selling, but I’d like to go find out what is being offered. All the same, I like to keep my eyes on the horizon, because things have a habit of coming up quick on us in IT, and if we don’t familiarize ourselves with the likely technology of the next few years, then we’ll be caught with our pants down. So I wanted to see what people were selling.

The conference is free. All you have to do is fill out the application for registration. Unfortunately, I don’t qualify:

Dear Matt,

Thank you for your interest in Network World Live’s IT Roadmap Conference & Expo in New York.

Unfortunately, after reviewing the information that you submitted, we determined that at this time, we are not able to confirm your seat on a complimentary basis.

As we noted on the registration form, this event is geared towards network and IT professionals in end-user type companies who actively purchase products and services – or – who will be doing so in the near future. We have a limited number of complimentary seats reserved for attendees who meet this criteria.
…
snip
…
Walk-ins or ineligible applicants arriving at the conference facility will NOT be admitted on the day of the event.
Thank you,

IT Roadmap Team
Network World Events & Executive Forums

(emphasis theirs)

Well, I do actively purchase technologies and products, but not at the scale that they’re looking for, I suppose. I don’t have 50 data centers, or “20,000 or more” servers, so I don’t get to go to their party and look at the toys.

It’s unfortunate for them and me, but somehow I think I’ll live. I just wanted to give you a tangible example of…well…I won’t go so far as to say discrimination, but maybe exclusion, that we small admins deal with from vendors.

Now it’s just back to the daily grind for me. I’ve been rehashing some Nagios configuration and I’ve unearthed an ancient relic! How fun! Configuration archaeology is a hobby of mine, and to find a gem that hasn’t (as far as I can tell) been mentioned on the official site since 2002? That’s GREAT! I’ve still got to go through the source code to make sure that it doesn’t do anything interesting, but it’s out of my config now.

As it turns out, my recent attention to Nagios is multifaceted. I’m cleaning up the config and tightening up the alert rules, but also, I’m going to be giving a 45 minute talk at the Professional IT Community Conference in May. If you’re in the northeast US, you should definitely make it! And you should hurry and register while the early bird special is going!

There is no downtime in that equation. Immediately following those emergencies, most of us drink depressants to bring ourselves down. On normal days, we require morning stimulants to bring ourselves up. I highly suspect that some of us are so called “adrenaline junkies” from the relative high that we get when there’s an immediate problem that no one can solve but ourselves.

This is unhealthy.

What we really need is to be able to step back and look at the pattern in our lives and say I don’t want to live with this stress.

When it first hit me that stress is probably the biggest single microproblem for admins, I wrote the following. I hope you find it relevant.

As sysadmins, we’re nearly always the go-to person for whatever happens. After a while, we start to get used to it, and lots of times, we can develop a hero complex, carrying the weight of the world on our shoulders, at least in our minds. This isn’t healthy for a lot of reasons, the most important of which is your health.

Here’s an example of what taking your job too seriously can do to you:

Part One

Part Two

Not to ruin the ending, but the most disgusting part is that, while the guy was taking medical leave, his company fired him. To be completely honest, he’s much better off without a company like that, and if your company would do the same thing, then so are you.

To quote Peter Gibbons, “We don’t have a lot of time on this earth. We weren’t meant to spend it this way. Human beings were not meant to sit in little cubicles staring at computer screens all day…”

Even one of the most preeminent Systems Administrators around, Tom Limoncelli advocates leaving the pressure at work when you head home. For those of us on call 24/7/365, that can be a little hard, but it’s important to try.

Fire fighting, as a sysadmin, means you don’t make any progress. You only work very hard to stay where you are. Working against entropy is difficult, and it can take a lot out of you. Some days are harder than others.

One day in early June, not long after I started this blog, I experienced a major setback. Also, a major power outage. Our entire backup facility lost power, and what’s worse, the generator refused to kick on. Our secondary site was down hard for days, until the power was restored to the downtown area of the village we were located in.

During the problem, though, we were able to turn a major issue into a net gain. Read on for the rest of the story…

It’s funny, sometimes, how we tolerate suboptimal or downright malproductive arrangements in our infrastructures, just because it’s inconvenient or inopportune to do it the “right way”. It seems like “the right way” either never comes, due to projects getting phased out, or it gets fixed during a cataclysmic upheaval, when it has become an immediate concern.

The case in point is my mail server. We have an A and a B mx record. Originally the B MX just stored mail until the A came back up, then it would get delivered. Everyone checks mail on A, so it can’t really be down during the day, and about 6 months ago, the office that B was at relocated and B was never set up. This left us with just A. To make matters worse, A was old enough that it was physically located in our backup site, which used to be our primary site. This was suboptimal. Of course there was talk about moving it to the primary site, but when could a maintenance window be created? And we’d risk the entire period of non-connectivity when it was being moved. No, management said, lets just leave it where it was.

Great strategy. It actually worked fine though, until this weekend.

I came in on Saturday, ready to do some major work on the blade systems I’m building for our new site. I sat down at my desk, ready to dive into work. Since I was alone, Raiders of the Lost Ark was playing on the laptop. I had just logged into the first server when the lights went off, and the telltale screech and whine from the server room told me that we’d lost main power.

In Granville, OH, that’s not a strange thing. We’ve got backup AC and a backup generator, so I wasn’t worried. It does have to be manually started, so I jogged into the server room and turned on the CFL floor lamp. At least I tried to. I looked at the generator control panel and it confirmed my fears. No generator power.

I tried for several minutes to start it, but nothing gave me the impression that anything would change, so I called my boss to let him know the situation, and that I was going to start shutting down machines. Since the only critical thing was mail, I suggested that he change DNS to point to an as-yet unassigned IP at the colocation, and that I could setup a postfix process there to queue the mail. He said that it would work, but he suggested an alternative approach.

Why not relocate the physical mail server to the colocation? A lightbulb went off. Of course, not only could I take care of that long standing problem, but because there was no power at all in the datacenter, the normal policy of no-downtime-for-repairs-and-upgrades was out the window.

The next morning, I left work to go home at 5am. The previous 15 hours had been spent completely rehauling the backup datacenter. With the mail relocated to the primary facility, once the power came on in the backup, I had free reign to cull everything unnecessary that had been accumulating.

There is now a pile of cables covering a square yard or so around 6 inches deep of power, ethernet, and copper/fiber cables. There are something like 96 ports worth of switches that I took out, multiple servers, KVMs, fiber switches, and general cruft. The servers are also arranged so that no half-depth servers are hiding between full depth. That was always a pet peeve of mine.

I thought about it while I was doing this, and if fighting normal issues is considered firefighting, then what I went through should have been considered forestfire fighting. And just like a forest fire, good can come from it. It takes the massive heat of a forestfire to crack open some pine cones. It also takes massive infrastructure downtime to make significant changes.

Anyway, I was curious how other people did it, so I asked. As it turns out, this post originally aired in July of 2008. I would guess that I had a couple of hundred readers. That’s a good range of experience to draw from, but I wanted a more broad view, so I submitted it to slashdot. And it got on the front page.

Thanks to Slashdot, this entry originally received 43 comments, which is right around 30 more than the next most popular story at that point. I’ve had a lot of people tell me that they found me because of that front page article. I didn’t submit it to drive people to the blog; I really did want to hear what people were doing with their own networks. Driving people to the blog was a completely satisfactory side effect, though

Before you leave this page, make sure to check out the original and read the comments. There’s a lot of funny (and interesting) ideas!

Enjoy!

Bob Plankers, over at The Lone Sysadmin wrote a couple days ago about getting busted while reading the wiki page on X-Men. He tried to cover it up by claiming to be researching future host names. Quick thinking, Bob. Good job!

It does bring up a good point, though. Internal naming schemes are something that everyone has an opinion on, and a load of suggestions.

At various places, I’ve used greek/roman gods, Simpsons characters, beer companies, wine labels, and fish.

At my current company, we used the beer and wine names. We absorbed another company that used fish. It worked fine for a while, but we grew in terms of servers and locations until it got unwieldy to remember A) all the names, and B) what each name did. You’d also start to get very similar names after a while. We’ve now got 4 physical locations, soon to be 5, and something like 50-60 servers (not counting network devices), no one would be able to keep them all straight (including the admin).

To improve the situation, we’re in the process of changing to location-based hostnames with a flat internal domain structure. For example, the 2ndary application server in Ohio is oh-app2, with the fake internal domain name trailing. The alpha site’s primary fileserver is a-fs1.

It’s no where near as fun as “wolverine.internal.com” but it certainly does tell you where you’re connecting to and what the machine does. What makes it interesting is when you go changing things like CVS repositories on people’s machines, mail servers, etc. The policy we’ve taken is to alias the old information to the new, and slowly phase out the old method.

What do you use as internal naming systems? What do you think would make an excellent scheme? Make sure to check the list to make sure it hasn’t been done before!

When I first looked at house wiring in a moderately complex 8 story building, I was sort of mystified. It was only after literally tracing wires and numbers around the various wiring closets that I understood what was happening.

This howto deals specifically with 66-blocks, but 110 blocks are also becoming common, just not in my neck of the woods.

Enjoy!

Punch down blocks are used for when you need to run wires long distances, typically between distribution points ( things like the MDF, or Main Distribution Facility, otherwise known as the main telco room on the primary floor of the building), comms closets, and the like. They are used, rather than normal RJ45 jacks, because they are simpler, less prone to breaking, and don’t introduce much, if any, extraneous electrical interference to the wire.

For the next few paragraphs, please refer to the following picture, which is a clear, understandable example of a punch down block:

Those grey things in the middle are termination points for single wires. When you deal with punch down blocks, you deal in pairs of wires, and you get one wire to one grey clip. Pretend there is an imaginary line down the middle of that patch panel, because the pairs on the left are separate from the pairs on the right. If you number a line going across, 1 2 3 4, 1 and 2 are a pair, 3 and 4 are an entirely different pair. In the picture, you can tell, because of the numbering scheme they’ve used. There are 4 clips for a pair, but only 2 clips are wired in the beginning. Each pair is numbered, so the phone company can say “Turn on pair 3044″.

Now, at our building, the phone company’s lines are on the right side of the wall. On the left side of the wall is another huge array of punchdown blocks. These are for the “house wiring”. When they built the building, they pre-ran hundreds of pairs of wires to each floor from this room, so that they wouldn’t have to redo it every time someone ordered a T1. Each wire to each floor is terminated to a pair on the left side of the wall in exactly the same manner as the one on the right (including leaving one set empty).

To connect the two sides, you run a twisted pair of wires (it looks like you took a section of cat5, stripped off the sheathing, and just used one set of wires) from the right side of the wall (from the pair of grey clips we left open on #3044) to the left side of the wall (say #514, the 14th pair to the 5th floor, again using the empty grey clips). If you look again at that picture, you can see 3043 has been wired across, because all 4 wires are clipped in, but 3044 has not, since only the rightmost clips have wires.

At this point, you have two wires coming in from the phone company to the punch down blocks on the right. Then you’ve got wires connecting those punch down blocks to the “house wiring” punchdown blocks on the left. Then you’ve got vertical wiring up to the floor that the wire ends at.

In the comms closet on that floor (also known as the IDF, or intermediate distribution facility), you have a very similar situation. On the right hand side, you’ve got the punch down block where the vertical cabling from the MDF terminates, and on the left, you’ve got a punch down block where the actual wires that end up in your office are terminated. You use another twisted pair of wires to connect the two sides, and at that point, the wire that ends up in your office is connected directly to the phone company, albeit through several punch down blocks and lots of wire.

Now, when it comes into your office, hopefully someone has had the courtesy to install a patch panel for you. The patch panel looks like this on the front:

and this on the back:

As you can tell from the photo on the back, wires are typically matched up color for color when it comes to straight CAT5 cables. When it comes to things like wiring T1s, you’re only using two wires, so as long as you remember which one goes to what wire, you’re ok.

So, i review, we’ve got phone company wires coming in, and terminated in the MDF. They’re connected across to the house wiring, which is run vertically to the IDF, and from the IDF, it goes to your space. All of this is accomplished with those magic little grey clips.

Now, if only the wires would go in there. It turns out that there’s a trick. Or a tool, really, called a punch down tool (creative, eh?). The cheapest punch down tool I’ve ever seen is a buck. It’ll work in a pinch, but the one you want is here:

The way you use it is to arrange the wire you want to punch down against the metal clip. There’s a very thin slit in the clip where the wire will end up. Press the tip of the punch down tool against the clip, and push. The spring loaded mechanism (in the expensive tool) or your elbow grease (in $.99 model) will push the wire to the bottom of the slit, and in the process, scrap away the plastic or teflon sheathing on the wire, allowing the metals to make contact. The expensive model will then use the spring action to slice the extra wiring off the end, eliminating extraneous electrical interference (when you’re dealing with hundreds of feet worth of cable, this is a good thing). In the cheap model, I’d recommend an Xacto knife to do the job.

As for maintenance, there’s not really much that can go wrong in a patch panel, as long as no one comes in and starts pulling on wires. Typically there’s a plastic case that goes over the entire block to prevent accidental snags from pulling wires loose.

The best advice is to document everything you can. Leave a hard copy of the documentation in the comms closet so that you can see what’s been done. Lots of times, the telephone tech will “tag” the lines that he’s installed on the right hand side. The tag usually has the numbers of the pairs that are activated on the telco side, and the phone numbers (or circuit IDs) that match those pairs.

(Photos courtesy of lil 1/2 pint, techmsg, dmitrybarsky)

Today is the deadline for proposals for papers, talks, and such.

We’re a little low on submissions so I’d like to make one more “beg”. We’d love to have a talk about PHP for sysadmins, something fun you’ve done with Arduino, your favorite JS library, a walk-through on setting up Google Apps. Demo your favorite open source project, or propose a panel of people to talk about something you find interesting (I can help find others for your panel). It is an excellent way to spread the word about a project you are involved with.

We’ve tried to make the proposal process really easy. Just send your
contact info and topic plus a 1-2 paragraph description to
submissions@lopsanj.org

For more info, contact me and/or view:
http://lopsanj.org/events/picc10/cfp

BTW, today is the deadline but we can grant extensions to anyone that writes and asks.

Since it was originally published, it’s been viewed over 2,000 times and is still in the top 10 blog entries, which means that still, not enough people are covering the physical infrastructure aspect of system administration.

Enjoy!

Anyway, this Friday, I’m going to be showing you what I know about server racks.

I started out on a network that had a bunch of tower machines on industrial shelves; the sort you pick up at Harbor Freight or Big Lots. When we moved to racks and rackmount servers, it was like a whole new world.

Instead of shelves for each server, rack hardware holds the server in place, usually suspended by the sides of the machine. They allow the server to slide in and out, sometimes permitting the removal of the server’s cover to access internal components. Different manufacturers have different locking mechanisms to keep the servers in place, but all rack kits I’ve seen come with instructions.

To anchor the rack hardware (also known as rails) to the rack itself, a variety of methods have been implemented. There are two main types of rack. Round hole racks, seen at the left, require a special type of rack hardware. Much more common is square hole racks, which require the use of rack nuts. The rack nuts act as screw anchors to keep the hardware in place. Some server manufacturers have created specific rack hardware that fits most square hole racks, and don’t require the use of rack nuts. Dell’s “rapidrail” system is one with which I’m very familiar. Typically you get the option of which rail system you want when you purchase the system.

Installing the rack nuts is made easier with a specialized tool. I call it the “rack tool”, but I’m sure there’s another name. The rack nut is place with the inside edge clip in place, through the hole. The tool is inserted through the hole, grabs the outside clip, and then you pull the hook towards you. This pulls the outside clip to the front of the hole, securing the nut in place.

A typical server will require eight nuts, usually at the top and bottom of each rack unit, on the right and left sides, front and back. Each rack unit consists of three square holes, and a rack nut is put in the top and bottom of both the right and the left sides. Several pieces of networking equipment have space for four screws, but I’ve found that they stay in place fine with two. I can’t really recommend it for other people, but if you’re low on rack nuts, it’s better than letting the switches just sit there (and it almost always seems like you have fewer rack nuts than you need once your rack starts growing). If you only use two screws to hold in your networking equipment, make sure it’s the bottom two. The center of gravity of a rackmount switch is always behind the screws, so if the top screws hold it up, the bottom has a tendency to swing out, and that’s not good for your rack or your hardware.

While I’m on the subject of swtches, let me give you this piece of advice. Mount your switches in the rear of the rack. It seems obvious, but you have no idea how many people mount them on the front in the beginning because “it looks cooler” and then regrets it when they continually have to run cable through the rack to the front.

Once your rack starts to fill out, heat will become an issue. When you align your rack for your air conditioner, another bit of common sense that’s frequently ignored. Air goes into the servers through the front, and hot air leaves through the back. This means that when you cool your rack, you should point the AC towards the front of your rack, not the back.

Air comes in here… And leaves back here

It’s probably not a stranger to anyone who’s used a computer, but the cables seem to have a mind of their own, and nowhere is it more apparent than a reasonably full server rack. Many higher-end solutions provide built-in cable management features, such as in-cabinet runs for power cables or network cables, swing arms for cabling runs, and various places to put tie-downs.

There is no end-all-be-all advice to rack management, but there are some tips I can give you from my own experience.

Use Velcro for cabling that is likely to change in the next year. Permanent or semi-permanent cabling can deal with plastic zipties, as long as they aren’t pulled too tight, but anytime you see yourself having to clip zipties to get access to a cable, use Velcro. It’s far too easy to accidentally snip an Ethernet cable in addition to the ziptie.

Your rackmount servers will, in many cases, come with cable management arms. Ignore them. Melt them down or throw them away, but all they’ve ever done for me is block heat from escaping out the back.

Label everything. That includes both ends of the wires. Do this for all wires, even power cables (or especially power cables). Write down which servers are powered by which power sources.

If you have a lot of similar servers, label the back of the servers too. Pulling the wrong wire from the wrong server is not my idea of a good time.

Keep your rack tool in a convenient, conspicuous spot. I ran a zip tie through the side of the rack, and hang mine there.

(Some photos were courtesy of Ronnie Garciavia Flickr)

So, this coming May 14th will be Standalone Sysadmin’s 2nd Blogiversary, meaning it’s been around since 2008. When I first started it, I had a lot more time to write in it than I do now, which means that I wrote more entries, and because I was pretty much bursting with information that I’d never told anyone, a lot of the things that I wrote were probably more useful than the stuff that I put out on a daily basis.

I switched this blog from Blogger to its own domain in July of 2009, and even though I did import all of my old articles (they’re available in the convenient sidebar over there —–>) it takes extra effort to go back and read old entries (not to mention that there are over 400 of them! Yikes).

So this leads to the inefficient situation we have before us. I wrote a lot of (what I hope is) useful stuff, but I wrote it before the blog got popular, and most of you have never seen it. This week, I’m going to try to fix that.

I’m taking a week off of writing actual blog entries (honestly? not much of a change, I know. Sorry about that) to post some old stuff that I suspect very few of you were around for. I’ve also encouraged some other bloggers to do the same, because I’d love to see some of their great old blog entries that are still useful today. I know they’re out there, but none of us have time to weed through the histories to find them. This week is about bringing them into the open.

So please forgive me if this week seems like a flashback episode on your favorite sitcom, but I sincerely hope that you find the posts useful and informative. Thanks!

(Incidentally, while looking for the picture of the t-rex included above, I found this. wow!)

In any event, if you are on the pacific coast, go inland and uphill. It may be for nothing, but it may not be, too. Remember that loss of property is inevitable, but loss of life isn’t. Protect yourself and your family.

Also, know the warning sign of a tsunami. A waterline that recedes quickly means a tsunami is imminent. According to reports from the Boxing Day tsunami, human curiosity drove people to investigate the lowered waterline, which was responsible for many, many deaths. If the waterline recedes rapidly, run away. Run uphill. The reports from Juan Fernandez were of 40 meter waves, which sounds too high, but if that really is the case, that’s around the same height as a 13 story building. Don’t take chances.

Go inland if you’re in the affected area.

Registration is open and the speakers have been announced!

The cost is low, and the benefits are huge. I know from Google Analytics that this site receives hits from over 500 unique users in the region of this conference, every month. We don’t have that kind of space at the conference. It’s going to sell out at some point, so make sure that you talk to your boss now about attending. We’ve even drafted a letter to help convince them that it’s worth your time and their money.

It isn’t often that you get a local conference with speakers like Tom Limoncelli and David Blank-Edelman, not to mention Eben M Haber from the IBM research lab in Almaden, CA! This conference is going to get you the biggest bang for your buck out there.

What I’d like you to do is to help me get the word out. Please. Not everyone reading this is in the NJ/NY/PA/CT/DE area. For those of you who aren’t, please tell other people. Follow us on twitter at @picconf, email the site (http://www.picconf.org) to anyone you know in the area who might be interested, tell user groups about it, heck, we’ve even got a facebook page that you can become a fan of.

This is absolutely a grass-roots kind of effort. We have a very small advertising budget, so I want to use that as intelligently as possible. That means getting your help for the initial waves, and to spread it by word of mouth, by email, link, tweet, IM, and whatever else you’ve got.

A very big thank you to every one of you out there who reads this blog and supports me. I appreciate all of you.

Today’s XKCD is excellent, and already has a huge following from the sysadmins on twitter.

The alt-text is:

The weird sense of duty really good sysadmins have can border on the sociopathic, but it’s nice to know that it stands between the forces of darkness and your cat blog’s servers.

He’s right, of course. Sysadmins in general can develop a hero complex.

It’s a complex topic, but the smartest people in systems administration today (read: not me) have been vocal that sysadmin heroism should be discouraged. I can agree with that, to a point. We should never rely on heroism to save the day, because that means our designs have failed. When we stop believing in miracles and start relying on them, we have made bad design decisions and the reliability of our network will suffer.

On the other hand, there are sometimes events which happen that are beyond our control, and it’s up to us to make it right. In those cases, there’s no rule or mandate that says “you – sysadmin: go above and beyond the call of duty and be a hero!” I think it’s more our mental alignment that says “It’s my job to make sure that things work. In order to make things work, I’ve got to climb on top of the roof in the middle of a blizzard and restart the generator” (something my boss has done multiple times, and I’m sure some of you have as well). It’s just the way it works. We think logically, if the job needs done, and it’s our job to do it, then we need to do the job. The peripheral variables are unimportant.

I think the comic is hilarious, but like most Mission Impossible / Jack Bauer / Die Hard scenarios, it’s a rare event. Don’t go take ju-jitsu just in case someone cuts your network cables. Have a redundant infrastructure so that it doesn’t matter if they get cut.

As it turns out, someone else noticed, too. I found a very insightful column written by Steve Schwartz, called No One Knows What the F%#* They’re Doing. I feel like I should warn you…the language in that article is a bit strong…consistently strong. “Shit” is used 24 times, as far as my word search can tell, but that’s not counting the many graphics. I wouldn’t normally link to gratuitous language, especially when it’s completely useless, but the actual content of the entry is insightful.

The premise is that, when it comes to knowledge, there are three categories. There are things you know. There are things you know that you don’t know. And then there are things that you don’t know that you don’t know. The third type is by far the biggest category, and it’s by far the most dangerous.

Steve reasons that while structured education’s stated goal is to increase our direct knowledge, i.e. things we know, the real goal is to eliminate things from the third category.

Near the end of the entry, he makes reference to one of favorite (and least favorite, at the same time) cognitive biases, the Dunning-Kruger Effect, which stated simply says that people who are incompetent don’t know it. Read the wiki entry for that, and then mull over in your mind the logical paradox you enter into.

I flip flop back and forth between knowing that I’m incompetent and thinking that I’m awesome. I’m aware of the Dunning-Kruger Effect. What does that say about me?

Someone asked me, not too long ago, how my knowledge in Linux Administration rated. I said ‘expert’. Then I punched myself, because I’m an idiot. Not because I can’t administer a linux machine, but because ‘expert’? Really? Why would I even say that, when I know that’s not the truth? I’m not a liar, but for a split second, I was an incompetent idiot. Then I immediately gained competency (or at least clarity). That’s a very brief window in which a lot of damage could have been done.

Larry Wall says that hubris is one of the virtues of a programmer. I’ve sometimes thought about the role of hubris in system administration, too. Generally, I think it’s a pretty common side effect of playing God with the infrastructure. I suspect it’s more common in smaller companies than larger companies, but then I’ve got some cognitive bias of my own going on there.

Just in case Dell decides to yank that discussion, here it is, verbatim:

From: linux-poweredge-bounces-Lists On Behalf Of Philip Tait
Sent: Friday, February 05, 2010 4:31 PM
To: linux-poweredge-Lists
Subject: Third-party drives not permitted on Gen 11 servers

I just received my first Gen11 server, R710, with H700 PERC. I removed
the supplied drives, and installed 4 Barracuda ES.2s. After doing a
“Clear Configuration” in the pre-boot RAID setup utility, I can perform
no operation with the drives – they are marked as “blocked”.

Is Dell preventing the use of 3rd-party HDDs now?

Thanks for any enlightenment.

Philip J. Tait
http://subarutelescope.org

Howard_Shoobe at Dell.com Howard_Shoobe at Dell.com
Tue Feb 9 16:17:54 CST 2010

Thank you very much for your comments and feedback regarding exclusive use of Dell drives. It is common practice in enterprise storage solutions to limit drive support to only those drives which have been qualified by the vendor. In the case of Dell’s PERC RAID controllers, we began informing customers when a non-Dell drive was detected with the introduction of PERC5 RAID controllers in early 2006. With the introduction of the PERC H700/H800 controllers, we began enabling only the use of Dell qualified drives.

There are a number of benefits for using Dell qualified drives in particular ensuring a positive experience and protecting our data.

While SAS and SATA are industry standards there are differences which occur in implementation. An analogy is that English is spoken in the UK, US and Australia. While the language is generally the same, there are subtle differences in word usage which can lead to confusion. This exists in storage subsystems as well. As these subsystems become more capable, faster and more complex, these differences in implementation can have greater impact.

Benefits of Dell’s Hard Disk and SSD drives are outlined in a white paper on Dell’s web site at http://www.dell.com/downloads/global/products/pvaul/en/dell-hard-drives-pov.pdf

I have mixed feelings about this. From Dell’s standpoint, I can see wanting the customer to have the best possible experience, and trying to ensure data integrity by enabling drives that only have their specific firmware and so on. And to be honest, this sort of thing is very common in the SAN arena. Most SAN manufacturers openly state that only their drives with their firmwares will work with their arrays. Of course, the cynic in me wonders how much that has to do with technology and how much has to do with the 4x-5x multiplier premium they get for their drives.

That being said, the consumer in me rages. Look at the verbiage of Howard’s response. “There are a number of benefits for using Dell qualified drives in particular ensuring a positive experience and protecting our data.”

It’s not your data, Howard, and it’s certainly not Dell’s. It’s mine. And if I want to shoot myself in the foot by throwing in a drive with the cache enabled, warn me, but let me do it. If I ignore those warnings and deem my data unimportant enough, then that’s my problem. Your servers aren’t so expensive that I’m going to be leasing them in many cases. That means some day, I’m going to have a whole pile of old Dell servers that are unusable because I can’t upgrade to new drives unless I buy them from you, and I’m sure as hell not going to do that at what you charge.

Grow up, issue a firmware that adds the warning back in, and let us use our hardware how we want.

If you don’t listen to Mind of Root, you really should, if you can find time in your day. A lot of the focus is on Microsoft-related topics, but it certainly isn’t exclusively MS. In fact, in the same episode they talk about sysadmins and scripting, they talk about physical cabling, Cisco VPNs and one of my favorites, documentation.

I still wish there were micro-podcasts for sysadmins. I would love 5 minute shorts on topics or news in the industry. If I had more time, I would start one myself. Maybe later, unless someone else hurries up and jumps on it (*hint* *hint*).

To Steve and the guys at MOR, thanks very much for the mention!