268

You know, you see pictures like below and sort of chuckle until you actually have to deal with it.

I have just inherited something that looks like the picture below. The culture of the organization does not tolerate down time very well, yet I have been tasked to 'clean it up'. The network functions as it is, and there doesn't seem to be rush to get it done, but I will have to tackle the bear at some point. I get the ugly eye when I mention anything about weekends.

So my question goes, is there sort of a structured approach to this problem?

My Ideas thus far:

  • Label, Label, Label
  • Make up my patch cables of desired length ahead of time
  • Do each subnet at a time (appears that each subnet are for different physical locations)
  • Replace one cable at a time for each subnet
  • It's easier to get forgiveness than permision?

A Rat's Nest

ewwhite
  • 194,921
  • 91
  • 434
  • 799
Chad Harrison
  • 6,960
  • 10
  • 28
  • 41
  • 94
    Things like this were the reason that I insisted on seeing the infrastructure before accepting any offer. – gWaldo Aug 24 '12 at 16:11
  • 97
    My suggestion would be to carefully document absolutely everything and keep it to yourself. Then unplug one end of every cable one weekend. Call your boss and renegotiate for a much, much better compensation package. :D – HopelessN00b Aug 24 '12 at 16:40
  • 6
    make sure that any devices that support it are set up for dual-connections, both switch-to-switch and switch-to-device. that way you can do a lot of this work without needing downtime. i cleaned up a similar mess with only 2 hours of downtime to a couple of servers and one router (and no switches). but the whole repatching project took me 24 man hours, 20 of which were during business hours. – longneck Aug 24 '12 at 17:10
  • 20
    As a note for future changes, make sure you set up a standard for adding/removing/changing cables. Even if you can't get everything organized immediately, you can get everything organized progressively by making sure that all changes to the network happen neatly and consistently. Also, make sure everyone working with you is on-board with the new standard operating procedure. It doesn't matter how nice you make things, everything will end up a mess again if the people who are working in the system are unaware of or don't like the new system. – zzzzBov Aug 24 '12 at 17:15
  • 45
    Looks like we got ourselves a new 404 image for the site – squillman Aug 24 '12 at 23:53
  • 4
    @hydroparadise - Do you have a *real* photo of what you're working with? – ewwhite Aug 25 '12 at 00:58
  • @squillman I thought that was the 404 page at one stage... Clearly the current 404 is much worse with only one colour cable. I can see at least 3 here :-) And going to the site linked to from the 404 page, you can see this image has been submitted there "Messy Backbone!". – Mark Hurd Aug 25 '12 at 12:47
  • 3
    @ewwhite Well I thought about it before posting the question, I just didn't want to inadvertently post anything that might be considered 'sensitive information' due to high security concerns. Still actually considering it though. – Chad Harrison Aug 25 '12 at 21:57
  • 5
    If it is in fact a "rat's" nest, sometimes the most efficient method is known as the 'when in rome' approach, so my suggestion: chew through all the cabling. – Jimmy Hoffa Aug 25 '12 at 22:15
  • 2
    @hydroparadise The sensitive information is usually _inside_ the boxes, not _outside_ it. Please do post higher quality pictures. – Michael Hampton Aug 25 '12 at 23:24
  • 2
    Where can I get a camera that displays the data stored inside of a computer?! – gWaldo Aug 25 '12 at 23:36
  • 3
    @gWaldo The NSA souvenir shop at Fort Meade. – Michael Hampton Aug 25 '12 at 23:43
  • I meant that there's server names, IP addresses, etc posted all over the equipment. Yet, they are super security conscious. – Chad Harrison Aug 26 '12 at 00:26
  • 4
    One place I worked dealt with a situation like this by moving to a new building. – O. Jones Aug 26 '12 at 11:55
  • @hydroparadise: can you clarify if your organization frowns on *all* downtime, or just business-hours downtime. I was assuming the former, but one of the answers below assumed that only business-hours downtime was a problem. It would be helpful to know beyond doubt which one you meant. – iconoclast Aug 27 '12 at 18:25
  • 1
    @iconoclast Well there's two parts to the down time; we have end users insdide and then we have services we provide to our customers. Downtime with end users is somewhat tolerated which is during business hours, but downtime on services is not so much tollerated which is *all* the time. It's funny though because the only time they would be willing to have downtime on serices *is* during business hours so that we at least have people available for customer support to kindly say "Why yes, we are down. Is there anything else I can help you with?" – Chad Harrison Aug 27 '12 at 19:00
  • 1
    This whole post/thread should be a sticky or the ServerFault equivalent. SO many very precious advices and tips and ideas in it. Amazing. – ItsGC Aug 30 '12 at 14:35
  • 4
    Looks like Production source code to me. – P.Brian.Mackey Aug 30 '12 at 20:56
  • 1
    That is awful. You should post the after picture aswell. – Marko Sep 27 '12 at 22:22
  • 4
    did you ever get around fixing it or did you really end up looking for a new job? – the-wabbit Mar 05 '13 at 09:49
  • We finally did as of last week. Forgot that we actually do have "scheduled down-time" procedures for the rest of employees to operate under. We used an approach similar to Jak's answer, meticulously documenting EVERYTHING. We had a few switches that needed replacing for compliance reasons, so we took advantage of the upgrade to get the switches much, much closer to the patch pannel. Now, 90% of the patch cables are under 2' while consolidating a number of the smaller switches. Took 3 of us 4 hours to get it completed with everything back up and running. Still, not as nice as ewwhite's pic! – Chad Harrison Aug 29 '13 at 20:07
  • 2
    As a side note, BEWARE OF LOOPS! Can kill a network in seconds. – Chad Harrison Aug 29 '13 at 20:08
  • 5
    Show us your "after" picture! – Michael Hampton Feb 15 '14 at 19:22
  • 1
    Might I suggest the [XM42](http://xm42.com/). – MDMoore313 Apr 14 '15 at 22:49
  • The picture is a little misleading. People have written some answers assuming this is your actual server room. – Burhan Ali Jun 23 '15 at 14:15

15 Answers15

151

In no particular order here are some suggestions that have been helpful to me over the years-

  1. Can any of the equipment in those racks be eliminated, upgraded or consolidated? It's hard to tell what's there, but in my experience these kinds of messes tend to be aggravated by gear that should have been pulled out years ago.

  2. Once you've got some idea of the minimum set of equipment then consider how best to lay it out. The criteria here may vary, but grouping by technology type or business function might make sense. Clearly the proximity of high density devices (i.e. switches) and patch panels and such will immediately be apparent.

  3. Use cable management!!! There are both horizontal and vertical cable management solutions. Use both - horizontals around patch panels and other significant concentrations, verticals next to switches and to facilitate risers.

  4. It's always surprising, but how power cables are routed should be considered. UPS units in the bottom of racks, PDU selection and diversity all need to be considered before pulling a cable.

  5. Keep inventory of common cable lengths. It's late at night and you want to go home. A 3' cable is what's necessary but the closest you have handy is 5'. This is how these kinds of messes develop.

  6. Documenting is part of the game, but the importance of labeling cannot be overstated. With clear labels and efficient/clean cabling the number of mistakes will be vastly decreased and troubleshooting simplified.

  7. Limit who can pull cables!!! Differing styles and degrees of attention to detail can yield chaos pretty quickly.

Jeff Atwood
  • 12,994
  • 20
  • 74
  • 92
rnxrx
  • 8,103
  • 3
  • 20
  • 30
  • 1
    As much as I like the idea of getting another company to do the work, I beleive I would have hard time getting the budget passed by the board since the mindset will likely be "we already have the skillset in house". Asside from that, these are great suggestions. – Chad Harrison Aug 24 '12 at 18:54
  • 28
    +1 for inventory of cables. Unless you are very good and have excellent test equipment, buy certified cables of the needed lengths. Saves much grief – Dave M Aug 24 '12 at 19:30
  • 3
    I completely agree on the premade cables. Especially with the advent of cat6 and beyond the tolerances are much tighter around all terminations. I can crimp a beautiful looking patch cable and can make neat 110 punchdowns on a panel, but neither necessarily implies that I've built a completely functional network until it has been actively certified. While it might seem to cost a bit more, the truth is that buying these elements as premade (and standardized) as possible saves a great deal in terms of headaches and mystery outages in the future. – rnxrx Aug 24 '12 at 20:10
  • @rnxrx: Where do you get certified, premade pre-punched patch panels? *(yay, alliteration!)* I would love to be able to drop in some working panels with the cables already run, rather than pulling all that pesky solid core cable through the walls. – Justin ᚅᚔᚈᚄᚒᚔ Aug 27 '12 at 16:12
  • @Justinᚅᚔᚈᚄᚒᚔ Pre-punched? Not sure what that means, but maybe you're looking for [this](http://www.staples.com/Belkin-24-Port-CAT6-Patch-Panel/product_768311?cid=PS:GooglePLAs:Technology:Cables_Hubs_Adapters_&_Switches:768311:F4P638-24-AB5)? – NReilingh Aug 27 '12 at 16:43
  • 10
    If you're going to re-cable this whole mess, it seems to me to be the perfect opportunity to work out a color-coding scheme. You know (or will soon know) what you have, and can figure out what categories make sense and assign colors to each, and post the color key on a laminated piece of paper nearby so everyone else who works with it can be consistent with your scheme. And you can leave the current cable colors/tones (black, white, and gray?) as "non-categorized" while you use (e.g.) blue, red, green, yellow for new cables. It'll also be easy to see what's been replaced and what hasn't. – iconoclast Aug 27 '12 at 17:04
  • @NReilingh: I was being a bit flippant, the wording of rnxrx's comment makes it seem like he was suggesting using "premade certified punchdowns", which is obviously impossible with current technology. – Justin ᚅᚔᚈᚄᚒᚔ Aug 27 '12 at 18:23
  • 1
    Sorry - my wording may have been unclear. "These elements" referred to the patches themselves. In the case of fiber, however, the use of MPO cables is a big win. Pre-made cables with modular connectors, snap-in breakouts, etc. No help as far at twisted pair, but potentially a godsend for horizontal scaling of fiber and the use of 40GE or 100GE. – rnxrx Aug 27 '12 at 19:37
  • 1
    @Justinᚅᚔᚈᚄᚒᚔ A whole number of cable manufacturers (including international ones like Panduit or Corning) offer pre-fabricated and certified cables with Cat6(A) module termination at various lengths. Take a look at the Catalogues for details: http://catalog.corning.com/CableSystems/en-GB/catalog/CategoryBrowser.aspx?cid=preconnectorized_copper_cable_assemblies_web&rot=copper_cable_assemblies_web. If your premises allow for it, installation is as simple as pull & click, although excess cable lengths have to be dealt with. – the-wabbit Mar 05 '13 at 09:22
141

Please call a cabling contractor in to spend a day or two onsite to "dress" your cables. I used to spend time dealing with this type of work on my own, but realize that cabling contractors are faster and more organized.

A good cabling contractor is better at this than you are!

They will have the right resources to tag, test, get custom lengths, dress cabling and install the right type of management infrastructure.

It's worth the time and expense in order to end up with a cleaner, more manageable solution.

If cost is an issue, then you can reduce the hired man-hours by bringing the contractor to handle any structured cabling needs (re-terminating patch panels, testing existing runs, etc.)... Looking at that photo, there will be a need... You can buy or have them provide a variety of patch cable lengths and perform that work on your own, tagging and labeling as needed.

I went from this before-to-after by using this approach. It took 6 hours of contractor help, and 8 hours of my work to get the result.

enter image description here

enter image description here

ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • 89
    That's pretty... – Aaron Copley Aug 24 '12 at 16:51
  • 6
    moonraker. what a lovely name for a cisco :) – petrus Aug 24 '12 at 16:52
  • 1
    You make a very compelling arguement for using a outside company. – Chad Harrison Aug 24 '12 at 16:59
  • 3
    The contractor did the patch panels, the 2-post rack installation, vertical and horizontal cable managers and ladder work. I performed the patching and color-coded the subnets. – ewwhite Aug 24 '12 at 17:04
  • Very well done. Cabling like this should be a standard but sadly you get people who go in plug in cables and let them just hang there. Color coding the subnets is a great idea. I've met some people who even label both ends of the cables which state exactly which port/switch/rack they plug into. – Patrick Aug 24 '12 at 18:33
  • If I can convince the higher ups to bring in an outside company, I'll gladly mark this correct correct answer. – Chad Harrison Aug 24 '12 at 18:57
  • How much money was spent on materials? – sehe Aug 24 '12 at 21:05
  • 1
    Estimate $2500-$3500 US in rack equipment. $300 US on cables. – ewwhite Aug 24 '12 at 21:19
  • 8
    Two problems with this because its a one off a) often pro cabling jobs will leave it hard to make changes b) cables get messy because of changes often in a rush, so it requires a change in procedure for tidiness to last – JamesRyan Aug 24 '12 at 21:43
  • 6
    @JamesRyan I don't see how my example above would be difficult to manage or modify. As for quick changes, it's possible to perform them outside of the cable management system, but it means that those changes will have to be corrected eventually. That's a discipline/organizational issue, not a problem with using a contractor. – ewwhite Aug 25 '12 at 11:03
  • And when you have regular quick changes to do, when do these eventually get corrected? The point is that the problem IS a discipline/organizational issue, 80% of the people here could run cables as neatly as a contractor if they had the convenience to do so. The thing is, that in itself that doesn't actually solve the problem. – JamesRyan Aug 26 '12 at 01:58
  • 1
    @AaronCopley Should see equinix in Chicago... (where some of Google is) they are like uber-pro cablers. – Ben DeMott Aug 27 '12 at 01:28
  • 1
    @BenDeMott They can be, but usually the rack-level cabling at Equinix is up to the customers. I'm responsible for a couple of messes there. The structured cabling is excellent. [Great facility](http://flic.kr/p/8aB4rw). – ewwhite Aug 27 '12 at 01:44
  • @ewwhite Thanks for sharing the photo, brings back good memories :) – Ben DeMott Aug 27 '12 at 02:01
  • +1: another awesome answer by ewwhite. – iconoclast Aug 27 '12 at 17:06
  • 1
    Great, but that doesn't look like the same rack/equipment in the pics? Was lots of it changed? – HaydnWVN Sep 20 '12 at 11:22
  • 2
    @HaydnWVN Yep. [The old rack was removed and the old cabling had to be destroyed to move the rack](http://i.stack.imgur.com/zuS4c.jpg). Total cleanup. – ewwhite Sep 20 '12 at 12:15
  • I've been doing similar here but all in-house... From a [16U wall mount mess](https://dl.dropbox.com/u/66653574/Patch%20cabinet%20original.JPG) to a [42U floor standing 20" deep MONSTER](https://dl.dropbox.com/u/66653574/patch%20cabinet%20now.JPG) (with an NVR in the bottom, ignore the labels as i've still lots of tidying to do lol) – HaydnWVN Sep 20 '12 at 13:32
  • 2
    I'd be happy if we could even get as clean as your "before" picture. – Joel Coel Apr 01 '13 at 18:47
  • If you have the time, you can always do a good job. Being organised is a highly transferable skill, PM, cabling, electronics, life.. take a pick. – Arthor Apr 27 '13 at 15:01
69

A co-worker and I recently cleaned up a mess that was pretty bad (I might post pictures later if I get some time) and I wholeheartedly disagree with the contractor approach. You will learn significantly more about the system itself and what deficiencies it has if you do the work yourself. Also when you make a mistake, as you or any other mortal is likely to do, you will have a much better idea of how to fix it. Having said all of this here are my suggestions.

  • Take your time: this is a marathon not a sprint.
  • First remove all the items that are not plugged in on one end. This took a surprising amount out in our situation.
  • Label, label, and label again.
  • Do a switch at a time.
  • Take your time and check your work.
iconoclast
  • 1,688
  • 2
  • 18
  • 30
PlTaylor
  • 1,067
  • 7
  • 8
  • 43
    +1 for the "remove all items that are not plugged" – woliveirajr Aug 24 '12 at 18:46
  • 2
    re "You will learn significantly more about the system itself and what deficiencies it has", what if you DO NOT want to learn about the system deficiencies? :) Really you will sooner or later but it's about choice... – akostadinov Aug 24 '12 at 20:33
  • 1
    The sooner you get neck deep in the problems the sooner you can proactively fix things. I have a huge preference for doing things myself whenever I can. You have fiber to run between buildings, hire a contractor. You need to splice a couple cables or buy pre-made lengths, do it your self. You will be happier in the long run when you have a more intimate knowledge of the system. – PlTaylor Aug 24 '12 at 20:52
  • 5
    I don't see why it has to be an either/or proposition. Why couldn't you work *with* the contractor, and learn from him, *and* benefit from his expertise. (BTW, +1 for the excellent points you make, even though I only partially agree.) – iconoclast Aug 27 '12 at 17:13
  • 1
    Nothing is black or white and this isn't an either or proposition. My personal bias is to do everything I can that will help my understanding of the system and how to troubleshoot it later. If the cabinet is a mess chances are there is a lot more wrong with it than just a rats nest. – PlTaylor Aug 27 '12 at 18:23
56

My sympathies to you. I was tasked with a similar problem for a number of cabinets, equally as horrendous.

The approach I took was as follows:

  1. Use a spread sheet to make a list of which port is connected to which port on which piece of equipment (arduous manual process of cable tracking). Try to use a spread sheet cell to represent a port, and order collections of cells as a visual representation of each piece of equipment.
  2. For each port, also note the VLAN/subnet etc (I did this via a web interface/cli)
  3. Write a small macro to theoretically re-wire your equipment based on cabling distance, or other factor of your choice. The result should give you which switch port maps to which patch panel port (or other equipment). It should also indicate how many cables you need and how long they need to be. I did this in a spread sheet by using cells as switch/patch-panel ports and it worked well. It produced a visual diagram of the wiring by drawing lines between cells and colouring the cells with conditional formatting to show their VLAN number(s).
  4. Once I had the schematic it took less than an hour after working hours to pull out all the cables and patch them up according to the generated diagram. Then another 30 minutes the next day to fix the "why can't I connect?!" problems :-)

There's really no easy way to do it, but at least the above approach saved me from pulling my hair out.

Good luck!

Screen shots:

Before:

Before

After:

After

Jak
  • 998
  • 9
  • 12
  • 5
    +1 for the spreadsheet (idea and shots). – Henk Aug 28 '12 at 18:28
  • 7
    @Jak care to share the spreadsheet with the re-wiring / VLAN reconfiguration algorithm? – the-wabbit Mar 05 '13 at 09:36
  • 3
    Maybe a program like Eagle PCB Design program (used to design circuit boards of schematics) can help to layout wiring using the built-in routing algorithm they are designed to avoid crossing wires, which will help solving the nest. other similar program will probably do the job – Vitim.us Apr 06 '13 at 23:57
  • 4
    I would love to share the code but I did this as part of a contract so I don't officially own it :-( Its done using VB macros in Excel by calculating the distance between cells. – Jak Apr 09 '13 at 15:17
  • _Spreadsheets_. Our systems professor once said that Excel is _the_ most important piece of enterprise software in the world, there are even banks that run solely on Excel. But this is the first time I see Excel used for cabling itself! – Pavel Jun 21 '17 at 08:00
27

All these answers are incorrect because they omit the critical first step: Define the project. Before you can use any of these ideas (many of which are very good), you need to be able to answer the following questions with absolute certainty

  • What systems you are touching-- when and for how long can they go down?
  • Which departments will be hit and when will they agree to be inconvenienced?
  • If there is a critical system, when can I shut it down? is it backed up?
  • How much money can I spend to do this?
  • How many people can I borrow?
  • How long can I take? (meaning both man-hours to do it and time when it must be done)
  • Is there any documentation and should I trust it?

Maybe you know all these things and you didn't go into detail. But unless everyone is on board with your plan, you can end up writing one of those "How I blew up my company" Infoworld columns.

I had this situation once. Turned out we had a box that had to be on and connected. Pull a wire for too long, it would hiccup and crash. If it went down, anyone using it was screwed. Plus, two other systems-- who rarely used it but continually monitored its status-- would freak.

It only took a few minutes with the manual to figure out the order the systems needed to be shut off and a few phone calls to schedule it and one "Don't log into this system remotely over the weekend!" mass e-mail. But that's the difference between nobody noticing and people hating you forever.

Management's responses to definition questions will help you plan other projects. Your work requires them to make occasional choices and you need to learn what, how and why they will choose. You might find you can't hire anyone to help, can't buy 1-foot cables to replace 8-footers, can't shut people down, can't stop taking "can you unjam my printer?" calls while you do it, can't get overtime and can't come in early, stay late or work weekends if it means you won't be there M-F. If so, you know you need a new job.

Geoff
  • 371
  • 2
  • 3
  • 5
    +1 for recognizing that this *is* a project. It's easy to overlook. – Chad Harrison Aug 30 '12 at 15:10
  • 1
    Thank you. All the work you perform in network administration or help desk is a project. "Reset password" or "change toner" don't require much definition... but the fourth time you unjam a printer, it's time to add tasks like "inspect rollers" or "call vendor" to the plan. – Geoff Aug 30 '12 at 15:19
20

If you cannot (for whatever reason; it's usually that the company doesn't want to spend the money) get a contractor:

  • Schedule a weekend outage. Nothing will be guaranteed to work during this time. If you can do full shutdowns, so much the better.
  • Get the boss to agree to comp your food, drink, and time.
  • Beforehand, map each cable run, source and destination.
  • Install the cable management
  • Have appropriately-sized cables on-hand beforehand. If you must crimp them yourself, do it before the outage. Label both ends of each cable now.
  • Pre-print labels. (BTW, buy a labelmaker, extra labels, and spare batteries.)
  • Replace one cable at a time (unless impossible).
  • Turn things back on with fingers crossed.
  • Test that things still work. Reboot a client PC and make sure that you can still get to things.
  • Go home victorious
gWaldo
  • 11,887
  • 8
  • 41
  • 68
  • 7
    "Test that things still work. Reboot a client PC and make sure that you can still get to things." - You cannot complete this step without first documenting what can access what. 'get to things' for an IT person may be completely different from 'do my job' to the actual users. – Freiheit Aug 24 '12 at 19:00
  • 3
    @Freiheit Your comment strikes me as a bit pedantic. Was I supposed to have an earlier step that says "document everything that all of your users could possibly need to do their jobs", and replace "can get to to things" with "attempt to do the entirety of their job"? Perhaps that would be the ideal in thoroughness, but it strikes me as excessive. – gWaldo Aug 24 '12 at 19:16
  • 9
    "document everything that all of your users could possibly need to do their jobs" is clearly excessive, but "can get to things" is incredibly vague. Especially after working a full day, off hours, and you just want to verify that the job is done and go home. Having a well defined and clear punch list of what things to check is important. It can be as simple as "Login as userbob on secretary workstation and access foo-app on jiggle-bunz-server". Just know what your requirements for "the job is done" are as best as you can. – Freiheit Aug 24 '12 at 19:30
  • 1
    Having an acceptance test is good. Having a fallback-plan when that test fails is better. – David Schmitt Aug 27 '12 at 10:50
  • 4
    sorry, jiggle-bunz is the name of my next loadbalancer. – ItsGC Sep 07 '12 at 06:09
15

I am amazed no one else has already said this:

Yes document, record, spreadsheet etc etc but photo photo photo and more photos. They won't help you put it back together, but they will save you going insane when you inevitably have to problem solve. Plus when it all goes wrong they will serve as a reminder why never to do it again! :)

I also like the comment of "removed things which aren't plugged in to anything" its amazing how much stuff like that there is!

Tim
  • 151
  • 2
14

If you can get a contractor to take care of it as ewwhite suggests, then let them figure it out.

Otherwise:

  1. I'd start by identifying exactly where everything is and where it goes.

  2. I'd suggest keeping an inventory (simple spreadsheet should work to start) of each device and it's connections, VLANs, source port, destination port, IPs, and any other pertinent information.

  3. Have your cables pre-made and pre-labeled before you start. Use cables that aren't overly long but still leave room for some movement/ slack.

  4. If you can't do it all in one go, start device by device.

  5. Use vertical and horizontal cable management as needed, and velcro to secure sections (generally every one foot).

  6. Something that may help, is to use a different color cable for your each logical subnet to help visualize things in the future.

  7. Test to make sure things are as they were before you started. It's useful to have a pre-made document of tests to perform so you can go through, perform the test, and document the result.

After you are finished, make sure to keep up with the cable management and avoid quick runs to get something up and going. You'll almost never come back to properly finish the job. You may also want to develop a standard policy/ procedure for all cabling. That way, moving forward, you won't create these sort of issues. The person inheriting it from you will thank you :)

HostBits
  • 11,776
  • 1
  • 24
  • 39
11

On labeling cables

  • I learned one important rule of labeling cables: Don't label your cables with the device/port, unless this is an absolute requirement and you're given the resources (label printer, blanks, where and when you need them.) If you do, the moment you have to repatch a cable you will be required to relabel the thing.
  • The best labeling scheme that I worked with involved labeling each cable with the same tag on both ends, perhaps followed with an 'A' or 'B' tag. That way, when you run across all of your panels and devices, you can match end-to-end connectivity with relative ease.
  • In the past I've worked with custom made cables, which actually had an offset length printed in ft and/or m. on the cable at regular intervals. This helped in actually identifying cables in a big ball of spaghetti. However, prefab cables are much more reliable but sadly lack that length indication.
Henk Langeveld
  • 1,294
  • 10
  • 25
  • 2
    You absolutely should label the cable with device/port **but only the device/port name at the end of the cable which is less likely to be unplugged when re-routing**. If you are routing cables from servers to switches, label the cable with the server name and interface number (of the server, not the switch) on both ends. If you are routing cables from switches to patch panels, the choice is more difficult and depends on your recabling habits - just *define* the end you are not going to plug when re-routing and use its name as the patch cable label. Again, on both ends. – the-wabbit Mar 05 '13 at 09:43
  • 1
    I really like your idea of mid-span cable labels. I always label both ends of each cable with a serial number and the length. Having the length labeled is handy when you're reaching into the cable box. I use a Brady ID Pal cable label maker. Be sure to get the "industrial" label tape -it actually sticks more or less permanently, unlike typical label maker tape. – Dan Pritts Mar 04 '16 at 17:49
10

Wow is all I can say. Here are my suggestions for all it's worth:

  1. Don't look to tackle this in a specific time frame. With a mess like this there's no need to put undue pressure on yourself to have it cleaned up by X date.

  2. Map out and document all of the physical connections (port x on device X connects to port x on device y, etc.).

  3. Map out the network logically (subnet A is for office A, subnet B is for DMZ, etc.).

  4. Figure out how you're going to organize/route the cables that takes into account the new layout in existence with the current layout. A cabling contractor can help with this. I'm not sure I'd want them doing the actual work though as they won't have any knowledge of network topology/dependencies and if things get connected incorrectly they'll be no help in figuring it out.

  5. If your proposed layout allows for you to swap/reroute one or a few cables at a time then do it that way. Find a few minutes every day where you can swap/reroute a handful of cables into the new layout. You'll need to plan for small windows of downtime/unavailability for those systems whose cables are being swapped out. As you stated it's easier to ask for forgiveness then permission but you don't want to over do it.

joeqwerty
  • 108,377
  • 6
  • 80
  • 171
7

Just thought I would post with something NOT to do.

I was in a similar situation a while ago although I only had two racks to deal with and not quite so many wires.

I thought I had a great idea which was okayed by the boss - the idea being that TCP/IP on windows is tuned to retry and timeout after 5 secs so if a cable is unplugged for a second or two it's not a big deal.

So I commandeered an assistant and we would get the new cable in place all nice and tidy, then simultaneously we would swap both ends with the old cable.

This was when we discovered one application we needed that was dependant on a flat file database was very sensitive to this and it's db got trashed so we had to restore that from backup!

Robin Gill
  • 2,503
  • 13
  • 13
5

My only suggestion would be about roll back process.

If you can yourself or one of your internship guys would take a picture of every single step. Or record video of important steps in case of there is a problem and want to rollback process.

I will be helpful also for further references if somebody else will join your team and tell him what you did.

teoman
  • 51
  • 1
4

If your boss is not willing to give you overtime, yet also wishes to avoid downtime as much as possible, it might be a good idea to inquire about coming in around 3pm one day, and staying late. This way, there will be no downtime and the company won't be ponying up for overtime. I've pulled jobs like that in the past, when the company didn't want to overtime me, and it was honestly fun. Bring some speakers with you, and your ipod, and cable like a madman to some metal.

acolyte
  • 417
  • 2
  • 13
  • How would this prevent downtime? Maybe no downtime during business hours, but it's not clear from the question that that's okay. I assumed they did not tolerate downtime at all, but perhaps he meant only during business hours. (At the company where I work, downtime would affect end customers, local retail stores which in some cases are open 24 hours a day, as well as other businesses that communicate with our systems, so any downtime at all is a problem.) – iconoclast Aug 27 '12 at 17:20
  • 1
    @iconoclast that is a very good point. I had assumed this was an office-setting where after hours downtime would be fine. – acolyte Aug 27 '12 at 18:08
3

I think nobody mentioned this yet:

I would ask my boss whether we could get a full new round of equipment to replace the old installation in new racks. Depending on its age that might even be cost effective.

ZaphodB
  • 653
  • 3
  • 9
  • +1 : when touching anything 5 or more years old, it should next be placed into the trash bin. With cable end(s) cut off. – Krista K Feb 10 '14 at 09:47
2

I have an idea that might also help, I think someone already mentioned color coding your LAN segments, but to make life a little easier you could simply buy whatever patch cables you will need in advance, rather than building them yourself and testing. I use a site called Monoprice, they are awesome, they have various cable length options and the cables are cheap. I will post a couple of links to the site below, hopefully this will help make the job go easier.

Cat 5e Ethernet Cables

Dave M
  • 4,494
  • 21
  • 30
  • 30
Frank R
  • 141
  • 3
  • Monoprice is cheap but i've had quality problems with items bought from them. CDW has pretty cheap prices on their house brand cables, which I believe are made by Belkin. I'm sure there are other inexpensive vendors. – Dan Pritts Mar 04 '16 at 17:53