10

I understand the basics of spanning tree, but that's about it. I'm hoping that someone can tell me if this will work as I want it to.

I've got two Cisco ACE load balancers setup for redundancy. Each ACE is connected to its own layer 2 switch. Currently each layer 2 switch is connected to its own 1 Gig fibre link to the CoLo. Each fibre link is setup for a different IP Subnet and our CoLo doesn't offer handling the BGP for us. We have to purchase our own routers to handle that, which is a project which is coming up.

(I've got the layer 2 switches in there because the CoLo provides fibre and the ACEs only have copper so all the switches are doing is changing the fibre to copper for me.)

So currently I can only use a single load ballancer because the ACEs don't support spanning tree. Now the layer 2 switches to support spanning tree, it is just disabled by default. Now if I were to enable spanning tree, and cross connect the layer 2 switches would everything work as expected, or would this cause the network to come crashing down?

I'm hoping to get some expert advise before I try this as it is a production network and I don't have a couple of extra Cisco ACE load ballancers to try it with in a lab.

UPDATE: based on the comments I need to include a diagram. Here's what we currently have.

The problem is that the second ISP link isn't usable to us at the moment because of the lack of BGP. So I want to cross connect the two network switches together. I've been told that if I connect the two ACEs together that'll complete the circle and cause network problems. So if I can connect the switches at the top which support spanning tree that should take care of the problem. Eventually there will be two routers between the top switches and the load ballancers to handle the BGP over the two network links.

Am I making sense? Sorry this is such a mess, I'm much more at home in SQL Server then Networking.

Glorfindel
  • 1,213
  • 3
  • 15
  • 22
mrdenny
  • 27,074
  • 4
  • 40
  • 68
  • It would be useful to know how the L3 traffic is going to get routed. What is the default route of the servers and where does that device sit in this? – chris Jul 08 '09 at 14:07
  • Chris, currently the ISP provides the L3. Eventually I'll be buying routers and putting them between the ACEs and the top network switches. – mrdenny Jul 08 '09 at 18:41
  • Thanks everyone for the sanity check on this. I'm happy to report we now have redundancy on our load balancers. Here's hoping we never need it. – mrdenny Jul 11 '09 at 00:17

5 Answers5

2

Update after you provided a diagram:

You already have a circle there at the bottom half of the diagram. It looks like the ACEs don't bridge, so if you don't have a problem there you shouldn't have a problem connecting the two top ones.

It's a bit hard to talk about the diagram if you don't name the devices, but let's say I name them left to right, top to bottom. You have a circle ACE1-SW3-ACE2-SW4-ACE1..., obviously there's no problem there (right?). I'm guessing you configured the ACEs so they don't bridge any traffic at all, and therefore no loop.

Why not connect ACE1 to SW2 and ACE2 to SW1? Then you have the same setup as the bottom part.

If you have a different VLAN in the top and bottom parts (not the same layer2 segment) then you can't have a spanning tree loop between them.

It would be clearer if you provided (obfuscated if you like, but make sure we can tell network A from B. Such as 10.123.0.0/24 and 10.123.1.0/24) IP networks on the map, and perhaps VLANs (if you use them).

Update after naming the switches:

If the ACE do routing, and therefore are the next-hop for the servers on 10.0.0.0/24 etc.., and don't do bridging (in the ACEs), then connecting the way I said above is safe.

Thomas
  • 1,446
  • 11
  • 16
  • I've added more info above which will hopefully show better when I've got now. Currently ACE2 is useless to us as the network link that it is connected to isn't used for the subnet as our colo won't handle the BGP for us. I'm trying to make it so that if ACE1 needs to be rebooted it can be without taking a site outage. – mrdenny Jul 08 '09 at 00:41
  • I've updated the pic with names to hopefully make it easier to talk about. If connecting ACE2 to SW1 (names from the updated pic) and ACE1 to SW2 will work without causing any problems I'll go for that. I'm heading up to the CoLo in LA tomorrow so I can try that then and make sure that its stable before I leave. Is there more info that I should add so that you see what we've got setup? – mrdenny Jul 09 '09 at 20:18
  • I connected the cross connect today. Everthing appears good, we can access the management IPs of both LBs from the internet and the network didn't come crashing down. (Granted I'm at the Starbucks across the tree at the moment, just in case.) – mrdenny Jul 11 '09 at 00:06
1

I think that what you want is to:

  1. Make ACE2 into a failover peer using the "ft peer" CLI commands
  2. Connect ACE2 to the same switch that ACE1 is connected to.

This gives you box redundancy (the ACE's pass heartbeat information between them) and switch redundancy (the downstream Catalysts are cross-connected). You aren't, of course, protected from a co-lo switch failure.

Are you sure you want to go BGP? Who's providing the ASN? Is your downstream network fully portable? You need to make sure that you've got a network architect who can explain the pros and cons to you.

If you don't do BGP, and your Catalysts are capable of layer-3 switching, then you might want to:

  1. Use VLANs to carve the Catalysts into virtual switches: inside and outside
  2. Use a floating static route or route policy to send packets to the desired ISP.

There's a few ways to solve this. You would be well-served by a few hours in front of a whiteboard.

ironchefoklahoma
  • 21
  • 3
  • When we move to BGP well be getting as ASN via our CoLo from ICANN (or who ever issues them, the CoLo gets the paperwork from us and gets it to the correct people). This whole mess started because we switched CoLos just before move in (management got involved) and the old CoLo handled all the BGP, but the new one doesn't and the hardware had all already been ordered. Currently ACE2 is already in failover mode. I can connect them to the same switch and be done with it? Won't that create a loop? – mrdenny Jul 08 '09 at 18:44
  • If the 2nd ACE is in failover mode it should be passive until it receives a notification from the other ACE. *Should*. I've been reading the 4700-series documentation and it's just awful. Although it states that the ACE's "don't support spanning-tree" it doesn't explain if they don't participate (like a host) or just pass the spanning-tree packets through (like a hub). I'd be heavily surprised if the ACE actually forms a loop. However, your Catalysts have STP on (unless you've explicitly disabled it) and they'll break any loop. – ironchefoklahoma Jul 09 '09 at 20:44
  • So you should be good to plug it in. – ironchefoklahoma Jul 09 '09 at 20:45
  • Glad the cross-connect was successful. – ironchefoklahoma Jul 12 '09 at 19:13
0

I have a similar situation where I have a quad port NIC that funnels FTP traffic from two core switches, and a myriad of remote switches.

I currently have the two core switches and quad-port NIC connected in a triangle, two Gig links on each side. Each of the pairs of GigE is aggregated to give a 2Gig link. I am using Intel NIC adaptive aggregation mode. That seems to work well.

If you had a single layer 2 switch with 2 Fiber ports up-linking to your colo, then the switch would be similar in spirit to my NIC. Then connect the Cisco devices to the switch, and you're good to go. But I do understand that you are losing redundancy here.

You're trying to create a fully redundant path from colo to your Cisco devices, correct?

Dennis Williamson
  • 60,515
  • 14
  • 113
  • 148
GregC
  • 879
  • 2
  • 8
  • 24
  • Correct, the end goal is a fully redendant path from the CoLo to the Cisco devices. This is a stop on the road to that goal until routers can be ordered and delivered. I've added more info above which will hopefully show better when I've got now. – mrdenny Jul 08 '09 at 00:40
  • Thank you for adding the details. I am with you, much more comfortable with a C++ or C# compiler. – GregC Jul 08 '09 at 12:49
0

As long as the two networks that are behind the ACE's aren't connected then you will be fine.

Suroot
  • 171
  • 2
0

If you have enough interfaces on the ASAs connect both ASAs to both (isp facing) switches and run vrrp or hsrp on the ASAs and then do policy routes or whatever to distribute the traffic across both links.

example from cisco

chris
  • 11,784
  • 6
  • 41
  • 51
  • We have ACEs not ASAs. When it comes to routing the ACEs are very limited. – mrdenny Jul 08 '09 at 05:52
  • What device in this picture is your default route? – chris Jul 08 '09 at 13:08
  • I've updated the picture to answer everyone's questions. The active route is the one connected to SW01. I'm basically trying to get ACE2 access to the upstream network that SW01 is connected to so that if ACE1 is rebooted ACE2 can takeover as the active load balancer. – mrdenny Jul 09 '09 at 20:15