Trouble configuring standard VLANs on BNT G8264 and ESXi 5.5

Question

I have a BNT (now an IBM) RackSwitch G8264 and am having problems trying to get standard VLANs configured and working with my two ESXi 5.5 hosts.

I've followed the documentation (which can be found here) for the version of switch software I am running, and I believe I have configured the VLANs correctly. However, when I create a standard port group on both my ESXi hosts and tag them with my newly created VLANs, traffic does not work. If I do not tag any traffic on the standard ESXi port group, everything works fine.

Here is my current setup:

2 x ESXi hosts running 5.5 update 1.
Standard switch with one physical adapter uplink.
One port group tagging on VLAN 202 with test VMs connected to them.
Switchports set to trunk mode with a default VLAN (for untagged packets) of 1 and allowed VLANs of 1 and 202.

If I set the VLAN ID in both port groups on each host to 202, the VMs on each host can not ping each other.

If I set the VLAN ID in both port groups on each host to 0 (None), the VMs on each host can ping each other successfully.

If I set the VLAN ID in both port groups on each host to 1, the VMs on each host can not ping each other.

So it seems that if I set any kind of VLAN tag on my port group on my ESXi hosts, communication fails.

What can I use on the switch or my ESXi hosts to inspect the frames to see if the tagging is happening? As far as I'm aware, you can't install Wireshark on a switch?

Any assistance would be appreciated.

Here is the entire dump of my switch config:

version "7.8.1"
switch-type "IBM Networking Operating System RackSwitch G8264"
iscli-new
!
ssh enable
!
interface port 53
    description "ESXi 1"
    switchport mode trunk
    switchport trunk allowed vlan 1,202
    exit
!
interface port 55
    description "ESXi 2"
    switchport mode trunk
    switchport trunk allowed vlan 1,202
    exit
!
vlan 202
    name "VLAN 202"
!
!spanning-tree mode disable
!no spanning-tree stg-auto
!
!interface ip 1
        addr <default>
        enable
!
!interface ip 128
        addr <default>
        enable

!end'

You can't install Wireshark on a switch but you can install it on a workstation and then have the switch trunk port traffic mirrored to the workstation switch port. Run Wireshark on the workstation and you should see all of the traffic transiting your switch trunk port. I haven't used VLANs with vSphere but I'm assuming you should see tagged frames being sent out of your switch trunk port to the vSwitch, which in turn should remove the tag before sending the frame to the VM. — joeqwerty, Jun 23 '14 at 13:28
The vSwitch should have no special configuration. Your port groups should be tagged appropriately. Are you relying on a default VLAN 1 for general traffic? I tend not to use VLAN 1 for anything once I start introducing other tags. — ewwhite, Jun 23 '14 at 14:04
on the vSwitch properties did you confirm that on the `NIC Teaming` tab that `Load Balancing` is set to `Route based on the originating virtual port ID`? Also, it looks like the vSwitch dropping VLAN 1 frames is the expected behavior (assuming that VLAN 1 is the default/native VLAN) - http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004074 — joeqwerty, Jun 23 '14 at 14:27
"You can't install Wireshark on a switch"… but @joeqwerty on the good switches you can have them capture `pcap` files :) — MikeyB, Jun 23 '14 at 17:15
Thanks for the comment. I only have one physical NIC in my standard switch, so shouldn't need to configure any load balancing or teaming. At this stage, I'm thinking it's more of a switch problem as I have similar hosts setup in another environment with Netgear switches, and they work fine. — cpjones44, Jun 23 '14 at 22:31

score 2 · Answer 1 · edited Apr 13 '17 at 12:14

2

The vSwitch should have no special configuration.

Your port groups should contain all of the actual VLAN tags. On the switch side, is VLAN 1 actually defined anywhere?

See my example from: vSwitch configuration with 12 uplinks

There's nothing untagged at the vSwitch level because it makes more sense to associate port groups with a VLAN once you start trunking back to the physical switch. As @joeqwerty mentioned above, vSphere doesn't support what you're doing with the native VLANs.

Caution: Native VLAN ID on ESXi/ESX VST Mode is not supported. Do not assign a VLAN to a port group that is same as the native VLAN ID of the physical switch. Native VLAN packets are not tagged with the VLAN ID on the outgoing traffic toward the ESXi/ESX host. Therefore, if the ESXi/ESX host is set to VST mode, it drops the packets that are lacking a VLAN tag.

edited Apr 13 '17 at 12:14

Community

1

answered Jun 23 '14 at 14:17

ewwhite

194,921
91
434
799

Thanks for your comment. I only have one physical NIC in my standard switch. I am trying not to run all my traffic down VLAN 1. I want to be able to create separate VLANs for all my traffic - but can't seem to do so. At this stage, I'm thinking it's more of a switch problem as I have similar hosts setup in another environment with Netgear switches, and they work fine. – cpjones44 Jun 23 '14 at 22:29
See the note above. If you introduce a tagged-VLAN port group, *ALL* of your portgroups need to be tagged. – ewwhite Jun 23 '14 at 22:43
As in, ALL the port groups on the virtual switch? At the moment, I have two port groups on each host. First one is for management - not assigned a VLAN ID - works as expected (as I can access the host). Second one is VLAN 202 which i have VMs connected to. Communication between hosts does not work. – cpjones44 Jun 23 '14 at 22:51
Yes, please tag all portgroups. – ewwhite Jun 23 '14 at 22:58
Hi there... I've tagged all port groups on both virtual switches with ID 202 and I am still unable to ping between VMs on VLAN 202. – cpjones44 Jun 23 '14 at 23:05
Look at my included image above. Each port group has a different associated VLAN. You can't do what you're trying to do and still use VLAN 1 – ewwhite Jun 24 '14 at 00:33
I'm not trying to use VLAN 1. My setup is much simpler that yours. I have two port group. One for my vmk on VLAN 202, another for my VM also on 202. One uplink for the switch. Both hosts are configured the same and it still doesn't work. If I don't have any VLAN IDs on the port groups, everything works. – cpjones44 Jun 24 '14 at 02:06
@Chris Any better if you remove the trunk vlan access controls (the `switchport trunk allowed vlan 1,202` part)? – Shane Madden Jun 24 '14 at 04:38
Hi @Shane, when I run the command to remove that access control on a trunk port, I get the error "Every port has to be a member of at least one VLAN, ports will be added to default VLAN." (which makes sense). Even if I just have one VLAN (the default of 1) it still doesn't work with my VLAN 202 port groups. Would it be possible there's something wrong with a physical NIC? I thought configuring VLANs was suppose to be the easiest part!! – cpjones44 Jun 24 '14 at 08:41
@Chris Are you running `switchport trunk allowed vlan remove 1,202`? If this thing's a decent rip-off of Cisco's command syntax, you need to be running `no switchport trunk allowed vlan 1,202`. You can be pretty confident that the tag is being correctly applied because changing the tagging settings elicits a change in the switch's behavior -- I'm pretty sure that the switch, and not the ESXi host, is to blame. So, also check what changes in behavior you get from `switchport trunk native vlan 202` and `vlan dot1q tag native`. – Shane Madden Jun 24 '14 at 16:06
2

Hi @Shane. I've managed to solve the problem. I was able to determine this by testing VLAN trunks with a downstream switch I connected. Everything worked fine first go. The issue with ESXi wasn't with the switch, it was with the driver I was using for my NICs. I have these Emulex 10g cards, and there were two lots of offline bundles I could use from the Emulex website. I installed the second available bundle (be3) and now everything works as expected. Thanks for your help :) – cpjones44 Jun 25 '14 at 06:20
@Chris Aha, you were right! Good find. You should put the info on the card model and driver in an answer, so people in the future googling for the same problem will find the info! – Shane Madden Jun 25 '14 at 06:24

Trouble configuring standard VLANs on BNT G8264 and ESXi 5.5

1 Answers1