OpenIndiana (illumos): vmxnet3 interface lost on reboot

Question

I want my VMware vmxnet3 interface to be brought up with DHCP on boot. I can manually configure the NIC with:

# ifconfig vmxnet3s0 plumb
# ipadm create-addr -T dhcp vmxnet3s0/v4dhcp

But after creating /etc/dhcp.vmxnet3s0 and rebooting, the interface is down and the logs show:

Aug 13 09:34:15 neumann vmxnet3s: [ID 654879 kern.notice] vmxnet3s:0: getcapab(0x200000) -> no
Aug 13 09:34:15 neumann vmxnet3s: [ID 715698 kern.notice] vmxnet3s:0: stop()
Aug 13 09:34:17 neumann vmxnet3s: [ID 654879 kern.notice] vmxnet3s:0: getcapab(0x200000) -> no
Aug 13 09:34:17 neumann vmxnet3s: [ID 920500 kern.notice] vmxnet3s:0: start()
Aug 13 09:34:17 neumann vmxnet3s: [ID 778983 kern.notice] vmxnet3s:0: getprop(TxRingSize) -> 256
Aug 13 09:34:17 neumann vmxnet3s: [ID 778983 kern.notice] vmxnet3s:0: getprop(RxRingSize) -> 256
Aug 13 09:34:17 neumann vmxnet3s: [ID 778983 kern.notice] vmxnet3s:0: getprop(RxBufPoolLimit) -> 512
Aug 13 09:34:17 neumann nwamd[491]: [ID 605049 daemon.error] 1: nwamd_set_unset_link_properties: dladm_set_linkprop failed: operation not supported
Aug 13 09:34:17 neumann vmxnet3s: [ID 654879 kern.notice] vmxnet3s:0: getcapab(0x20000) -> no
Aug 13 09:34:17 neumann nwamd[491]: [ID 751932 daemon.error] 1: nwamd_down_interface: ipadm_delete_addr failed on vmxnet3s0: Object not found
Aug 13 09:34:17 neumann nwamd[491]: [ID 819019 daemon.error] 1: nwamd_plumb_unplumb_interface: plumb IPv4 failed for vmxnet3s0: Operation not supported on disabled object
Aug 13 09:34:17 neumann nwamd[491]: [ID 160156 daemon.error] 1: nwamd_plumb_unplumb_interface: plumb IPv6 failed for vmxnet3s0: Operation not supported on disabled object
Aug 13 09:34:17 neumann nwamd[491]: [ID 771489 daemon.error] 1: add_ip_address: ipadm_create_addr failed on vmxnet3s0: Operation not supported on disabled object
Aug 13 09:34:17 neumann nwamd[491]: [ID 405346 daemon.error] 9: start_dhcp: ipadm_create_addr failed for vmxnet3s0: Operation not supported on disabled object

I then tried disabling network/physical:nwam in favour of network/physical:default. This works, the interface is brought up but physical:default fails and my network services (e.g. NFS) refuse to start.

# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
    inet 127.0.0.1 netmask ff000000 
vmxnet3s0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:1: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:2: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:3: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:4: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:5: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:6: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:7: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
vmxnet3s0:8: flags=1004842<BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 9000 index 2
    inet 192.168.178.248 netmask ffffff00 broadcast 192.168.178.255
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
    inet6 ::1/128 
vmxnet3s0: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 9000 index 2
    inet6 ::/0

# cat /var/svc/log/network-physical\:default.log
[ Aug 16 09:46:39 Enabled. ]
[ Aug 16 09:46:41 Executing start method ("/lib/svc/method/net-physical"). ]
[ Aug 16 09:46:41 Timeout override by svc.startd.  Using infinite timeout. ]
starting DHCP on primary interface vmxnet3s0
ifconfig: vmxnet3s0: DHCP is already running
[ Aug 16 09:46:43 Method "start" exited with status 96. ]

NFS server not running:

# svcs -xv network/nfs/server
svc:/network/nfs/server:default (NFS server)
State: offline since August 16, 2012 09:46:40 AM UTC
Reason: Service svc:/network/physical:default
        is not running because a method failed.
   See: http://illumos.org/msg/SMF-8000-GE
  Path: svc:/network/nfs/server:default
          svc:/milestone/network:default
            svc:/network/physical:default
Reason: Service svc:/network/physical:nwam is disabled.
   See: http://illumos.org/msg/SMF-8000-GE
  Path: svc:/network/nfs/server:default
          svc:/milestone/network:default
            svc:/network/physical:nwam
Reason: Service svc:/network/nfs/nlockmgr:default is disabled.
   See: http://illumos.org/msg/SMF-8000-GE
  Path: svc:/network/nfs/server:default
          svc:/network/nfs/nlockmgr:default
   See: man -M /usr/share/man -s 1M nfsd
Impact: This service is not running.

I'm new to the world of Solaris, so any help solving would be much appreciated. Thanks!

Answer 1

Don't use ifconfig to plumb the interface use ipadm create-if instead. You've also managed to configure the adaptor 9 times, all of which are configured to use DHCP. You should run ipadm delete-addr on all 9 and perhaps even ipadm delete-if vmxnet3.

Once you've done that, you should also stop using vmxnet3 interfaces in your virtual machine configuration and use e1000 interfaces instead, I've had much better luck with those.

More info: ifconfig vs ipadm comparison and example of network setup using dladm and ipadm in OmniOS (also applies to other Illumos deritives like OpenIndiana).