0

After Xencenter-toolstack restart i lost connection from xencenter

I try to restart using ssh - putty following message is showing and i lost the browser access also ............

All xe command is not working and postponed on ssh ....

@xs01p04 log]# xe-toolstack-restart Stopping xapi: [ OK ]
Stopping the v6 licensing daemon: [ OK ]
Stopping the memory ballooning daemon: [ OK ]
Stopping perfmon: [FAILED]
Stopping the xenopsd daemon: [ OK ]
Stopping the XCP RRDD daemon: [ OK ]
Stopping the XCP networking daemon: [ OK ]
Stopping the fork/exec daemon: [ OK ]
Starting the fork/exec daemon: [ OK ]
Starting the XCP networking daemon: . [ OK ]
Starting the XCP RRDD daemon: [ OK ]
Starting the xenopsd daemon: [ OK ]
Starting perfmon: [ OK ]
Starting the memory ballooning daemon: [ OK ]
Starting the v6 licensing daemon: [ OK ]
Starting xapi: OK [ OK ]

Thank Advance ...

Update #1

@yagmoth555

1) i try to migrate one vm to another storage location that time some unexpected error occurred and yello state is appear Then i create new vm from snapshot

Dead vm still there

After one week 2) yesterday i try to reboot a vm and its again yellow state ( its not previous one)

Then i try to restart xentoolstack from xencenter after few minute i lost xencenter connection I cannot connect through xencenter

But https:ip is appearing

Then i login through ssh and restart toollstack through command line After that i lost http acess also

Nom XE command is hanged up but Vm are still working

Update #2 @yagmoth555 Thanks advance

tail -n 15 /var/log/messages

Oct 16 20:08:30 xs01p04 ovsdb-server: 01993|reconnect|INFO|ssl:10.101.2.52:6632: waiting 8 seconds before reconnect

Oct 16 20:08:33 xs01p04 fe: 26045 (/usr/bin/ovs-appctl bond/show bond2) exitted with code 2

Oct 16 20:08:33 xs01p04 xcp-networkd: [error|xs01p04|1|monitor_thread|network_utils] Call '/usr/bin/ovs-appctl bond/show bond2' exited with code 2 [stdout = ''; stderr = 'no such bond ovs-appctl: ovs-vswitchd: server returned reply code 501 ']

Oct 16 20:08:38 xs01p04 ovsdb-server: 01994|reconnect|INFO|ssl:10.101.2.52:6632: connecting...

Oct 16 20:08:38 xs01p04 ovsdb-server: 01995|reconnect|WARN|ssl:10.101.2.52:6632: connection attempt failed (No route to host)

Oct 16 20:08:38 xs01p04 ovsdb-server: 01996|reconnect|INFO|ssl:10.101.2.52:6632: waiting 8 seconds before reconnect

Oct 16 20:08:38 xs01p04 fe: 26054 (/usr/bin/ovs-appctl bond/show bond2) exitted with code 2 Oct 16 20:08:38 xs01p04 xcp-networkd: [error|xs01p04|1|monitor_thread|network_utils] Call '/usr/bin/ovs-appctl bond/show bond2' exited with code 2 [stdout = ''; stderr = 'no such bond ovs-appctl: ovs-vswitchd: server returned reply code 501 ']

Oct 16 20:08:43 xs01p04 fe: 26063 (/usr/bin/ovs-appctl bond/show bond2) exitted with code 2

Oct 16 20:08:43 xs01p04 xcp-networkd: [error|xs01p04|1|monitor_thread|network_utils] Call '/usr/bin/ovs-appctl bond/show bond2' exited with code 2 [stdout = ''; stderr = 'no such bond ovs-appctl: ovs-vswitchd: server returned reply code 501 ']

Oct 16 20:08:46 xs01p04 ovsdb-server: 01997|reconnect|INFO|ssl:10.101.2.52:6632: connecting...

Oct 16 20:08:46 xs01p04 ovsdb-server: 01998|reconnect|WARN|ssl:10.101.2.52:6632: connection attempt failed (No route to host)

Oct 16 20:08:46 xs01p04 ovsdb-server: 01999|reconnect|INFO|ssl:10.101.2.52:6632: waiting 8 seconds before reconnect

Oct 16 20:08:48 xs01p04 fe: 26081 (/usr/bin/ovs-appctl bond/show bond2) exitted with code 2

Oct 16 20:08:48 xs01p04 xcp-networkd: [error|xs01p04|1|monitor_thread|network_utils] Call '/usr/bin/ovs-appctl bond/show bond2' exited with code 2 [stdout = ''; stderr = 'no such bond ovs-appctl: ovs-vswitchd: server returned reply code 501 ']

Update #3 @yagmoth555 Thanks advance
xe-toolstack-restart Stopping xapi: [ OK ]

Stopping the v6 licensing daemon: [ OK ]

Stopping the memory ballooning daemon: [ OK ]

Stopping perfmon: [FAILED]

Stopping the xenopsd daemon: [ OK ]

Stopping the XCP RRDD daemon: [ OK ]

Stopping the XCP networking daemon: [ OK ]

Stopping the fork/exec daemon: [ OK ]

Starting the fork/exec daemon: [ OK ]

Starting the XCP networking daemon: . [ OK ]

Starting the XCP RRDD daemon: [ OK ]

Starting the xenopsd daemon: [ OK ]

Starting perfmon: [ OK ]

Starting the memory ballooning daemon: [ OK ]

Starting the v6 licensing daemon: [ OK ]

Starting xapi: OK [ OK ]

service xapissl restart

which: no stunnel4 in (/sbin:/usr/sbin:/bin:/usr/bin) Stopping xapi SSL: Starting xapi SSL:

Joseph
  • 61
  • 3
  • 8
  • Did you closed both VM, and simply rebooted the server first ? – yagmoth555 Oct 16 '15 at 13:03
  • @ yagmoth555 , I rebooted sever still XE-command sill frorzen and only one vm is automatically up and ............................. I can see only one vm by domain_lis command ....................... xencenter is not connecting – Joseph Oct 16 '15 at 14:42
  • start by doing a df -m via the console/ssh on the host, is the disk is full ? – yagmoth555 Oct 16 '15 at 15:56
  • disk is not full only 79% ................... – Joseph Oct 16 '15 at 16:08
  • Your state.db is probably corrupt then. cd /var/xapi, check the file timestamp, if old, it's corrupt. if so, mv state.db state.db_bak, after service xapi restart – yagmoth555 Oct 16 '15 at 16:34
  • I did but its same ... after xapapi restart, state.db not automatically generating .... ... – Joseph Oct 16 '15 at 16:49
  • just seen your new log, never seen that error. I would do xe-switch-network-backend bridge to disable openvswitch, but it's a guess at this point. – yagmoth555 Oct 16 '15 at 17:40
  • after all that, if your data is on a different partition I would re-install xen over your install without touching the vm storage location, and re-join the member to the pool. be sure to have backup of the vm – yagmoth555 Oct 16 '15 at 18:33

2 Answers2

1

xe toolstack-restart will always cut your xencenter connection. (but the vm continue to run correctly on the host)

Please log in normal console (iDRAC, iLO, etc..), as that mean your XAPI can't restart and your server state is unknown.

I would check with dmesg or in /var/log for xensource.log for any errors.

yagmoth555
  • 16,300
  • 4
  • 26
  • 48
  • Brother , Thanks for your replay .. ,,,,, I can acess server through ssh and i can access all vm ........ can you explain little bit more i am not familiar in xenserver .... – Joseph Oct 15 '15 at 20:34
  • @JobishJose The toolstack-restart cut the comminucation with the console (but the vm continue to work), but usually you can reconnect back some minute later with xencenter when the service restart. I would check dmesg to check if a error occurred somewhere. (or xensource.log in /var/log) – yagmoth555 Oct 15 '15 at 20:45
0

Finally i fixed that issue We have two servers masters & slave .. some issue slave not responding properly ... that's why all XEcommand and all frozen

I shout down both servers and restart the master first and then slave finally

fixed all issue thanks

Joseph
  • 61
  • 3
  • 8