23

I'm trying to remotely reboot a hung 2003 server.

The server is pingable but I cannot remote into it.

psshutdown does not work
shutdown -i does not work

What can I do now that the server is back up to make sure one of these solutions will work in the future?

warren
  • 17,829
  • 23
  • 82
  • 134
MathewC
  • 6,877
  • 9
  • 38
  • 53
  • 13
    :) couldn't resist.... http://thedailywtf.com/Articles/ITAPPMONROBOT.aspx – woliveirajr Jan 23 '12 at 15:47
  • 1
    If this server has IPMI capabilities go and reset it in person if you can (of get some remote hands and eyes) so that someone can set the IPMI details for future remote access. Or have it connected to a managed PDU or similar devices, so you can hard cycle that servers power port. – jwbensley Jan 23 '12 at 20:05
  • 1
    @woliveirajr - _Years later, and long after Erik had left, the faulty server was taken offline and replaced with a new one..._ Nothing's so permanent as a temporary fix. The world's full of them! – Fiasco Labs Jan 24 '12 at 00:43
  • 1
    @woliveirajr - To this day in that server room people swear they can still hear the tuhumping of an opening/closing cd tray. The ghost of ITCUPHOLDERROBOT. - Thanks for sharing this. – stefgosselin Jan 25 '12 at 18:59

3 Answers3

31
  • Log into your KVM unit
  • Trigger a hard reset.

Servers off site should always have an IPMI daughterboard with KVM functionality. This allows you to remote control the server including turning it on or off. Servers without that are not servers.

Bryan
  • 7,538
  • 15
  • 68
  • 92
TomTom
  • 50,857
  • 7
  • 52
  • 134
  • 16
    +1 for Servers without that are not servers! – Khaled Jan 23 '12 at 14:39
  • I don't have my hands on the purse strings, and the system is still up. So I'm looking for a way to soft cycle it. – MathewC Jan 23 '12 at 15:03
  • 5
    Thee is none. Simple. Server needs reboot buttnio pressed = no soft cycle possible. This is why tehre is special tech that is - when bought wihth a server - very cheap. Someone on your end botchered that, now live with it. – TomTom Jan 23 '12 at 17:08
  • 2
    In otherwords a relay that closes momentarily when you ping it with a code. Looks like a job for an Adafruit Arudino with built-in Ethernet, a TIP-120 transistor, anti-kick diode and a relay. Wire it across the reset switch and give it a thump when the server dies. Then you can have a "Real Server". Not so botched tech for a more liveable world! – Fiasco Labs Jan 24 '12 at 00:48
  • Pros work with pro equipment, Fiasco. What about updating your BIOS remotely, or reinstalling the OS? A properKVM system allows you to attach a remote DVD image to the server without the OS getting involved. I rather have the ability to diagnose and fix remote servers without going there - and guesss what, it works nicely in cases of "no boot record found" also. – TomTom Jan 24 '12 at 04:49
17

You can try shutdown /m \\computer /r /t 0 to try and force an immediate reboot and force-close any open programs.

If that doesn't work, you're probably going to have to power cycle it. If you have a DRAC/iLO or similar LOM card, now would be the time to take advantage of it.

MDMarra
  • 100,183
  • 32
  • 195
  • 326
  • you'd also need the /f switch to force close open programs – August Jan 23 '12 at 14:44
  • 2
    @August `/f` is implied by `/t` here, so it would be superfluous. It wouldn't hurt anything if you use it out of habit, though. – MDMarra Jan 23 '12 at 15:18
  • @MDMarra, nice find! `The /f parameter is implied when a value greater than 0 is specified for the /t parameter.` That seems to imply you need to have something more than 0. – Nixphoe Jan 25 '12 at 22:06
  • 1
    @Nixphoe despite what's documented, I can assure you that `/f` is implied with `/t` even if you use 0. `/t 0` will cause an immediate forced shutdown of the target machine and force-close all open applications that don't exit in an immediate manner, – MDMarra Jan 25 '12 at 22:37
17

If Windows doesn't respond to the usual requests to reboot, there's not much else you can do at that point to ensure similar situations in the future to be prevented. A few times I've managed to reboot a hung Windows OS that didn't respond to shutdown or psshutdown by using SCCM client center or pskill to kill lsass.exe. Once that is killed, the kernel forcibly bring the system down for a reboot after 60 seconds. More often than not, however, lsass cannot be killed and that trick won't work.

What kind of hardware is this server running on? Can you install Server 2008R2 instead? While server 2003 is pretty stable, it's still a long way from the stability of 2008, if that's an option. Another option would be to virtualize the 2003 server if it has stability issues due to some odd program it needs to run or some other configuration that cannot be changed to resolve the stability issues. That way you can reset the virtual session if it hangs.

There's many options to resolve your situation, however none of them will involve something as simple as changing a few settings in your Windows Server 2003 install or installing a piece of software. Usually by the time a Windows OS is to the point you describe, it's hung to the point that nothing short of a power/reset button press will suffice.

Daniel Winks
  • 366
  • 1
  • 2
  • 1
    So far this is the only answer that actually answered the question. I know all about kvms and hard reboots. I did NOT know about the killing lsass.exe trick. Thanks! – MathewC Jan 27 '12 at 15:14
  • For my hung Server 2003 machine killing `csrss`did the trick. It initiated a BSOD event, which took a _very_ long time to memory dump, but afterwards the machine came back up more or less operational. At least operational enough that I could log in and begin troubleshooting the underlying cause. – matt wilkie Feb 29 '16 at 21:25
  • pskill lsass worked for me on a hung w2k8 r2 server that didn't repond to shutdown.exe or psshutdown.exe – JimSTAT Jul 02 '16 at 12:42
  • you can also try to use PSKILL to kill WINLOGON (it worked for me on a 2012 r2 server). I found the pskill advice here: https://community.spiceworks.com/how_to/1935-unstick-a-remote-reboot – Max Jul 06 '20 at 09:45