Windows 2008: WinSXS directory growing uncontrollably, blocking server

Question

I run a (remotely hosted) virtual Server with Windows 2008 Server for a client. Initially, it had 10 GB of space. During the course of a few weeks - during which nothing was done on the machine except normal work using a web-basedt icket system - , Windows began to fill up its infamous "winsxs" directory so much that in the end, the hard disk was full and we had to order another 5 GB. Now, three weeks later, these 5 GB have been consumed by winsxs as well, and again I can't work on the machine. Winsxs is now 8 GB big, the rest of the windows directory 5 GB.

I have found various sources on the web that describe the same problem. Apparently, Windows 2008 stores all language versions for all DLLs it downloads in the normal updating process. Just deleting stuff there is described as mortally dangerous as it contains vital components. I have not found any kind of tool or instructions to identify and remove those files that are no longer needed.

What can I do? Is this normal behaviour and if it is, how do other servers with equally limited space manage? Is there something I can turn off or on?

Of the pre-defined server roles, only "File services" (or whatever it's called in english, it's a swiss server) is activated. In addition, I have installed Apache, mySQL, and Subversion. Automatic updates are activated.

Edit: The problem persists.

Note: I am aware that the WinSXS directory consist mainly of symlinks and that users often panic looking at its size. Still, Out of 15 GB of space, I have 1.5 MB used by programs and data, and nothing left. I'm glad I can even access the damn machine. *I have already freed up 1 GB of data, which was filled by the windows Windows within 24 hours. It's like in a horror movie. What I have tried:

• Installing SP2 (which comes with compcln.exe) is not an option, as the disk space is not enough for even that.
• There is no vsp1clean.exe on the machine, probably because SP1 has already been merged into the system. In fact, there exists no file named *cln.exe anywhere.
• There are no shadow copies. Shadow copies are not active.
• As far as I can tell, there are no system restore points active.
• The only server role activated is "file server".
• The standard "cleanup" function (right-click on C: drive) is offering me a baffling 2 MB in trash contents and temporary internet files.
• Using one of the "cleanup winsxs" scripts around is not an option for me, they all look too shady. I can't find anything directly from Microsoft addressing this issue.

Answer 1

The WinSxS directory doesn't take up nearly the space reported by Explorer since it uses Hard Links to physical files, not actual files. Explorer just has issues reporting the size of hard links.

This article on Disk Space (referenced here http://aspoc.net/archives/2008/11/20/winsxs-disk-space-and-windows-7/) has a great explanation on the WinSxS directory.

As to your actual disk usage problem - You can try to run COMPCLN.EXE to see if you can clean up any old service pack and hot fix files, which should help quite a bit. I would also look at any logging directories to see if theres something else going on.

Answer 2

I was in the same boat as you. I had 5 brand new 2008 VMs and I was watching them steadily require more and more disk space over the last year. Let me illustrate with windrstat.

Here's a screen capture of a brand new 2008 server:

3.8 GB winsxs directory

Now here's a screen capture of a production 2008 server:

5.4GB winsxs directory

Finally a screen capture of a brand new 2008R2 server: 5.4GB winsxs directory

From what I can tell, there is no way to truncate the winsxs directory. WinSxS stands for Windows Side-by-Side. In their attempt to reduce the effects DLL hell (i.e. Dependencies) Microsoft decided to keep every version of every DLL ever installed on your system in the winsxs directory.

This means two things (I think):

1. While there are some hard links in winsxs, the only hard links should be that of the current DLLs. In other words, the current DLLs that are active in the system are hardlinked back to this winsxs folder.

2. All of the previous versions of the DLLs are still available, and backed up in the winsxs directory. As you can see in all the examples, if the winsxs directory really was "mostly hard links" there would be no way it could take up more than 50% of your current disk usage. Now even if it were true that the winsxs directory is "mostly hardlinks", 50% usage would mean your entire C: drive was hardlinked back into the winsxs directory.

I tried everything. Deleting old files, uninstalling patches, deleting the "$" patch directories. Everything.

Finally I ended up growing all the C: partitions on all my servers to 30GB each. This will only be a temporary fix, as the winsxs directory will continue to grow. Good news though, as the latest version of NTFS allows you to grow your partitions without reboot. How convenient.

WinSxS is not just an annoying directory, it's a new methodology for all Windows developers. WinSxS will not be going away anytime soon. Developers are using WinSxS and depending on it to be there until a new methodology of DLL archiving or dependency resolution becomes available.

On every new windows 2008/7/Vista box I feel that a 30GB C: is almost right. Enough room (for now) for patches, logs, and a few applications.

I can't wait until someone figures all this out and starts reactivating pre-patch DLLs with rogue-manifests, that become attack/exploit vectors.

Answer 3

15 GB won't cut it in my experience, especially if it's a 64 bit machine with some activity...

Extending the VPS disk and then the system partition should be easy without having to re-install anything, however expensive it is. Offering less than 60GB system disk for a modern Windows Server VPS doesn't seem thought through in the first place even if ~30GB could cut it for a while :7

Going for 32 bit install will remove a bit of overhead, and going for a Server Core will remove a lot. My 32 bit Server 2008 Core file server which runs svnserve, dfs, print serving and so forth and is updated regularly is around 5 GB... while my 64 bit Server 2008 Standard which services web applications (asp.net and php using IIS) takes up 30 GB right now. Both these metrics are excluding the data stores/partitions of course, only the Windows system, and both are just examples I have 64 bit Server 2008 system disks that takes up anything from 10 to 35 GB right now.

Every tool, upgrade or application you install will grow the winsxs directory and uninstalling stuff won't always have any effect on winsxs as the system may think those dependencies are used by other tools.

Answer 4

This is probably too late for you, and only based on Vista experience, but:

1. Don't believe those who say 'its mostly hard links, so the folder isn't as large as it appears' - this is true, but if you ran out of disc space, it was too large.
2. The only 'cure' ever suggested is to remove the applications that added the bloat to the folder - but there is no documented way that I know of to find the culprits.

3. My 'cure' after failing to find satisfactory answers was to brutally remove the largest (and unwanted) folders, eg:

   • 6 versions of 'Microsoft natural language search', at ~300kB each
   • Many oriental fonts I will never use (sometimes 6 vns of these, mostly identical)
   • etc, etc.

Assuming you start with a good backup (image), try it and see!

Answer 5

If it is possible you could try moving WinSXS to another partition as a workaround. To my knowledge there isn't any way to get rid of the contents of WinSXS. This cute little thingy is in the heart of Vista/Win2008 operation, so... I haven't heard anything better as moving it to another partition.

Here's a link to a blog describing how to do it: Click

I would suggest you do a full backup (even better: an image) of your entire C: partition before trying the procedure. Mind you that I tried this only on Vista, not on Win2008 Server.

Answer 6

If WinDirStat and the OS itself (just lookung under My Computer) is reporting the same amount of space used, its irrlevant if its miscounting the sizes as hard links. If the OS thinks you are out of space -- YOU ARE OUT OF SPACE.

I have this issue on a Storage Server 2008 and a DC (Server 2008). No other programs but Windows updates have been installed on the DC, both servers are consuming 15GB of space under WinSXS. The Storage Server was preinstalled by HP with 40GB on C:, so I'm also suffering from this issue. Down to 5GB free space.

This was fixed in Windows 8/Server 2013 by using DISM to cleanup superseded updates (dism.exe /online /cleanup-image /startcomponentcleanup) so clearly this is an issue they acknowledged, unfortunately with no fix other than to upgrade for those stuck with 2008.

Answer 7

In Windows Server 2008 R2 you can use DISM to clean up WinSxS, of course you won't be able to roll back updates after you do this. Here's an explanation: http://www.sqlskills.com/blogs/glenn/how-to-reclaim-disk-space-in-your-windowswinsxs-directory-on-windows-server-2008-r2-sp1/

In summary, you need to run the following command as an administrator:

DISM.exe /online /Cleanup-Image /spsuperseded

I know this might not be a relevant solution for you since you have Windows Server 2008 (not R2 apparently), but I found this question when looking for a solution for R2 so it might be useful for some people.

Answer 8

If the main culprit is WindowsUpdate, then I would say your only alternative to buying more space or performing system file surgery is to limit the growth caused by Microsoft's trigger-happy updates.

Since you primarily use a non-Microsoft application platform, you can probably do without 90-95% of the "imporant" updates being pushed down.

Thus, what you can do is disable automatic updates and then be VERY selective about which "important" updates to apply. Only apply those that patch vulnerabilities that DIRECTLY compromise your security plan. Most "important" updates probably don't apply to your security scenario, and if they do you can probably modify your security plan to do without them (e.g. don't use IE on the server, use alternate file services, tighten network security).

Answer 9

http://tech.xlab.si/2009/07/move-winsxs-folder/

Found this article but never tried it.. looks like there is a way to move over the winsxs folder . if nyone can try this please go thrugh and let me know

Answer 10

The Microsoft given System Requirements for Server 2008 are "Disk space: Minimum 32Gb or greater".

http://www.microsoft.com/windowsserver2008/en/us/system-requirements.aspx

Why do you think it can run comfortably in half that space?

Answer 11

A couple of suggestions:

• Where is your swap file?

• Setup a test machine with the same software to see if you can replicate the problem. This is definitely not normal, and the only thing I can think of is that something on that box is constantly modifying a DLL (or something like that) and so Windows is constantly making backups of it.

15GB is very much on the low side for a Windows 2008 install -- my standard C: drive for Windows 2008 builds is 60GB.

Answer 12

One option to use - if it's just a cheap old VPS that you use for simple things, then it's worth enabling NTFS compression on the disk or specific folders:

This might squeeze some space out of the drive. The articles says its for XP but it's the same steps on Win7/2008:

• To compress a file or folder on an NTFS drive

• How To Use File Compression

  1. Right-click the volume you want to compress, and then click Properties.
  2. On the General tab, click to select the Compress Volume to save disk space check box, and then click OK.
  3. In Confirm Attribute Changes, click the option you want.

Answer 13

I've run into the same problem. Surely WinSxs is partly to blame... But I think the problem is partially due to tools like WinDirStat counting the sizes of files and junctions to files the same way. A junction is only a pointer to the file, not a duplicate off all the blocks that make up the file.

I have found that VSS is a larger culprit in this problem. Use the following commands to see if VSS is consuming too much of your disk space, reclaim that space, or set a smaller limit:

vssadmin list shadowstorage

vssadmin Delete ShadowStorage /For=C: /On=C:

vssadmin Resize ShadowStorage /For=C: /On=C: /MaxSize=20%