2

I am using Windows Server 2008, running in a Hyper-V VM. The system performs very badly. We're pretty sure it's due to insufficient memory. I've looked at How to find out if my Windows Server 2003 needs more memory?, and I'm pretty sure from seeing > 300 pages input per second, that memory is the issue.

However, I'm asking a broader question. I'd like to focus the attention of Management on the amount of time that's being wasted while waiting for these page faults. Is there a way to determine how much time is being spent waiting on paging or other resources? I'm especially interested in time spent by interactive users, but a number that includes non-interactive use would also be helpful.

To clarify, so this seems less like whining:

This system is so slow that it has taken literally minutes, while one notepad.exe window was active, to activate a second notepad.exe window that was already open and displaying. That's just activation. The performance isn't slightly bad. I think it amounts to tens of man-hours per day wasted.

I'm looking for a way to make it clear to Management how much time is being wasted. This is happening at a time when we don't actually have ten man-hours to waste.

Sorry I didn't provide more detail.

This server is being used in SharePoint (WSS 3.0) development. It is running IIS 7 set up with at least one Application Pool per developer. Each developer has one or more web applications, each set up in an AppPool assigned to that developers credentials. We're running Visual Studio 2008 SP1 and SQL Server 2008. The SQL Server data is on a separate virtual disk from the OS.

I've seen up to 8 developers on the system at once. The server is configured with 2GB of RAM, and more RAM is not trivial at this time, due to limitations of the host computer. I hope that this will get corrected if I can present enough reason to correct it, in the form of: this is how much time was wasted.

EDIT

Thanks for the answers and comments. I agree with the solution - get the SharePoint load onto a 4-8GB server, and move SQL Server onto a second one, with 2-4GB.

But my question is more like: are there any performance counters or tools that can tell me the amount of time spent waiting for page reads and writes? Any that can tell me amount of time spent waiting on queued disk I/O?

It's possible to get performance counters like "Pages input per second", but it's harder to say what values of that counter is "too much". It would be better for my purpose if there were "counters" that can say how much time is being spent because of the pages input per second.

Community
  • 1
John Saunders
  • 425
  • 7
  • 22
  • Wouldn't that depend on the hardware? To what end are you trying to get the information, like to figure out if all these page faults at the end of the day meant you lost five minutes of your life waiting because your server was delayed with page faults? – Bart Silverstrim Oct 27 '09 at 20:12
  • No, because at times it's taking **minutes** to switch between _notepad.exe_ windows. – John Saunders Oct 27 '09 at 20:24
  • I think it would depend on your hardware as to how long you're losing time. If you wanted you could probably just time things between click and ability to use an application and compare it to a more responsive workstation. It still all comes down to usage patterns, though. A "slow" system still isn't that big a deal if there isn't someone tapping their foot while they're waiting, unless I'm misreading your question. – Bart Silverstrim Oct 27 '09 at 20:28
  • The system is so slow at these times that everyone is tapping their foot. I'd like to quantify the foot-tapping, if possible. – John Saunders Oct 27 '09 at 20:29
  • "everyone is tapping their foot"? How many people do you have on one server at any given time. You haven't indicated this is a Terminal Server, or whatever Microsoft has decided to call it these days, so I would expect no more than one admin on that machine at any given time. Perhaps you can supply some details, such as the machine specs, what it's used for and how it's being used. – John Gardeniers Oct 27 '09 at 20:55
  • 3
    There's a reason why you plan and size things before deploying them. – duffbeer703 Oct 27 '09 at 22:04
  • Very true, but this one is already deployed (not by me), so the question is: what now, and how do I convince Management that this needs to be corrected. Convincing them with numbers would be easier than convincing them without numbers. – John Saunders Oct 27 '09 at 22:08
  • 1
    Convince them with Microsoft minimum supported system requirements for each of those services... SQL and Sharepoint and IIS Application pools... I feel sorry for the machine already ^^ – Oskar Duveborn Oct 27 '09 at 22:26
  • If you're trying to convince management via numbers, simply take a stopwatch and record all the developers's time in opening VS/ solutions/compiling/closing down/searching/etc. Do the math and come up with a nice *big* average. For the sake of example each dev loses: 28min/day. 28 x 8 developers = 244min/day x 5/daysweek = 1220 min. That's 20 hours amongst 8 devs. Pitch the idea by saying a new server you could *theoretically* close that gap by 60% down to 732 min = 12.2 hours you could make a case. Who cares how accurate *your* numbers are as long as you get the *needed* resources. – osij2is Oct 27 '09 at 22:51

3 Answers3

8

This server is being used in SharePoint (WSS 3.0) development. It is running IIS 7 set up with at least one Application Pool per developer. Each developer has one or more web applications, each set up in an AppPool assigned to that developers credentials. We're running Visual Studio 2008 SP1 and SQL Server 2008. The SQL Server data is on a separate virtual disk from the OS.

I've seen up to 8 developers on the system at once. The server is configured with 2GB of RAM, and more RAM is not trivial at this time, due to limitations of the host computer. I hope that this will get corrected if I can present enough reason to correct it, in the form of: this is how much time was wasted.

Without even looking at the hardware requirements of SharePoint 2007, having up to 8 developers with their own app pools. I assume that SQL Server is one another VM, but if it's one the same Win2k8 machine, there's no question of what the issue is.

AppPools for VisualStudio development (versions from 2005-2008) can easily grow to 150MB-250MB per AppPool pending numerous factors. 250mb x 8devs = 2gb used. Let's not forget about the memory the OS needs by itself and possibly SQL Server. In a nutshell: You simply don't have enough RAM. Load up on the RAM, as much as you can get. I wouldn't be surprised if it solved a majority of the "slowness" of the server at least from the background perspective and possibly the foreground.

FYI: Microsoft recommends 4GB minimum for SharePoint 2007 Application Servers (link) but in reality, anything Microsoft eats up resources as much as possible. Now in the same URL, they mention a 2GB minimum for standalone SharePoint servers but if developers are directly working on that server with their own AppPool, it's clear that IIS is eating up all available memory for the AppPools/Sites. Don't go with the minimum RAM recommended. Try to double, triple, quadruple RAM requirements if possible (budget permitting).

Edit: You mention that the SQL Server data is on a separate virtual disk, but is SQL Server installed on the same SharePoint server? How much free storage is available on the SharePoint server as well? These additional factors can easily consume server resources and should not be neglected.

Edited again: Since you mentioned (and I failed to read) that "RAM is not trivial at this time, due to limitations of the host computer", there's only one real solution: get a new host computer. 2GB is not enough to run SharePoint/SQL/IIS/whatever period. Sorry to say, but IMHO, a machine running this should have a minimum of 8GB RAM.

Edited after OP edit:

But my question is more like: are there any performance counters or tools that can tell me the amount of time spent waiting for page reads and writes? Any that can tell me amount of time spent waiting on queued disk I/O?

I haven't run into your exact situation, but there's a good article (link) from Microsoft TechNet on the basics of the monitor metrics. I'm not sure if time spent waiting on queued disk I/O is the best way to get what you want (I'm assuming management backing).

It's possible to get performance counters like "Pages input per second", but it's harder to say what values of that counter is "too much". It would be better for my purpose if there were "counters" that can say how much time is being spent because of the pages input per second.

An article from Windows Networking (link) on Server 2003 explains those counters better than I could. FTA:

The Memory\Pages/sec counter indicates the number of paging operations to disk during the measuring interval, and this is the primary counter to watch for indication of possible insufficient RAM to meet your server's needs. A good idea here is to configure a perfmon alert that triggers when the number of pages per second exceeds 50 per paging disk on your system. Another key counter to watch here is Memory\Available Bytes, and if this counter is greater than 10% of the actual RAM in your machine then you probably have more than enough RAM and don't need to worry.

You should do two things with the Memory\Available Bytes counter: create a performance log for this counter and monitor it regularly to see if any downward trend develops, and set an alert to trigger if it drops below 2% of the installed RAM. If a downward trend does develop, you can monitor Process(instance)\Working Set for each process instance to determine which process is consuming larger and larger amounts of RAM. Process(instance)\Working Set measures the size of the working set for each process, which indicates the number of allocated pages the process can address without generating a page fault. A related counter is Memory\Cache Bytes, which measures the working set for the system i.e. the number of allocated pages kernel threads can address without generating a page fault.

Finally, another corroborating indicator of insufficient RAM is Memory\Transition Faults/sec, which measures how often recently trimmed page on the standby list are re-referenced. If this counter slowly starts to rise over time then it could also indicating you're reaching a point where you no longer have enough RAM for your server to function well.

So I'd say this explanation really addresses your metrics and understanding of what the metrics actually mean to you. Performance Monitor is a bit tricky especially if you don't fully understand what the counters mean in the grand scheme of things. I usually find myself reading about counters as it's easy to forget their "real world" meaning.

Community
  • 1
osij2is
  • 3,875
  • 2
  • 23
  • 31
0

It sounds like you need to start using perfmon. You'll want to set counters for things like memory (available MBytes (my guess is it will be near or at 0), % committed Bytes in use) and perhaps some disk monitors. Your hard disk is most likely grinding away non-stop growing your paging file to it's maximum size. However, I'm only offering these suggestions so you can view the performance yourself. The answer to the problem was in the first answer: memory. If your memory limitations for the server in question is 2GB then you need to purchase or throw together an extra server. I'm not sure if you have the licenses for another but 2GB is not enough ram to run SharePoint let alone the plethora of other applications / environments you have running. That being said, there's nothing like a good challenge. :) Good luck!

JohnyD
  • 1,614
  • 2
  • 20
  • 29
-1

A server isn't optmized for foreground tasks, so switching between notepad windows on it won't be a valid performance metric. You need to fire up Process Explorer or Task Manager and start looking at some stats in there, then maybe do some performance counters and see what they give you back. Also examine the other tasks running on the server (you don't say what it's roles are so it's impossible to be more specific).

Maximus Minimus
  • 8,937
  • 1
  • 22
  • 36