8

My organization has a large deployment of Symantec Endpoint Protection (SEP) (~20k clients) with a single SEPM instance running in an ESX VM. We do have many remote clients designated as Group Update Providers (GUPs) where possible.

What our sysadmins have reported is that the SEP software does not have any native way to throttle its use of network bandwidth. It needs to send a 'full' definition update to every client, some hundreds of Mbyte in size. We found that the SEPM will practically accept 1000s of client check-in requests, and will send all clients updates at the maximum data rate possible.

We need some way of reducing the amount of bandwidth used by the SEPM to update clients natively, so that there is headroom on its network connection for management traffic (remote in, check the SEP console, etc).

So far, to mitigate flooding the entire network, we have throttled the SEPM traffic externally (at VM and switching level), which works to prevent congestion at the head-end network. However, that won't guarantee any bandwidth for management traffic.

We'd like to implement some change at the OS or application level to throttle the traffic without needing some heavyweight QoS deployment at 100s of offices. Ideally we would like to be able to throttle the amount of traffic used per client for SEP updates.

Please let me know if you have any ideas how to achieve this goal.

hmallett
  • 2,425
  • 14
  • 26
trp
  • 135
  • 1
  • 7
  • Found this article, will be trying some of the ideas therein: http://www.symantec.com/connect/articles/tips-installing-sep-low-bandwidth-environment – trp Sep 22 '12 at 15:48
  • Hints in the article are no good with the new version of SEP, since they switch webservers to Apache. – trp Sep 26 '12 at 21:34
  • I work with SEPM and Windows and QoS is the only way to do what you ask. However, the SEP's should *not* be asking for full def download each time. Unless the SEPM and its GUP's are only set to hold <5 defs. Ensure they hold at least 10, since there's default 4 def updates/day – Lizz Jan 05 '13 at 06:13
  • It looks like potentially configuring some Apache mods may be the right answer, but the wheels of progress turn very slowly in my organization. – trp Jan 05 '13 at 16:50
  • Have you done any baselining, pulled data off your switchgear so you know what percentage of your total bandwidth SEM is using? – JamesBarnett Jan 06 '13 at 18:22
  • Did you randomize the update distribution process to spread it across several hours? – Martino Dino Jan 06 '13 at 20:58
  • @JamesBarnett, yes, we do have some understanding of this. SEPM will practically consume as much as possible if it has enough work to do and the downstream networks are fast enough. – trp Jan 07 '13 at 21:35
  • @MartinoDino, yes, we have generally got it spread out wherever possible. This does not solve the problem of e.g. a site with many hosts hosts loses its wan circuit for 12 hours, then the wan circuit is restored and all hosts begin checking into the SEPM again and downloading their updates, which saturates the connection at the remote location. – trp Jan 07 '13 at 21:38
  • If you have configured Group Update Providers correctly, then you should not encounter the scenario where a site that has lost its WAN connection for some time suddenly floods the SEPM server with download requests when the connection comes back online. Only the GUP at the site should download the update from the SEPM server and the clients in the site should pull from the GUP. – August Jan 09 '13 at 20:27
  • Not true, any number of hosts including GUPs can request updates, and only once the GUPs have received and installed their updates (e.g. in parallel with 10s or 100s of other hosts) will the SEPM server redirect clients to download their updates from a GUP. – trp Jan 10 '13 at 01:34
  • 2
    http://www.symantec.com/business/support/index?page=content&id=TECH96419 - Set "Maximum time that clients try to download updates from a Group Update Provider before trying the default management server" to Never. Also you can throttle the bandwidth on the GUPs if they are still demanding too much with the setting "Maximum bandwidth allowed for Group Update Provider downloads from the management server" – August Jan 10 '13 at 12:40
  • 1
    Thanks, that helps a lot as well. It still doesn't cover the case where there is no GUP available. I want to be able to control the amount of bandwidth used by the SEPM per client regardless of its GUP status. (Could all hosts be marked as GUPs and throttled with the maximum-download option here?) – trp Jan 10 '13 at 17:03
  • If there are so many sites that managing the GUPs in each one is such a headache, then why don't you just configure the clients to download updates directly from Symantec instead of the SEPM server? If you are still hellbent on throttling the bandwidth at the client, then maybe you can use Miles Erickson's suggestion below on the client exe that is responsible for getting the updates with LiveUpdate. I just watched with procexp and I think it is ccSvcHst.exe, but not certain. SEPM appears not to have a built-in throttler for the clients without using GUP. – August Jan 10 '13 at 20:34
  • @August I'm willing to award the bounty to your response re: GUP configuration (max time / max bandwidth settings) since this most closely answers the question in my opinion. Go ahead and promote it to an actual answer if you like. We worked with Symantec yesterday and they have ultimately suggested that we do a broad deployment of internal LiveUpdate servers for our particular environment, which will allow finer-tuned bandwidth and update distribution management in general. – trp Jan 11 '13 at 17:34

2 Answers2

5

You can throttle the SEP Manager process's bandwidth using policy-based QoS features that are built into Windows Server 2008 and newer.

  1. Log into the server and open gpedit.msc.

  2. Navigate to Computer Configuration\Windows Settings\Policy-based QoS.

  3. Right-click on Policy-based QoS and click Create new policy...

  4. For Policy Name, enter SEPM Throttling.

  5. Uncheck Specify DSCP Value.

  6. Check Specify Outbound Throttle Rate.

  7. Enter the desired maximum rate in megabits per second, and select Mbps (instead of Kbps) from the dropdown list.

  8. Click Next.

  9. Click Only appliations with this executable name.

  10. Enter the SEPM web server process name (probably httpd.exe).

  11. Click Next twice, then click Finish.

Skyhawk
  • 14,149
  • 3
  • 52
  • 95
  • For whatever it's worth, SEP tends to be more problematic than certain other enterprise antivirus products. SEP is insufficiently configurable, has excessive CPU/network/disk overhead, and can cause serious problems on busy servers with high CPU utilization (especially terminal servers), even with sane exclusions configured. I prefer to recommend ESET to my clients. – Skyhawk Jan 07 '13 at 03:36
  • thanks for the tip! This still doesn't solve the problem of throttling the bandwidth /per client/, only the total bandwidth used by the server (which we are getting through throttling at the VM). We need to avoid both choking the server's network and also choking the clients' networks. – trp Jan 07 '13 at 21:45
2

I think your best solution is to configure your remote clients to either:

  1. only pull updates from the GUPs in each remote site, throttling the bandwidth on the GUP using the LiveUpdate policy settings

    or

  2. just let your remote clients go directly to Symantec for updates and not to your SEPM server

This article would help you configure it the first way- symantec.com/business/support/…:

  • Set "Maximum time that clients try to download updates from a Group Update Provider before trying the default management server" to Never.
  • Also you can throttle the bandwidth on the GUPs if they are still demanding too much with the setting "Maximum bandwidth allowed for Group Update Provider downloads from the management server"

If you have so many remote sites that managing the GUPs in each site is a headache, then I'd go for the second option.

Unfortunately, Symantec doesn't appear to have a built-in way to throttle bandwidth on the remote clients directly, only on the GUP clients.

Community
  • 1
August
  • 3,114
  • 15
  • 17