0

So I'm working on a website uptime monitor that should check thousands of websites per minute by doing a simple http call to them and checking the received status code.

I've tested it using multiple processes of a node.js implementation to ensure that the bottleneck is not a programming or processing one.

Anyways I've hit a wall. The maximum number of sites I'm able to check is roughly 2000 per minute. This number doesn't change if I run 1 instance or 10 instances of the code.

If I run multiple processes, the throughput of each process reduces so that the total output is still 2000 sites checked per minute.

The actualy network bandwidth doesn't seem to be too high. I mean 2000 websites would mean roughly 1 mb of data (and that's being generous). So it's probably an issue somewhere else and i'm trying to pinpoint that.

Tested on digital ocean.

Thanks in advance.

Ahmed-Anas
  • 101
  • 1

1 Answers1

0
  • Install monitoring.
    • Gather data about your system.
    • Analyse data.
    • Make decisions.
    • Make changes.
    • Monitor changes
    • Rinse and repeat as necessary.

If that fails, guess. My guess is Gaussian radiation interacting with the Van Allen belt.

Tested on digital ocean.

Try different provider that has a better network.

user9517
  • 114,104
  • 20
  • 206
  • 289