1

I'm using an Internet plan of 100GB bandwidth monthly from my ISP, and I made a simple web crawler for fun and run it on my personal computer 24/7. The crawler is consuming all of the bandwidth, and I configured it to skip downloading media files (images, videos, sound files), so I can make as many requests and collect as much HTML pages as possible.

I don't know how ISPs work. Do they record the number of requests? or just bandwidth?

Can my ISP detect that? or are they going to see me as a normal user who just makes too many requests above the average?

schroeder
  • 123,438
  • 55
  • 284
  • 319
AccountantM
  • 296
  • 1
  • 6
  • 2
    Your plan, as you stated it, is "100GB bandwidth monthly". That's measured in data volume, not in the number of requests. So I think you answered your own question on that one. – schroeder Sep 03 '19 at 08:58
  • 1
    Whether your particular ISP will, or can, classify the activity as automated or human is entirely up to that particular ISP. – schroeder Sep 03 '19 at 08:59
  • @schroeder Aha, so they only care about the bandwidth, and I'm fine with what I'm doing, thank you. – AccountantM Sep 03 '19 at 09:00
  • 2
    @AccountantM I wouldn't jump to the conclusion that they only care about bandwidth. It's absolutely up to the ISP. I've heard stories about ISPs throttling users for all sorts of reasons (web crawlers, torrents, hosting a web server, etc.). – Omegastick Sep 03 '19 at 09:42

0 Answers0