27

I sell a product to customers, and as part of this product I have a website where customers can upload data for processing. The data is of considerable size (gigabytes).

I am looking to buy extra bandwidth for my customers, and to make the arrangement with their ISPs myself so that the experience is seamless to the end users. Many of my customers are on university or corporate networks where they would be unable to make the arrangement themselves even if they wanted to. The extra bandwidth would apply only to connections to my website, not to the customer's other connections.

Basically I am looking for this sort of arrangement:

Diagram showing money flowing from customer to me and from me to ISP, data flowing from customer to ISP and from ISP to me

Is this sort of thing possible?

Edit: Now that the United States has ended net neutrality, is it possible?

Owen
  • 381
  • 3
  • 7
  • 8
    On the flip side of this, I think ISPs would love to be able to charge both the client and the server for connectivity -- charge the user for the line and 'bandwidth' and charge the server not to limit bandwidth to synthetically low throughput. – Jason Martin Jun 13 '17 at 23:47
  • 1
    Fundamentally, the problem is simply that your bandwidth is not enough to generate any economy of scale, so the amount they would have to charge you would be exorbitant. Think about a company that sells 100 widgets a week and suddenly you want to show up at random times and immediately get 1,000 widgets. Do they keep 1,000 widgets on hand all the time just for you? How do they pay for that when you don't need widgets? – David Schwartz Jun 13 '17 at 23:53
  • 7
    Anything is possible. I doubt it'll be easy. In fact, I think it'll fall into the too hard and expensive basket when you examine it in detail. You'd be best to start a dialog with your customers providers. But my gut feel is that it's not practical. – hookenz Jun 13 '17 at 23:54
  • Corp? Sure. My company peers with several of our customers, but I can tell you it is non trivial and a lot of legal/contractual involvement. They do this with us for reasons well beyond bandwidth. (availability, latency, security, extra privacy above and beyond https, traffic metrics) and these are very large companies. Residential ISP's? Not even possible in most cases, as they have asymetric limits you can't get around. I can't speak to school networks. – Aaron Jun 14 '17 at 00:23
  • 9
    See what Ryan Griggs said down below in an answer - use external hard drives and the USPS. As the old quote goes "never underestimate the bandwidth of a station wagon full of backup tapes". And in some cases, you just *can't* get more bandwidth - my home DSL has an absolute limit of 6mb down and 768k up, nothing aside from running new wires can do anything for my speed. I work at a college with a massive pipe, once a term we do a postal exchange with one of our service providers to move about half a terabyte of data back and forth... it works well, mail Monday receive Thursday. – ivanivan Jun 14 '17 at 01:28
  • 4
    @ivanivan More to the point. At today's price point and densities. Plus the ready availability of cheap labour (students), one can simple use postal service and SD cards. "Gigabytes of data" can now easily fit on something smaller than my fingernail... – Aron Jun 14 '17 at 02:50
  • 2
    Do you really mean bandwidth, i.e. speed, or do you mean data of a volume contract that they won't have to pay otherwise for? If it is about bandwidth, this is quite often a hardware limitation thing, if I wanted to upgrade my bandwidth, they would need to lay new/more fibers down to my premise. – PlasmaHH Jun 14 '17 at 08:54
  • This is not allowed by the law of Net neutrality, withs states that provides must threat every site with the rules – Ferrybig Jun 14 '17 at 09:11
  • 2
    Isn't this what Private Peering is for? Don't know many details about that, so no answer, but i.e. Hetzner has ongoing agreements with lots of ISPs and service companies like Youtube, big and small, where they presumably interconnect their networks and guarantee direct-line-like data transfer between their centers. Not sure, but this is usually done at interchanges where all partners own links (like DE-CIX, where e.g. Hetzner, Deutsche Telekom and Deutsches Forschungsnetz meet up, which would enable a peering for all german universities, a _lot_ of private end users and Hetzner tenants.) – hiergiltdiestfu Jun 14 '17 at 13:17
  • @Ferrybig Though the opposition to net neutrality from ISPs and other organizations shows that they'd love to be able to do this. – JAB Jun 14 '17 at 16:18
  • 2
    If your customers are in datacenters, then you could do something like this. AWS and Microsoft already do. It would cost a lot of money. – mfinni Jun 14 '17 at 20:48
  • So you want to kill net neutrality? – Kevin Jun 15 '17 at 06:22
  • 3
    This has nothing whatsoever to do with net neutrality. Such an arrangement would not technically be capital-I "Internet." This question isn't about prioritization, reservation, or favoring certain traffic -- it's about an additional allocation of bandwidth, over what the customer purchases, entirely outside the scope of neutrality, perfectly legal, logistically impractical, and prohibitively expensive. However... to even ask the question reflects a fundamental lack of understanding of how companies and end users purchase and how providers sell, provision, and deliver Internet bandwidth. – Michael - sqlbot Jun 15 '17 at 17:51

10 Answers10

48

For all intents and purposes, no, this is not possible. Even if it were, the technical and contractual logistics required would cripple your business.

Think through this a bit more: Joe user at University signs up for your service. You then approach one of the University's many providers (which one? How might you know what provider Joe user's traffic egresses out of today, let alone tomorrow when things change). So then you have to make agreements with all of their providers. But then you realize that somehow you need to make an addendum to a contract that was made between the provider and the university, without the university's involvement?!? How exactly do you expect that to work? Oh and then, you realize that Joe user's traffic is likely subject to heavy traffic shaping and that any "extra bandwidth" you could procure (if such a thing were even possible) would be pointless due to traffic shaping rules. Even if traffic shaping rules weren't in play, why do you think traffic to/from your site should get special treatment? How do you thing the network people would feel about that?

See? It's impossible for many, many reasons. Honestly, though, I think you're proposing a solution for a non-existent problem. If your customers are on university or corporate networks, there is likely plenty of bandwidth to spare. A few gigabytes is not that much data, and is lost in the noise when viewed with all of the other traffic on the network.

EEAA
  • 108,414
  • 18
  • 172
  • 242
  • 3
    Interesting points. Of course, whether a few gigabytes is a lot of data depends on the context. I guess it would be more accurate for me to say that a substantial amount of a user's time is spent waiting for data to upload. – Owen Jun 13 '17 at 23:52
  • 8
    "why do you think traffic to/from your site should get special treatment?" - I think the answer to this one is pretty obvious: it would be because he paid for it to get special treatment. – user253751 Jun 14 '17 at 02:04
  • 6
    @immibis Right, but he did so without consent of the network team, which is impossible and/or unethical depending on the situation. – EEAA Jun 14 '17 at 02:05
  • 3
    @Owen In that case, it helps to be more concrete. As an example, how much data is uploaded, and how long is the user waiting? What is the resulting upload speed, and how does that compare to the theoretical bandwith of the user's connection? Are you sure the bottlenecks aren't elsewhere (e.g. slow base64-encoding in javascript in the user's browser)? – marcelm Jun 14 '17 at 16:18
  • Question sounds like a request for a peering arrangement which is eminently possible, particularly with academic networks. – Flexo Jun 14 '17 at 19:52
  • 1
    @Flexo Not without the IT department's involvement, though, which is what the OP's goal is. – EEAA Jun 15 '17 at 00:52
  • @Owen Paying the ISP's is very unlikely to get faster upload speeds... even if the ISP wanted to, it's a single connection to the campus network or home or whatever... there's also equipment problems (some modems can only go so fast, or routers might be setup to throttle/shape traffic anyway, etc). What you can do, it make sure your service isn't the limiting factor. Your service needs a good download pipe, and a decent upload pipe (for acks, etc). Make sure your server never limits the uploaded traffic... – SnakeDoc Jun 16 '17 at 17:01
  • @SnakeDoc interestingly, my residential connection is 40Mbits. I could pay another £15 to the same provider for 70 Mbits. – Tim Jun 18 '17 at 08:48
  • @Tim that's what I wonder about. I know many ISPs offer different throughputs at different prices (my ISP has five throughput options). Is this a soft limit or do they actually go hook up a new cable? – Owen Jun 21 '17 at 01:39
  • @Owen The answer to that question depends on a very large set of variables, too many to dive into here. – EEAA Jun 21 '17 at 03:20
  • @Owen in my case there's no new cable between 40 and 70. There is between 17 and 40. – Tim Jun 21 '17 at 05:11
28

Depending on the frequency at which you exchange data with the customer, it could be cheaper, faster, and more efficient to mail them a storage drive, then pay to have it overnighted back to you when populated with data. This would cost a tiny fraction of what your agreement plan would cost long-term. Turnaround time may be nearly the same if lots of data is involved and their upload speed is low, as most asymmetric connections provided by most ISPs tend to be.

Ryan Griggs
  • 885
  • 2
  • 12
  • 27
17

It sounds like in effect you want to pay a users ISP to zero-rate traffic to your site, similar how to some cell carriers allow you to stream video from certain websites without impacting your allocation. If you are a major company like Google or Netflix then this has a ghost of a chance of being feasible, otherwise most companies will not talk to you -- its not worth their time to implement the necessary infrastructure for such a small user base.

The corporate or school clients (I am thinking such as a hotel) may very well not have the capacity to scale their bandwidth in an economic way either. They may have the equivalent of a T1 (hence the connection metering), and the only way to go above that is to buy another entire T1. You aren't going to want to pay for that. If your service is profitable enough to make those kind of deals anyway then your clients would be able to afford commercial tier ISPs anyway and won't need you to do so.

If you approached someplace that could meaningfully scale their bandwidth, they'd still want you to foot the bill for the whole years worth of extra bandwidth since they likely have a yearly contract. Anything government or education related (state schools) would have its own entire issues with procuring extra capacity at your behest.

So, if you have very very deep pockets and an army of lawyers and marketers then you might make some progress, but the model just does not scale and I can't fathom how it would be profitable.

Jason Martin
  • 4,865
  • 15
  • 24
  • Very interesting. I'm surprised this sort of thing is not possible given how useful it would be for some applications. But I see your point about the contractual overhead. – Owen Jun 13 '17 at 23:51
  • 3
    @Owen this kind of agreement run afoul of net neutrality. If I have a website, I can kill my superior competitors simply by pay the ISP to zero-rating my website. When more companies did that, nothing stops the ISP to artificially increase the base price for 'standard' connection, expecting most consumer to only use the zero-rated sites. – Martheen Jun 15 '17 at 06:46
  • 1
    @Owen It is because "some" applications are soo small and unwilling to day. Want to have special banwidth - SIMPLE: Get a quote for a leased line from your hosting provider to your customer. Customer pays, finished. This is what Azure Cloud can do for example. You will soon realize your customers - wont't value their time that high. – TomTom Jun 15 '17 at 07:28
  • All the answers here are good, but I'm accepting this one because it has the most useful detail. – Owen Jun 18 '17 at 03:38
  • The answer stresses the "it's too expensive for you" argument too much, for my taste. You have no idea what money is involved with the product sold by the OP. It would be better if it restricted itself to the technical aspect (the "T1 line" part of the argument) - which happily rules out most if not any solutions. But "commercial tier ISPs" do exist, end-to-end guaranteed bandwidths exist, and depending on the price he charges for his product, may just be very feasible. – AnoE Jun 18 '17 at 10:30
  • 1
    @AnoE The OP states many of the clients are on corporate or university networks where they can't get more bandwidth on their own. That screams 'dorm' or equivalent to me as departments would be able to get more bandwidth if needed. That kind of client can't afford a service that can afford to individually negotiate with ISPs to work through these issues. Of course I am extrapolating from limited information, but I think it is a reasonable extrapolation. – Jason Martin Jun 18 '17 at 14:54
7

TLDR:

No it is not possible because it is likely to be illegal in the country you are operating in. The proviso of course being that if you are in the US, the FCC/Trump are currently in the process removing the laws that protect users from the evils of such an arrangement.

Net Neutrality

Hi Ryan, I think you are asking the wrong question. Of course what you are asking is technically possible. In fact arrangements have been made in the past to that effect.

The better questions is "Is this arrangement LEGAL and if not, why not".

You question falls under the realm of Net Neutrality, which is where all ISPs treat all data the same. You want this because the world you imagine, taken to the logical conclusion would mean that you cannot access any website (at any appreciable speed) without that website forking lots of money to your ISP, assuming the ISP accepts their money.

Take a theoretical case study. Bob lives in a rural area which is only served by a fictional ISP. We shall call this Cast-Com. Bob likes his TV streaming and has signed up for a third party TV streaming service, we shall call WebFlicks.

However Cast-Com is owned by Temporal Magazine, who also have been their own inferior TV streaming service, which they would like to push onto their Cast-Com customers. Cast-Com begins throttling back the bandwidth from Cast-Com customers to WebFlicks locking Cast-Com users to the inferior TV service.

Of course this is all theoretical and has never happened before...

Aron
  • 209
  • 1
  • 5
  • 7
    The net neutrality thing is correct, but the way you have written this answer is overly biased. It will benefit from being far more objective – Suppen Jun 14 '17 at 05:22
  • 2
    Today, grocery stores get to choose which brands of mustard they offer and are free to demand that mustard manufacturers pay them for shelf space. Yet, the logical conclusion of this is *not* that mustard manufacturers shell out lots of money to grocery stores or that only very few brands of mustard are available in rural areas. Amazingly, businesses actually have a natural tendency to provide the products their customers want because they get the most business and can charge the most for their products this way. There is a reason it's all theoretical and has never happened before. – David Schwartz Jun 14 '17 at 06:03
  • @DavidSchwartz I see you don't speak sarcasm as a first language. This has happened as a specific case, but "the names have been changed to protect the innocent". – Aron Jun 14 '17 at 06:06
  • @Aron You mean it happened *once* (because one company tried to strong arm the other) and the free market worked it out? Oh. (And, ironically, in this case net neutrality would have favored the company that tried to strong arm the other and disrupted the fair balance that the free market struck.) – David Schwartz Jun 14 '17 at 06:23
  • 4
    @DavidSchwartz The free market didn't win out. The FCC approved the [FCC Open Internet Order 2010](https://en.wikipedia.org/wiki/FCC_Open_Internet_Order_2010). It wasn't ONCE, just with Netflix, there was the same case against Verizon as well. Also, net neutrality would not have benefited Comcast, as Comcast was trying to force users onto their own Time Warner based service, WHICH only had Time Warner content. Net Neutrality would allow people to vote with their feet and use Netflix instead of Time Warner. – Aron Jun 14 '17 at 06:41
  • There's an interesting point here that's getting lost behind snark and the need to show everyone how clever you've been "changing names to protect the innocent". This is an interesting comment but not really an answer right now. – Rob Moir Jun 14 '17 at 16:43
  • @Aron You mean the order that was vacated and likely had almost no impact on the situation which resolved itself because all of the involved companies had to provide the products their customers want to stay in business? – David Schwartz Jun 14 '17 at 17:32
  • 1
    Peering agreements aren't illegal anywhere, are inherently not neutral and shouldn't be illegal – Flexo Jun 14 '17 at 19:50
  • This is the right answer – Kik Jun 16 '17 at 19:09
  • @Flexo True. But this isn't peering. Peering involves creating physical point to point connections between the two parties, which increases the overall bandwidth capacity of the internet. Whereas the OP is suggesting bribes an ISP to reduce bandwidth to other websites. – Aron Jun 17 '17 at 07:53
5

Couple of points:

  • Couple of GBs is not that large these days. Most universities are on a very good bandwidth connection anyways. I knew our Physics Dept. (Wisconsin, USA) worked on the ATLAS / LHC projects where they used to fill a terabyte HDD with data in about a month all streamed in from the experimental sites over the internet for analysis. And this was back circa. 2010.

  • Rather than get data to your "site" could you get the "site" to the data? i.e. Load your code on a server and just install the server on site? Or do you need a cluster / cloud etc?

  • I know of one arrangement in the past where the university had two sites separated by 100 miles and due to the needs there were bandwidth bottlenecks. So they had a provider add fiber to connect those. And the network was configured to bypass that traffic directly.

  • In another case there was a local ISP who had a lot of university staff and students etc. log in so a lot of local traffic. They had worked out some kind of arrangement where they co-located some equipment on site at the University and the traffic to the University selectively bypassed the public internet due to routing rules.

curious_cat
  • 359
  • 2
  • 10
5

If you think you'll have sufficient traffic (think at a minimum several PBs worth month reasonably sustained), you could look at a "private peering" solution with their ISP (so traffic would go from your client, to their ISP and then directly to your datacenter). It'll cost a lot to setup (the cheapest option would be if you and the ISP both have facilities in the same internet exchange: most expensive would be fibre cables being routed between them and you), but they'll get savings as they wouldn't then have to pay to go out to the general internet before reaching you.

If traffic levels get even higher, you could then look at then running connections directly to your client with weighting on their BGP compatible router so traffic destined for you goes over the direct link.

I used to work for a major UK educational SaaS provider and we were looking at a private peering solution with JANET (who is the "main" ISP serving UK schools and universities) so clients could access our service in a similar way to you, but the cost/benefit ratio came nowhere near reasonable for our several hundred TBs per month.

Richy B.
  • 213
  • 1
  • 6
  • Weirdly this is the only right answer here and seems to be overlooked completely in net neutrality hysteria. – Flexo Sep 19 '18 at 16:21
2

If they are spending days, and not hours uploading data, shipping the data directly to your data center is probably the fastest. UPS shipping a box of TB SSD's is a much higher bandwidth then most internet connections. You can also pay for shipping for your customers, allowing you to pay more for higher speeds.

I am sure this is not the type of response you expected, but it is quite common when uploading data into the cloud for new companies to mail disc drives in to be loaded.

Dan Ciborowski - MSFT
  • 121
  • 4
2

I've been working in Telecoms for 10 years and my experience says this would be too expensive to implement in the operator's systems. It's not worth expanding on why, but a lot of answers above have touched on the major technical problems. However, if you are prepared to pay, you can utilise a load balancing solution to your problem. Either with the use of additional LTE/WiMax modems or with a combination of multiple traditional ADSL connections. Or both.

Genesis_GDK
  • 21
  • 3
0

Your best option is to create a program or use an existing solution for your customers to compress their files and then send them to your server to be processed.

Darren Rainey
  • 1
  • 1
-1

No, not without becoming an ISP and hooking your own lines up to customer premises'.

Jim B
  • 23,938
  • 4
  • 35
  • 58
  • Several years ago I worked with a local ISP who bought bandwidth from AT&T and resold it to their customers as a local DSL service. They used AT&T's phone lines, so no new wires were necessary, and the customers were billed and serviced by the local provider with AT&T carrying the data. – Ryan Griggs Jun 14 '17 at 01:53
  • 1
    @RyanGriggs Sure, that's possible in many places, but with that plan you'll have to become your customers' ISP. – user253751 Jun 14 '17 at 02:05
  • @immibis exactly. My point was that it is theoretically possible... but probably not desirable or cost-effective. – Ryan Griggs Jun 14 '17 at 02:07