Do text based browsers reduce network traffic?

27

5

Do text based browsers such as lynx, links and elinks consume less bandwidth than a GUI based browser (ie, Firefox, Chrome etc.)?

I'm guessing no reduction in traffic.
Rationale: I think a text based browser downloads the entire page as it is offered by the server. Any streamlining or reduction of page widgetry is done locally.

Maybe there is some reduction in traffic, as most text based browsers won't execute page scripts or SWF which might cause more traffic.

Paulb

Posted 2014-09-20T10:25:49.800

Reputation: 737

19Or actually download images.. – Journeyman Geek – 2014-09-20T13:26:20.703

4Think in terms of expecting a three or more orders of magnitude reduction. – user2338816 – 2014-09-21T00:36:34.703

1@user2338816 Yes, there may be a difference of three orders of magnitude. Try YouTube! [ Adding later: ] Oops, that's another three! – Volker Siegel – 2014-09-21T09:52:16.593

3@user2338816 three orders of magnitude would be unlikely. For example, for this particular page the original html document is some 10% of the whole downloadable sources disregarding caching, so just a single order of magnitude; and many heavyweight items (javascript libraries, large images, etc) are successfully cached, often reused across many pages and thus get downloaded very rarely, so their size doesn't really represent their impact on total network traffic. – Peteris – 2014-09-21T10:09:10.800

1@Peteris 3 might be a little high, but 2 certainly is not. Let's say the 10% you notice here is the same across most regular sites. Then take into account that video traffic amounts to 78% of all video traffic. This means for the rest of the 22% of the traffic, we can expect 2.2% to be text. Now, this is napkin math, but 2 orders of magnitude seems to be where it is. – corsiKa – 2014-09-22T16:13:34.070

@Peteris Yeah, you're probably right. Two is a better guide. I was actually slightly surprised at the size of this page, which is close to an order of magnitude larger than reported sizes of, say, a small number of random pages from a few microsoft.com and ibm.com tech forum sites that I compared. The physical sizes of many sites' pages have gotten so high that they've apparently reduced the ratio. (Not by reducing non-text content, perhaps unfortunately.) – user2338816 – 2014-09-23T05:35:52.657

Did we factor in HTML is highly compressible in stream so the source transmitted is much smaller sent via gzip, while many images can not be compressed further. – Sun – 2014-09-25T04:58:38.137

Answers

53

Web server doesn't send "entire website", but documents that browsers request.

For example when you access https://www.google.com/ the browser queries server for the document https://www.google.com/. The server processess the request and sends back some HTML code.

Then the browser checks what the server has sent. In this case it's HTML webpage, so it parses the document and looks for referenced scripts, stylesheets, images, fonts etc.

At this stage the browser has finished downloading that document, but still doesn't have downloaded referenced documents. It can choose to do so or to skip them. Regular browsers will try to download all referenced documents for best viewing experience. If you have an ad blocker (like Adblock) or privacy plugin (Ghostery, NoScript), it may block some resources too.

Then the browser downloads referenced documents one by one, each time asking the server explicitly for a single resource. In our Google example the browser will find following references, just to name a few of them:

(actual files may be different for different users, browsers and sessions and may change over time)

Text-based browsers don't download images, Flash files, HTML5 video etc. so they download less data.


@NathanOsman makes a good point in comments: Sometimes small images are embedded directly in HTML documents and in those cases downloading them cannot be avoided. This is another trick used to reduce number of requests. They are very small, though, otherwise the overhead of encoding binary file in base64 is too big. There are few such images on Google.com: (base64 encoded size / decoded size)

  • 19×11 keyboard icon (106 B / 76 B)
  • 28×38 microphone icon (334 B / 248 B)
  • 1×1 px transparent GIF (62 B / 43 B) which shows up in Chrome Dev Tools Resources tab, but I couldn't find it in the source - probably added later with JavaScript
  • 1×1 px corrupted GIF file that appears twice (34 B / 23 B). Its purpose is a mystery to me.

gronostaj

Posted 2014-09-20T10:25:49.800

Reputation: 33 047

1I liked the combined images link - that is a neat trick. – prateek61 – 2014-09-20T15:05:50.423

12

@prateek61 All the major websites use it; it's actually a technique borrowed from video games. :) In fact, many web developers even call it "CSS sprites" or similar (at Amazon it is referred to as "spriting" but I don't know how common that variation is).

– fluffy – 2014-09-21T01:06:12.077

3Well, almost - it is possible to embed images directly in HTML using the data URI scheme (data:). – Nathan Osman – 2014-09-22T01:33:00.520

Yeah, if you're willing to sacrifice most of the content of the internet, you can reduce bandwidth. Seems logical... Let's not forget that 78% of the internet is video traffic...

– corsiKa – 2014-09-22T16:15:26.757

25

I would suspect that they do. I don't believe that text-based browsers will (by default) even download resources such as images or external entities, such as fonts (if needed), scripts, etc.

I did some basic testing with tcpdump trying to get this IANA page (http://www.iana.org/domains/reserved) with both lynx and then wget, and here were my results (HTTP commands only, I can provide the rest if needed).

lynx http://www.iana.org/domains/reserved

  4   0.072774 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /domains/reserved HTTP/1.0
 10   0.146971   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (text/html)

wget -p http://www.iana.org/domains/reserved

  4   0.072139 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /domains/reserved HTTP/1.0
 22   0.145905   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (text/html)
 28   0.219381 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /robots.txt HTTP/1.0
 30   0.291877   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (text/plain)
 32   0.292550 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2013.1/screen.css HTTP/1.0
 94   0.440388   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (text/css)
100   0.514652 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2013.1/print.css HTTP/1.0
132   0.660071   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (text/css)
138   0.733546 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_img/bookmark_icon.ico HTTP/1.0
154   0.878227   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)
160   0.950713 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_js/2013.1/jquery.js HTTP/1.0
277   1.172095   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/x-javascript)
283   1.244571 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_js/2013.1/iana.js HTTP/1.0
285   1.317059   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK
287   1.317609 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_img/2013.1/iana-logo-header.svg HTTP/1.0
332   1.464356   192.0.32.8 -> xx.xx.xx.xx HTTP/XML HTTP/1.1 200 OK
337   1.536749 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_img/2013.1/icann-logo.svg HTTP/1.0
348   1.610449   192.0.32.8 -> xx.xx.xx.xx HTTP/XML HTTP/1.1 200 OK
353   1.682727 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2013.1/fonts/OpenSans-Light.ttf HTTP/1.0
658   2.552776   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)
663   2.625015 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2013.1/fonts/OpenSans-Regular.ttf HTTP/1.0
926   3.063537   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)
932   3.135931 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2013.1/fonts/OpenSans-Semibold.ttf HTTP/1.0
1216   3.573481   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)
1222   3.645984 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2013.1/fonts/OpenSans-Bold.ttf HTTP/1.0
1500   4.012966   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)
1506   4.085693 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2013.1/fonts/Inconsolata.otf HTTP/1.0
1584   4.304016   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)
1589   4.376612 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_img/2011.1/icons/icon_alert.png HTTP/1.0
1592   4.449311   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (PNG)
1594   4.449930 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_img/2013.1/iana-logo-homepage.png HTTP/1.0
1627   4.596125   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (PNG)
1633   4.668596 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_img/2013.1/iana-logo-homepage@2x.png HTTP/1.0
1704   4.895581   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (PNG)
1710   4.968097 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2011.1/fonts/OpenSans-Light.ttf HTTP/1.0
1982   5.364584   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)
1988   5.438091 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2011.1/fonts/OpenSans-Regular.ttf HTTP/1.0
2243   5.830353   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)
2249   5.902861 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2011.1/fonts/OpenSans-SemiBold.ttf HTTP/1.0
2259   5.976674   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 404 NOT FOUND  (text/html)
2263   6.047876 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2011.1/fonts/OpenSans-Bold.ttf HTTP/1.0
2533   6.415590   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)
2539   6.487909 xx.xx.xx.xx -> 192.0.32.8   HTTP GET /_css/2011.1/fonts/Inconsolata.otf HTTP/1.0
2616   6.720477   192.0.32.8 -> xx.xx.xx.xx HTTP HTTP/1.1 200 OK  (application/octet-stream)

So I realize this is not a great test as wget may download resources that a browser might not download, but the example I think holds - there are many more requests required to render content on a GUI browser. As such, it GUI browsers will generally cause more network traffic than text based browsers.

prateek61

Posted 2014-09-20T10:25:49.800

Reputation: 1 146

I don't think wget can be considered a browser. Better try with elinks or something similar. – Darkhogg – 2014-09-21T15:22:18.450

So I used wget simply to demonstrate all the HTTP requests and responses that will be made. The -p parameter is defined as the following: -p, --page-requisites get all images, etc. needed to display HTML page.. I didn't want to use an actual GUI browser as they tend to make other requests which I didn't want to have to filter out. – prateek61 – 2014-09-21T15:26:15.480

3I like this answer also. The wget inspection was interesting. – Paulb – 2014-09-22T12:21:53.477

1

I think text based Browsers will reduce the amount of transfered data significantly as they will not request all those bloated web 2.0 high Resolution Images, Videos and interactive stuff (Flash and others).

I suggest you just test this by Setting up an IPtables Rule that will Count the amount of traffic hitting a specific IPtables rule.

For example create a rule for port 80+443 with traffic counting and browse the web with a normal browser, reset IPtables Counter and do the same with a textbased browser.

Keep in mind that you cannot compare both runs 100% because dynamic webcontent (ads and stuff) might vary on each Access.

Sebastian Brabetz

Posted 2014-09-20T10:25:49.800

Reputation: 131