How do I use the Google data compression proxy on Firefox?

20

12

I am using GNU Icecat (GNU version of Mozilla Firefox) on Trisquel GNU/Linux. In order to minimize/optimize data usage while web surfing, I want to use Google's data compression proxy.

So, how do I use Google Data Compression Proxy on Firefox?

Note/History: I was wandering around to get solution to my problem. The Answer brought me to a chat-room, where I finally found a way.

Pandya

Posted 2015-07-27T14:48:43.620

Reputation: 333

1https://wiki.mozilla.org/Mobile/Janus Grab Janus from github and run your own service? – ssnobody – 2015-08-18T01:02:11.813

https://code.google.com/p/datacompressionproxy/ May also be useful code... – ssnobody – 2015-08-18T01:03:54.620

1As a note, directly setting HTTP proxy to compress.googlezip.net:80 or 74.125.205.211:80 connected me to the Data Compression Proxy but I was unable to load any sites with: "This page cannot be loaded via the Chrome Data Compression Proxy. Try reloading the page." I suppose the proxy could be checking for some Chrome-specific header... – ssnobody – 2015-08-18T01:11:40.537

Looks like the specific header the proxy is looking for is Chrome-Proxy. Code for generating that header correctly is within background.js in datacompressionproxy linked above. – ssnobody – 2015-08-18T01:14:45.343

Answers

19

Using the Google Chrome Data Compression Proxy on non-Chrome browsers involves two steps.

  1. Set your HTTP proxy to proxy.googlezip.net and port 80.
  2. For each of your requests, calculate and inject a Chrome-Proxy header into your request.

The Chrome-Proxy header should contain the following string: ps=<timestamp>-<num1>-<num2>-<num3>, sid=<md5 string>, b=<build>, p=<patch>, c=<platform> where:

  • timestamp: is the current number of seconds elapsed since unix epoch
  • num1, num2, num3: are random numbers which can be set to 0
  • md5 string: is the md5 of the auth string
  • auth string: "<timestamp>" + "<auth key>" + "<timestamp>"
  • auth key: ac4500dd3b7579186c1b0620614fdb1f7d61f944
  • build: Chrome build number, can be set to 2214
  • patch: Chrome patch number, can be set to 115
  • platform: Chrome platform, can be set to "win"

As an full example of the header you might have: Chrome-Proxy: ps=1439961190-0-0-0, sid=9fb96126616582c4be88ab7fe26ef593, b=2214, p=115, c=win

Now, to implement this practically, I think the easiest solution is to use an extension but there does not appear to be one for firefox that already has this functionality.

Therefore, I have implemented my own firefox extension for Firefox version 38 or above that allows anyone to use the Google Chrome Data Compression Proxy on firefox and am including the code below.

The following is the procedure that I used to develop and test the extension, and would allow others to regenerate the extension from scratch.


  1. Get and install nodejs
  2. Use npm from nodejs to install jpm via npm install jpm

    Note that some versions of Ubuntu package node as nodejs so if jpm can't find node you may need to install another distribution of nodejs or create a link for node to nodejs

  3. Create a new directory for the extension and cd into it

  4. jpm init
  5. Answer prompts with defaults
  6. Replace your index.js with the code provided below
  7. jpm run

    Note that if jpm is unable to find firefox in the standard locations, you'll also need to pass the -b flag to jpm to tell it where to find your browser. (e.g. jpm run -b /path/to/Firefox/Nightly)

You should now have firefox open with this new extension loaded and all traffic proxied through the Google Chrome Data Compression Proxy during this session.

Because jpm run creates a temporary profile, you will need to jpm run this extension anytime you'd like your traffic proxied. You could also jpm xpi and Install Add-on From File in Add-On manager.

You can verify that your data is being proxied by examining the response headers for the Via header which indicates data being sent over the Chrome Compression Proxy (e.g. Via: "1.1 Chrome-Compression-Proxy")

You can determine how much data you are saving by comparing the Content-Length header to the X-Original-Content-Length header in server responses.


Code for index.js follows:

/*
 * Data Compressor Proxy 0.0.1
 * License: MPL
 *
 * Based on
 * Data Compression Proxy Extension for Google Chrome on Desktop
 * (c) 2014 Jerzy Glowacki. License: Apache 2.0

 * and includes
 * JavaScript MD5 1.0.1
 * https://github.com/blueimp/JavaScript-MD5
 * Copyright 2011, Sebastian Tschan
 * https://blueimp.net
 * Licensed under the MIT license:
 * http://www.opensource.org/licenses/MIT
 *
 * JavaScript MD5 1.0.1 is Based on
 * A JavaScript implementation of the RSA Data Security, Inc. MD5 Message
 * Digest Algorithm, as defined in RFC 1321.
 * Version 2.2 Copyright (C) Paul Johnston 1999 - 2009
 * Other contributors: Greg Holt, Andrew Kepert, Ydnar, Lostinet
 * Distributed under the BSD License
 * See http://pajhome.org.uk/crypt/md5 for more info.
 */

var self = require('sdk/self');
var { Class } = require('sdk/core/heritage');
var { Unknown } = require('sdk/platform/xpcom');
var {Cc, Ci} = require("chrome");
var prefsvc = require("sdk/preferences/service");

var httpRequestObserver = Class(
{
  extends:  Unknown,
  interfaces: [ 'nsIObserver' ],
  topic: '*',
  observe: function(subject, topic, data)
  {
    if (topic == "http-on-modify-request") {
      var httpChannel = subject.QueryInterface(Ci.nsIHttpChannel);
      httpChannel.setRequestHeader("Chrome-Proxy", this.authHeader(), false);
    }
  },

  get observerService() {
    return Cc["@mozilla.org/observer-service;1"]
                     .getService(Ci.nsIObserverService);
  },

  authHeader : function() {
    var authValue = 'ac4500dd3b7579186c1b0620614fdb1f7d61f944';
    var timestamp = Date.now().toString().substring(0, 10);
    return 'ps=' + timestamp + '-' + '0' + '-' + '0' + '-' + '0' + ', sid=' + md5(timestamp + authValue + timestamp) + ', b=2214' + ', p=115' + ', c=win';
  },

  register: function()
  {
    this.observerService.addObserver(this, "http-on-modify-request", false);
  },

  unregister: function()
  {
    this.observerService.removeObserver(this, "http-on-modify-request");
  }
});

var httprequestobserver = httpRequestObserver();
httprequestobserver.register();
prefsvc.set("network.proxy.http", "proxy.googlezip.net");
prefsvc.set("network.proxy.http_port", 80);
prefsvc.set("network.proxy.type", 1);

!function(a){"use strict";function b(a,b){var c=(65535&a)+(65535&b),d=(a>>16)+(b>>16)+(c>>16);return d<<16|65535&c}function c(a,b){return a<<b|a>>>32-b}function d(a,d,e,f,g,h){return b(c(b(b(d,a),b(f,h)),g),e)}function e(a,b,c,e,f,g,h){return d(b&c|~b&e,a,b,f,g,h)}function f(a,b,c,e,f,g,h){return d(b&e|c&~e,a,b,f,g,h)}function g(a,b,c,e,f,g,h){return d(b^c^e,a,b,f,g,h)}function h(a,b,c,e,f,g,h){return d(c^(b|~e),a,b,f,g,h)}function i(a,c){a[c>>5]|=128<<c%32,a[(c+64>>>9<<4)+14]=c;var d,i,j,k,l,m=1732584193,n=-271733879,o=-1732584194,p=271733878;for(d=0;d<a.length;d+=16)i=m,j=n,k=o,l=p,m=e(m,n,o,p,a[d],7,-680876936),p=e(p,m,n,o,a[d+1],12,-389564586),o=e(o,p,m,n,a[d+2],17,606105819),n=e(n,o,p,m,a[d+3],22,-1044525330),m=e(m,n,o,p,a[d+4],7,-176418897),p=e(p,m,n,o,a[d+5],12,1200080426),o=e(o,p,m,n,a[d+6],17,-1473231341),n=e(n,o,p,m,a[d+7],22,-45705983),m=e(m,n,o,p,a[d+8],7,1770035416),p=e(p,m,n,o,a[d+9],12,-1958414417),o=e(o,p,m,n,a[d+10],17,-42063),n=e(n,o,p,m,a[d+11],22,-1990404162),m=e(m,n,o,p,a[d+12],7,1804603682),p=e(p,m,n,o,a[d+13],12,-40341101),o=e(o,p,m,n,a[d+14],17,-1502002290),n=e(n,o,p,m,a[d+15],22,1236535329),m=f(m,n,o,p,a[d+1],5,-165796510),p=f(p,m,n,o,a[d+6],9,-1069501632),o=f(o,p,m,n,a[d+11],14,643717713),n=f(n,o,p,m,a[d],20,-373897302),m=f(m,n,o,p,a[d+5],5,-701558691),p=f(p,m,n,o,a[d+10],9,38016083),o=f(o,p,m,n,a[d+15],14,-660478335),n=f(n,o,p,m,a[d+4],20,-405537848),m=f(m,n,o,p,a[d+9],5,568446438),p=f(p,m,n,o,a[d+14],9,-1019803690),o=f(o,p,m,n,a[d+3],14,-187363961),n=f(n,o,p,m,a[d+8],20,1163531501),m=f(m,n,o,p,a[d+13],5,-1444681467),p=f(p,m,n,o,a[d+2],9,-51403784),o=f(o,p,m,n,a[d+7],14,1735328473),n=f(n,o,p,m,a[d+12],20,-1926607734),m=g(m,n,o,p,a[d+5],4,-378558),p=g(p,m,n,o,a[d+8],11,-2022574463),o=g(o,p,m,n,a[d+11],16,1839030562),n=g(n,o,p,m,a[d+14],23,-35309556),m=g(m,n,o,p,a[d+1],4,-1530992060),p=g(p,m,n,o,a[d+4],11,1272893353),o=g(o,p,m,n,a[d+7],16,-155497632),n=g(n,o,p,m,a[d+10],23,-1094730640),m=g(m,n,o,p,a[d+13],4,681279174),p=g(p,m,n,o,a[d],11,-358537222),o=g(o,p,m,n,a[d+3],16,-722521979),n=g(n,o,p,m,a[d+6],23,76029189),m=g(m,n,o,p,a[d+9],4,-640364487),p=g(p,m,n,o,a[d+12],11,-421815835),o=g(o,p,m,n,a[d+15],16,530742520),n=g(n,o,p,m,a[d+2],23,-995338651),m=h(m,n,o,p,a[d],6,-198630844),p=h(p,m,n,o,a[d+7],10,1126891415),o=h(o,p,m,n,a[d+14],15,-1416354905),n=h(n,o,p,m,a[d+5],21,-57434055),m=h(m,n,o,p,a[d+12],6,1700485571),p=h(p,m,n,o,a[d+3],10,-1894986606),o=h(o,p,m,n,a[d+10],15,-1051523),n=h(n,o,p,m,a[d+1],21,-2054922799),m=h(m,n,o,p,a[d+8],6,1873313359),p=h(p,m,n,o,a[d+15],10,-30611744),o=h(o,p,m,n,a[d+6],15,-1560198380),n=h(n,o,p,m,a[d+13],21,1309151649),m=h(m,n,o,p,a[d+4],6,-145523070),p=h(p,m,n,o,a[d+11],10,-1120210379),o=h(o,p,m,n,a[d+2],15,718787259),n=h(n,o,p,m,a[d+9],21,-343485551),m=b(m,i),n=b(n,j),o=b(o,k),p=b(p,l);return[m,n,o,p]}function j(a){var b,c="";for(b=0;b<32*a.length;b+=8)c+=String.fromCharCode(a[b>>5]>>>b%32&255);return c}function k(a){var b,c=[];for(c[(a.length>>2)-1]=void 0,b=0;b<c.length;b+=1)c[b]=0;for(b=0;b<8*a.length;b+=8)c[b>>5]|=(255&a.charCodeAt(b/8))<<b%32;return c}function l(a){return j(i(k(a),8*a.length))}function m(a,b){var c,d,e=k(a),f=[],g=[];for(f[15]=g[15]=void 0,e.length>16&&(e=i(e,8*a.length)),c=0;16>c;c+=1)f[c]=909522486^e[c],g[c]=1549556828^e[c];return d=i(f.concat(k(b)),512+8*b.length),j(i(g.concat(d),640))}function n(a){var b,c,d="0123456789abcdef",e="";for(c=0;c<a.length;c+=1)b=a.charCodeAt(c),e+=d.charAt(b>>>4&15)+d.charAt(15&b);return e}function o(a){return unescape(encodeURIComponent(a))}function p(a){return l(o(a))}function q(a){return n(p(a))}function r(a,b){return m(o(a),o(b))}function s(a,b){return n(r(a,b))}function t(a,b,c){return b?c?r(b,a):s(b,a):c?p(a):q(a)}"function"==typeof define&&define.amd?define(function(){return t}):a.md5=t}(this);

ssnobody

Posted 2015-07-27T14:48:43.620

Reputation: 2 510

3Something worth noting is that on ubuntu 14.04 (and trisquel is an ideologically pure varient on this, the node command is nodejs not node. I tend to recommend using a newer version on an alternate repo as a result - nodesource is what I favour. – Journeyman Geek – 2015-08-22T07:26:44.763

1

Finally It works well on Abrowser 38 on Trisquel GNU/Linux

– Pandya – 2015-08-24T12:49:13.227

Can you help me at there

– Pandya – 2015-10-02T10:30:01.337

Use jpm xpi to build an XPI and install it. – ssnobody – 2015-10-02T13:55:52.537

@ssnobody thanks ; you've already mentioned in your answer but I missed it! – Pandya – 2015-10-04T07:07:46.240

Enabling DCP in Firefox is pointless since it uses WebP and Firefox does not support WebP. – niutech – 2016-01-25T17:15:09.413

I've used it to test the extension and shown that it does in fact provide actual savings (though not drastic ones). In addition to the "transcoding" of images to WebP you reference the data compression proxy also performs "Content-aware compression" and "minification of HTML, JavaScript and CSS resources". Presumably Firefox supports HTML, JavaScript and CSS, and it certainly doesn't have any trouble displaying images when using the ext. – ssnobody – 2016-01-26T06:21:06.627

As a note, users are free to examine those headers I mention above that detail their savings if they are curious about how much they are gaining through use of the Data Compression Proxy. – ssnobody – 2016-01-26T06:24:11.227

7

There is now a Google DataSaver proxy for Firefox extension posted in the Mozilla Add-ons website.

Google data compression proxy for Firefox-enables the DataSaver feature for Firefox

galacticninja

Posted 2015-07-27T14:48:43.620

Reputation: 5 348

Whoever built it used the code I posted above but made several improvements like private browsing support and whitelisting. Very Nice! – ssnobody – 2016-08-08T22:39:42.087