Is "HTTPS Everywhere" still relevant?

95

14

HTTPS Everywhere is a browser extension, a collaboration between The Tor Project and the Electronic Frontier Foundation, that automates rewriting requests for HTTP URLs to the secure HTTPS alternative if available. It's apparently been around for roughly a decade but was never on my radar until someone recently asked about it. Trying to research it has produced a mixed bag of information.

  1. Regardless of need, it isn't clear how useful it is "out-of-the-box". Various articles refer to the need to supplement defaults with whitelists and rules to get the full benefits. So implementing it appears to not be a trivial task.

  2. At least at one time, a substantial portion of web sites were HTTP-only, so using such software could provide only limited benefit. It appears that sites dealing with sensitive personal data have pretty much moved to HTTPS-only. Google implemented various measures to incentivize web sites to convert to HTTPS. It isn't clear how big of a problem HTTP still is (or if it still is, whether the problem is quickly disappearing).

    It also isn't clear whether sites converting to HTTPS are retaining the HTTP links just for legacy visitors, and automatically redirecting to their HTTPS site.

  3. The major browsers appear to all have either incorporated logic to prefer HTTPS sites when available, or are well into the process of implementing it. At least Google (haven't seen anything about other search engines), has a program by the same name (not clear if it is actually the same product), to automatically attempt an HTTPS connection on searches.

  4. Three years ago or so, there were articles about "why you need to install HTTPS Everywhere". A number of more recent articles have suggested that people should stop suggesting that people install this software. The gist seems to relate to browsers already duplicating the functionality.

So it isn't clear whether HTTP is still a substantive problem needing a solution, and if so, whether software that tries HTTPS links first can solve what's left of it. Has this whole issue been overtaken by events?

I'm looking for context rather than opinion (i.e., the facts describing the current situation rather than opinion about how good or bad it is, or whether I need the software). For example, do the major browsers now provide the remedy that HTTPS Everywhere was developed for? Is HTTP now virtually limited to sites where there's no personal data? Is there government or industry regulation that is intended to render this a non-issue? In other words, the kinds of objective information that will allow me (and others) to understand the current state of affairs in order to form my own opinion and determine relevance for myself.

fixer1234

Posted 2019-10-27T02:06:25.153

Reputation: 24 254

1I'm not sure how this plays into this, if at all, but in some cases, HTTPS is not supported by a browser or OS, in which case HTTP is required. I've gotten complaints when my website automatically rewrote HTTPS to HTTP, because this made my site entirely inaccessible to some people. Now my site defaults to HTTP, although it does support HTTP. Ideally, in the future I'll have it default to HTTPS, but not rewrite HTTP to HTTPS, which should never be done as a general best practice. – InterLinked – 2019-10-29T12:41:14.020

1

@InterLinked there are ways for your server to automatically upgrade to HTTPS only if it is correctly supported (and continue using HTTP for archaic noncompliant clients) - see https://scotthelme.co.uk/we-dont-do-https-for-backwards-compatibility/ for example

– Matija Nalis – 2019-11-13T17:22:55.603

@MatijaNalis Thanks for the link, I'll look into it. I'm curious if it works for browsers that claim to support SSL but don't do it properly. For example, support for SSL in Chrome 49 on Windows XP is basically nonexistent - you just get SSL certificate errors. Pages work perfectly in Internet Explorer and Firefox. HTTP also works, but many (maybe not all) HTTPS sites do not work. I'd need a solution that would handle this kind of situation, too. – InterLinked – 2019-11-13T18:20:38.163

@InterLinked Some of them should work - for example https image one. Browser will try to get 1x1 pixel image over HTTPS, and only if it succeeds it will receive HSTS header which will force all future connections to that domain to use HTTPS. If TLS failed for any reason, the browser would not receive HSTS headers over it, and would continue using website via HTTP (only without showing 1x1 https image). If you want quick test, I've implemented some (but not this 1x1 image) of the upgrade methods here

– Matija Nalis – 2019-11-13T23:52:30.743

@InterLinked When you say rewrite, do you also mean redirect? Because this is what I'm doing: Redirect permanent / https://${subname}.${name}.${tld}/, does this not qualify as a general best practice? – Chazy Chaz – 2019-11-14T14:41:49.840

Well, I've tried both rewrite using htaccess and other methods of redirecting. It's not a general best practice to redirect unconditionally because it won't work for all clients. – InterLinked – 2019-11-14T21:48:16.763

Answers

75

HTTPS Everywhere certainly used to be more necessary during the days of mixed content and half-hearted website configurations. The web is certainly more mature nowadays, with technologies like HSTS which can be used by any site, and public key pinning for the bigger players (now deprecated in favor of Certificate Transparency - thanks to Justin for informing me).

So, whether the extension is useful depends heavily on your individual use case. Making custom rules for websites that serve both HTTP and HTTPS is something the extension excels at, and I'm not aware of any others that do a similar job. Even in situations where a website doesn't support HTTPS, the extension will ensure that any references to third-party domains such as CDNs will be upgraded to HTTPS, even if the original reference was protocol-neutral.

BoffinBrain

Posted 2019-10-27T02:06:25.153

Reputation: 1 896

So it sounds like the major browsers don't fully handle this yet, and the Internet is improved but not "cured"? "whether the extension is useful depends heavily on your individual use case": If that's the case, that implies that the need hasn't completely gone away. Thanks. +1. Let me give this some time for other answers before accepting. – fixer1234 – 2019-10-27T02:59:41.727

6@fixer1234 Brave (a Chromium-based privacy-focused browser that can even use Chrome extensions) does have this functionality built in. If its counter is correct, then it certainly has upgraded a lot of my connections to HTTPS which seems useful to me, regardless of whether you get it via built-in functionality or an extension. I agree with this answer's take: the state of things is vastly better than it was, but mistakes still happen, and smaller websites have a hard time getting things right so it will never be "cured" as long as the default protocol is unsecured. – briantist – 2019-10-27T05:48:05.543

1I was gonna say that yeah by now it is pretty useless, a tiny improvement at best since browsers try to get HTTPS sites if possible. (or soon will be) But I haven't thought about it upgrading static resources (of CDNs). And since it is whitelist based and doesn't need to check if the domain also serves HTTPS it can upgrade directly which certainly seems to be beneficial. (Even if the target resource gets redirected to the HTTPS variation. Because with HTTPS Everywhere the HTTP request never even gets sent) – BrainStone – 2019-10-27T17:22:18.463

1Plenty of sites don't use HSTS headers/preload because they want to support the widest array of people they can. I just checked Amazon.com and it doesn't appear to return a Strict-Transport-Security header, for example. HTTPS Everywhere would protect against a malicious link to Amazon that used http to exploit the connection. – IllusiveBrian – 2019-10-27T22:03:33.440

3Yes, it certainly does, because the extension's list of domains is larger than the list of pinned keys that are shipped with the browser. I'm still not sure why a website wouldn't use HSTS, though, since older devices that only support SSL etc. will not care about that header anyway. – BoffinBrain – 2019-10-27T22:05:05.047

@BoffinBrain TLS 1.0 and 1.1 are supposed to be deprecated but some browsers still only support them and not TLS 1.2, so sites may want to give those users the option to downgrade to http rather than not be able to use their site when they stop supporting 1.0 and 1.1. Eventually some vulnerability will be found with 1.2 or it will just be considered too weak compared to the then modern TLS1.5 and sites will stop supporting it as well, and a whole bunch of software made without support for TLS1.3 will be forced to drop to http. Not saying I support that thinking entirely, but it has merit. – IllusiveBrian – 2019-10-27T22:12:27.963

1That sounds like a terrible idea! Using SSL or old TLS is still better than HTTP. Nobody should be encouraging a downgrade to no encryption at all. – BoffinBrain – 2019-10-27T22:19:16.107

@BoffinBrain Not supporting deprecated transport security versions prevents version downgrade attacks. More users use software the supports both TLS1.2 and lower versions than that don't support TLS1.2, so there is also the argument that they are protecting people keeping their software reasonably up to date over people not making that effort, given that the former pool is larger and downgrade attacks are hard for a user to spot. – IllusiveBrian – 2019-10-27T22:24:22.523

Also, I was wrong about Amazon, they just have www.amazon.com listed, but not amazon.com (which is just a redirect to www.amazon.com). – IllusiveBrian – 2019-10-27T22:30:40.923

Hmm. Yep, I guess downgrades are bad too. I just had a peek at Amazon and thankfully there is a HSTS after the redirects (I think the header is disallowed on a HTTP response, which makes sense). – BoffinBrain – 2019-10-27T23:19:34.237

4

Note that HTTP Public Key Pinning was deprecated in 2018 and was replaced by Certificate Transparency

– Justin Lessard – 2019-10-28T14:26:14.757

Another reason is also that some websites used to serve different content, based on whether they were called via HTTP or HTTPS. Since this is now mostly recognized as an anti-patter, HSTS can now be deployed more widely. – MechMK1 – 2019-10-28T16:22:35.467

Nothing that wrong with serving content files like JQuery over HTTP rather than HTTPS, it's a generic file that doesn't contain anything sensitive and it's marginally faster too. – None – 2019-10-29T02:23:14.430

5@LTPCGO It's EVERYTHING WRONG to let anyone inject any javascript into your "secure" page by performing MITM. – user11153 – 2019-10-29T09:52:36.193

@user11153 true for script content...my point is still valid for e.g. images. Most browsers seem to have some sort of warning though if a page mixes content. – None – 2019-10-29T16:46:34.843

1

@user11153 You can use subresource integrity to make sure that the CDN shows the same file that you expected.

– Michael - Where's Clay Shirky – 2019-10-29T21:22:23.077

@Michael I don't think it's a good enough reason to avoid https, especially since a good chunk of your visitors will use a browser that does not implement that. – Shautieh – 2019-11-13T10:38:02.613

10

Speaking as a previous ruleset contributor to HTTPS Everywhere, I have the following to offer.

  • The HTTPS Everywhere project periodically tests all of their rewriting rules and disables those which fail for any reason. This ensures a relatively quick response to changing website configurations, but can lead to a significant portion of the rulesets being disabled unless significant maintenance effort is expended. Suggestions that the central rulesets should be supplemented mainly arise from ignorance that these central rulesets can and should be corrected. It's a matter of volunteer availability.

  • Significant progress has been made in moving the web to HTTPS-only, but many sites are still misconfigured and many more have not implemented the crucial HSTS preload protection needed to prevent first-connection attacks. Sites which implement this protection are shortly thereafter removed from HTTPS Everywhere's rulesets.

  • Web browser technology is very useful, but anything they do beyond the HSTS preload list is only nice to have. HTTPS Everywhere provides a stopgap for sites which have not enabled HSTS through the browser and essentially need a custom community-maintained HSTS configuration.

In summary, it doesn't harm to keep it installed. Bear with it for a few more years and hopefully all this will become redundant.

Bardi Harborow

Posted 2019-10-27T02:06:25.153

Reputation: 275

8

While improved awareness of HTTPS and HSTS have certainly brought security standards forward, there is still use for the HTTPS Everywhere extension:

HSTS is great at protecting against HTTP downgrade attacks but one thing to notice is that it is based on a trust on first use model. This means your first connection to the site must be through HTTPS or else the HSTS protection can be compromised (for example a HTTP to HTTPS 301 redirect is a window of opportunity for an attack).

HSTS normally protects against this with the HTST preload list, a list of domains built into the browser which forces the first connection to use only HTTPS for those sites. However getting onto the list (and waiting for the change to be applied in browsers) takes some time and not every site bothers to register itself. This is where the browser extension helps out by ensuring all first connections are through HTTPS only.

Another smaller case is when the website's HTTPS is located on a different path from the usual. For example a website might have http://www.example while having their secure site on https://secure.example. HTTPS Everywhere keeps a database of domains to ensure you are going to the correct URL for HTTPS.

Footnote: public key pinning also helps, but even Chrome decided to remove it for low adoption rates and the potential for being a foot-gun.

AlphaD

Posted 2019-10-27T02:06:25.153

Reputation: 181

1I'm a hobbyist site maintainer, and implemented HTTPS+HSTS on my site. Getting on the HSTS preload list took me about 20 minutes, 10 of which was reading what to do. I do agree that some wouldn't bother to do a complete job in this though. – Jonathan – 2019-10-28T08:29:31.073

@Jonathan Are you sure it only took you 20 mins? It might take only that much to apply to join, but it normally requires longer for the new version of the browser to be pushed out with the updated list. It is supposed to be embedded into the software. – AlphaD – 2019-10-28T08:59:47.050

1@Jonathan Also clarified the answer a bit to address your point. Thanks. – AlphaD – 2019-10-28T09:13:49.017

3

@Jonathan it's often easier to make this sort of stuff work on small or hobbyist sites than on big widely used sites. https://nickcraver.com/blog/2017/05/22/https-on-stack-overflow/ is informative.

– James_pic – 2019-10-28T11:28:01.087

@Alphad: Indeed 20 min to apply and verify that the application went through. I never tracked when the list actually got distributed to browsers - the perks of being a hobbyist... – Jonathan – 2019-10-28T12:08:42.680

@James_pic: Indeed. I did have some other troubles outside of this list - for example, getting WordPress to generate links correctly - the whole job was ~2 hours, and that with the certificate handled automatically by my hosting provider and Let's Encrypt. – Jonathan – 2019-10-28T12:10:27.717

2@Jonathan A big problem for domains with large number of hostnames and subdomains is that the preload feature requires includeSubdomains to be enabled. It's usually quite easy to enable HSTS on the www-subdomain but suddenly having it enabled everywhere is risky, especially since it's irreversible. Making an inventory and HTTPS-enabling every service with no way back is a lot of work for many enterprises. – Martijn Heemels – 2019-10-28T12:37:00.263

6

I have noticed there are still a couple of websites around which have https support but do not redirect http traffic to https. The extension isn't nearly as useful as it used to be however. A few years back websites like youtube, wikipedia and reddit had https support but defaulted to http. HTTPS everywhere solved that and is still solving the problem for the small handful of websites that still default to http but have https support.

Qwertie

Posted 2019-10-27T02:06:25.153

Reputation: 713

0

This will certainly come over as controversial, but nevertheless it's how I see it...

There is a bit of a misunderstanding in the necessity of HTTPS which was most probably spread deliberately. Like with every half-truth, there is some truth in the arguments, but also a whole lot of lies.

HTTPS (or TLS) does have some very useful and desirable properties (authentication and confidentiality) which are absolutely mandatory for some (think banking), and arguably necessary for quite a few, maybe the majority of, services. Anything that contains personal identification data, basically.
That being the case, there are many things for which HTTPS is utterly unnecessary, and on the other hand, HTTPS used improperly, i.e. with mixed content, can be pretty insecure (almost like no HTTPS), which was the justification for HTTPS everywhere in the first place. And yes, in the light of some sites back in the days that indeed offered a mandatory HTTPS kind of service with mixed content, it certainly did have some merit.

That, and then there's of course a good amount of paranoia that some people have and that is being actively promoted, about the whole world actually being interested in every little, unimportant thing in the unimportant little lives of everybody. Sure enough, after posting everything you've done today (with photos and geotag!) on Instagram where literally the whole world can read it, good job at having done so securely, via an encrypted channel. Also, it's important that nobody finds out about what you do on the internet in general. That, and there's this conspiracy where they alter news articles and feed you with false information to, uh... I don't know what for (actually, there is some truth in that, too, because that is just what e.g. Google does -- only at a different level, what's being changed is not the actual contents, but which content you're being shown at all, but that's irrespective of HTTPS being used). The silver bullet HTTPS prevents all these bad things! So clearly, everything needs to be HTTPS/TLS.

Regardless, even when used properly, HTTPS still fails to provide the service that you wish. For one reason, the entire chain of certificates works on the assumption that you can "trust" someone (say, Comodo) who makes money from selling certificates, without actually having a reason to trust them. And then, not just governments but also large enterprises (and schools, and antivirus, and who knows who else...) actively subvert the certificate chain by installing root certificates for the only purpose of, well, effectively breaking the system.
So, no, communications are not guaranteed to be confidential, and no, they are not authenticated in a reliable way. Not as much as you would think, anyway. Using your employer's laptop? Using your kid's computer? Lost cause. Antivirus installed on your computer? All bets are open.
But at least you know that a site is safe, the green thingie on your browser tells you so, and it warns about risky sites. Right, everybody can get a non-green certificate for free (avoiding the scary warning), and a green-badge certificate for very little money. It has absolutely no meaning.
I seriously hope you have TLS enabled for accessing your gmail account, too. Because, you know, that makes it secure, you don't want someone on the wire to read your mails, do you. Sure enough, Google will not read your entirely unencrypted mails while they are stored on their server. Sure enough, being a US company, they will not provide the contents to a particular governmental organization.

Now the real reason why you must have HTTPS everywhere is that companies like Google, Microsoft, or Amazon, and with them all providers who sell bandwidth, want that.

They do not want everybody and their grandmother to set up a credit-card computer as a transparent web proxy which not only reduces your bandwidth consumption by caching resources, but also filters out their advertizing and tracking stuff. Sure, you can always add a browser plugin which does the same thing. Except, you must maintain it on every computer that you have in your house, and on some (Fire TV) it is outright impossible without rendering the device unusable, or you must root it (think Android phone) which also isn't necessarily destruction-free (thank you so much for Samsung Knox, so awesome).
Luckily, you can just cut the crap globally, for all devices within your network by having a transparent proxy right behind your cable/dsl modem, which costs you 20€ and 3 mins setup. Oh heck, what a catastrophe! You are to download exactly the version they want (including "personalization"), and when they want it, including all beacons and whatnot. So that is the true reason why you need HTTPS everywhere.

Ironically, the companies that promoted HTTPS and emphasized how e.g. TLS not only hides the actual content, but also the exact URL that you clicked on (like http://somesite.com/dirty_porn_pic.jpg) and such... in reality they are exactly the ones who go to any length to fingerprint your system, identify you, keep an infinite history, track every single click you do, and collect every possible piece of information including where you go and when, and your heartbeat. Or, the contents of any file on your computer. Ever wondered how Amazon does it so they incidentially recommend XYZ on your PC after you searched Google for XYZ on your phone five mins earlier? Different company, different device, one supposedly cannot possibly know both devices are owned by the same person. I did in fact wonder how it's done, since in my understanding, whatever they need to do to achieve that certainly is not compliant with the law (in the EU at least). But apparently, that's not a hindrance.

HTTPS actually helps in doing all these borderline-legitimate things, both by providing a false sense of security, by obscuring what is being sent, and by no longer making people ask: "Hey, what's that encrypted traffic coming from my device anyway!?". Because, you know, all traffic should be encrypted, that's a good thing. Encrypted stuff is not suspicious, it's probably harmless. Nobody is hiding something.

Damon

Posted 2019-10-27T02:06:25.153

Reputation: 4 002

Thanks for weighing in. I'm pretty unfamiliar with all this stuff, which is why I posted the question in the first place. Much of what's in this answer just went over my head. Also, there's liberal use of sarcasm, which I can appreciate when I understand it, but it makes things confusing when I don't know the subject. Any chance you could add a "for dummies" summary with some context/explanation of the key points? – fixer1234 – 2019-10-29T17:09:23.503

1

@fixer1234 I wouldn't worry about this answer too much. It kind of reads like they believe you may be wearing a tin foil hat... or even that HTTPS itself is some sort of tin foil hat. Security and privacy is more important than this answer suggests, even if you "have nothing to hide." ... tl;dr All those "conspiracies" about government and hackers spying are false or overblown, and using HTTPS makes your traffic look suspect (so, don't encrypt anything at all?)

– phyrfox – 2019-10-29T23:52:29.523

@phyrfox: On the contrary, the tinfoil hat is explicitly being put on your head (both by actual tinfoil-hat wearers, and people who have a financial interest). While encryption (and authentication) is good and necessary where it's necessary, it is by no means necessary when reading an "ordinary" website. It isn't even desirable (desirable for you) because it prevents easy caching/reusing of content. Of course, caching is undesirable for other parties (e.g. content providers) since you are to get your personalized content, which is an awful euphemism for something very, very bad. – Damon – 2019-10-30T12:02:42.420

It's a real thing. People who speak of "fake news" are usually ridiculed, but if you are being honest, this is, sadly, exactly what today's "personalized content" world looks like. While the news per se are not fake (well, sometimes they are), the selection that you are being shown differs (try loading the same news page together with a coworker for example, you'll be surprised). Information should be truthful, complete, and unbiased, but it's not. Encryption helps doing that because content is "automatically" individual. – Damon – 2019-10-30T12:11:57.913

0

I always thought HTTPS Everywhere was developed to prevent SSL-strip attacks, but maybe this is only a side effect. SSL-strip is still a problem though and with HTTPS Everywhere you can prevent it.

If an attacker can trick the user into using HTTP for the first request, (s)he can intercept the communication, use HTTPS to contact the server, modify the response and return it to the user. It is e.g. possible to modify all links in the result so they do not use SSL anymore, or they can be rewritten to contact an HTTPS url which is under control of the attacker.

This is where HTTPS-everywhere jumps in, this first HTTP request would be executed as HTTPS, so an attacker has no chance to intercept the traffic.

martinstoeckli

Posted 2019-10-27T02:06:25.153

Reputation: 189