1

Browsers and standards bodies favor HSTS with Preload because it avoids ever sending an http request to a website that supports https. This is good, because cleartext http requests can be intercepted to set up Man in The Middle attacks.

But a number of websites explain that a centralized Preload list doesn't scale up well to the mostly https web that has been proposed by W3C, the EFF, and others. Managing one centralized list creates a bottleneck for looking up, adding, and deleting list items.

Yet this technology has been implemented rather than, say, to use DNS, which is already nicely distributed and is already used by browsers to lookup URL domain names.

Of course, the DNS is not yet secure, and proposals to make it secure are controversial. But why would the DNS have to be secure to hold one more bit of information (whether the domain can support https--and ONLY https--or not)?

In the worst case, a malicious MiTM attack could make it seem that a website is insecure when it is actually secure. But in this case, an insecure connection would simply fail. This failure would deny the malicious user any advantage.

So naturally I'm wondering why a centralized HSTS with Preload is preferred over adding a new flag to DNS zones for indicating that the domain supports https connections.

David Spector
  • 99
  • 6
  • 1
    You are assuming that the site is fully HTTPS on all pages of the site. – schroeder Dec 27 '19 at 23:14
  • 1
    "adding a new flag to DNS zones" is no small ask. I think what you are looking for is a new default in browsers that always default to HTTPS but alert when it is http. That does not require a "preload list" and can be implemented easy without international agreement immediately. – schroeder Dec 27 '19 at 23:15
  • 1
    Preload is not meant to encompass the entire Internet. It is meant to be a stop-gap until a tipping point is reached for a critical mass of domains and sites. – schroeder Dec 27 '19 at 23:17
  • Schroeder: Yes, and if a website designer references http pages, this will have to be changed. This is stronger security than HSTS, which allows subdomains, images, etc., to be http requests. Is that why browsers prefer HSTS? – David Spector Dec 27 '19 at 23:17
  • 1
    Your question is ill-formed. Why doe browser prefer HSTS? ***Because the thing you want in DNS does not exist yet*** – schroeder Dec 27 '19 at 23:18
  • Schroeder: no: alerting when a site is http is for the future, when we have an all https web. But even then, there will be old links around, and there is the IoT, which require the cheap internet access of http. Preload is a transition strategy, but so would be using a new DNS flag. – David Spector Dec 27 '19 at 23:20
  • Schroeder, your objections so far are all ill-founded. Lots of new features have been added to DNS, such as DKIM. Such features work nicely. An "https" flag would similarly work nicely. Yes, there would be some work to add it, but why would that work not be preferable to the existing security problems? – David Spector Dec 27 '19 at 23:22
  • 1
    Your logic and your approach do not make sense. DKIM was not a small ask. And I never said that it was impossible or wouldn't work. I'm saying that with enough work and international collaboration, something like this might work (if secured). The "work" is simply to default to HTTPS. Browsers can do that right now. I don't see a benefit for DNS to take this on now, and there are dependencies to work out first. – schroeder Dec 27 '19 at 23:30
  • Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/102622/discussion-between-david-spector-and-schroeder). – David Spector Dec 28 '19 at 02:37
  • What you are describing sounds similar to DANE, where certificates are bound to DNS names using DNSSEC. See https://tools.ietf.org/html/rfc7671. I'm not sure if it's part of the spec, but it would seem that If the browser finds a DANE record, then it can assume that the host supports https, and therefore should only attempt to connect by https. – mti2935 Dec 29 '19 at 12:58
  • See the dialog between Steffen Ullrich and I, following Steffen Ullrich's answer at https://security.stackexchange.com/questions/179122/is-dane-the-dns-variant-of-http-public-key-pinning-hpkp/179136?noredirect=1#comment455806_179136 for more info on how this might work. – mti2935 Dec 30 '19 at 03:45
  • @mti2935 Sounds very good. One of the biggest hurdles to the adoption of https is the fact that tools (server, ftp, shell) that need to be secure need to have two local files for each domain (certificate and private key), and need to be linked to these files. Putting security into DNS might eliminate this need. There still is the problem of authentication--how do we do secure lookups that are guaranteed not to be available to malicious users? – David Spector Dec 30 '19 at 15:28
  • DANE: "This document improves on that situation by enabling the administrators of domain names to specify the keys used in that domain's TLS server." Of course, the obvious problem with this is that malicious users administer their own domain names using malicious domain registrars and malicious IP allocators. I hope there is a solution. Meanwhile, I hope my question here is a contribution in some way. HSTS won't work in the long run as it exists now. – David Spector Dec 30 '19 at 15:33
  • David Spector, Interesting thoughts. With regard to the problem where 'malicious users administer their own domain names using malicious domain registrars and malicious IP allocators', I think what you are talking about here is phishing sites and the like. I don't think this is a problem that DANE aims to solve, but protocols like PAKE and SRP (where the client and the server mutually authenticate each other) would be useful here, although adoption has been slow. – mti2935 Dec 30 '19 at 15:40
  • I think there are two problems with PAKE and SRP. The first is that they are only methods for two or more parties to establish cryptographic keys based on one or more party's knowledge of a password. That doesn't solve malicious IP allocators (example: large blocks of IP addresses are available to malicious users in certain countries), etc. The second problem is that exchanging a random secret takes a significant amount of time, unless done on specialized hardware. In this question I am asking about HSTS, which only provides secure connections. – David Spector Dec 31 '19 at 16:25
  • Preventing access to the Internet to malicious users requires a very different kind of solution, and might require laws and punishments in addition to technical solutions. For example, suppose everyone had to register (once only) to use the Internet. The advantages are obvious, but the disadvantages are difficult and many. – David Spector Dec 31 '19 at 16:25

2 Answers2

3

But why would the DNS have to be secure to hold one more bit of information (whether the domain can support https--and ONLY https--or not)?

A man-in-the-middle attack could modify the DNS responses to remove the bit saying that a site requires HTTPS.

In the worst case, a malicious MiTM attack could make it seem that a website is insecure when it is actually secure. But in this case, an insecure connection would simply fail. This failure would deny the malicious user any advantage.

A man-in-the-middle attack could make the insecure requests work.

Say paypal.com requires HTTPS connections (and doesn't respond to HTTP connections, or responds only with a redirect to https://paypal.com), and has the HTTPS-only DNS flag set. If a user is MITM-attacked, then the MITM can strip the HTTPS-only DNS flag from the DNS response for paypal.com, and then when the user makes an insecure HTTP connection to paypal.com, the MITM can proxy that connection to https://paypal.com and see all the requests and responses in plaintext. The user will believe they're talking with paypal.com over HTTP, and paypal.com will believe they're talking to the user over HTTPS.

Macil
  • 1,482
  • 9
  • 11
  • I think there is an error in your hypothetical case. You say, "Say paypal.com requires HTTPS connections (and doesn't respond to HTTP connections, or responds only with a redirect to https://paypal.com)". But these two situations are **very** different. An https-only site under the DNS idea would NEVER respond to an http connection, and would NEVER redirect anywhere. It would not even Listen for an http connection. This prevents MiTM attacks. – David Spector Dec 27 '19 at 23:46
  • 2
    @DavidSpector You're right about one part - a truly HTTPS only site wouldn't even bother returning the redirect. However, this wouldn't protect against a MitM attack. The point is that the MitM intercepts the HTTP request from the user and then sends an identical HTTPS request to paypal, returning the response similarly. As a result paypal only ever sees an HTTPS client, the user doesn't realize that they are communicating over insecure HTTP, and the attacker gets full control over the paypal session. – Conor Mancone Dec 28 '19 at 00:06
  • 1
    @DavidSpector It doesn't matter that paypal.com doesn't respond to HTTP connections, because if the user makes a request for http:// paypal.com, the MITM could make a request to https:// paypal.com on the user's behalf and then give that response to the user. – Macil Dec 28 '19 at 00:06
  • Macil, I think you are wrong. If the user's request for http://paypal.com is never answered (because paypal.com has no listener for port 80), then the malicious user can never see paypal's response. Therefore, there can be no Mit – David Spector Dec 28 '19 at 02:35
  • 4
    @DavidSpector The whole point of a MitM is that they are in the middle and can intercept all traffic. The request doesn't have to actually go to PayPal for the MitM to intercept it. In fact, the whole point is for the MitM to stop it from getting there in the first place – Conor Mancone Dec 28 '19 at 04:16
  • @Conor But if the browser NEVER makes an http request on the part of the user, then a malicious user CANNOT ever intercept the traffic from the user. Do you agree? The whole point of the HSTS and DNS strategies that I described (which are enforced by browsers) is to prevent ANY http requests for https-only websites other than for looking up in the DNS, which is harmless (while still permitting users to enter http URLs to visit websites). – David Spector Dec 28 '19 at 12:06
  • 4
    @DavidSpector 1. As you have mentioned in the original post, the attacker can alter the DNS flag, so the client tries to connect through plain HTTP. 2. Although the actual website does not respond to HTTP (and maybe does not even listen on 80), the attacker can. – v6ak Dec 28 '19 at 12:20
  • 1
    @DavidSpector I'm just repeating voak, who is saying the same thing as everyone else here. The DNS request is over plain text and can be intercepted/modified/faked easily by a MitM. As a result someone who is in a position to intercept and HTTP request is also in a place to intercept a DNS request. Therefore the attacker can modify the DNS response to make the browser thing HTTP is okay, and then intercept the resultant HTTP request, performing a successful MitM attack. – Conor Mancone Dec 28 '19 at 12:29
  • Conor and v6ak: Oh, I see your point. So DNS must be made secure before the Web will be truly secure, and I've been told that that is controversial (possibly because of old links, IoT, etc.). The only solution using DNS would require both an insecure and a secure DNS during the transition, requiring TWO requests to lookup an insecure URL. I guess people felt that adding an additional request for insecure DNS lookup was an unacceptable overhead. But there is still the Preload list problem! It won't scale up. Two DNSs, while not trivial to implement, would solve that . – David Spector Dec 29 '19 at 11:32
  • @DavidSpector If you have two DNSes, how would you use the insecure one? Just having a single and secure DNS would be enough. However, it is not always available, for example in case of a Wi-Fi captive portals. If you fall back to plain old insecure DNS in case of failure when connecting to DNSSEC, you essentially allow a downgrade attack. – v6ak Jan 02 '20 at 23:07
  • BTW, please use @nick when replying. Without that, it does not create any notification. (Also, I am “v6ak”, not “voak”.) – v6ak Jan 02 '20 at 23:08
  • @v6ak Thank you for reminding me to use names to create notifications. There must be two separate DNSs, if a DNS (distributed) design is to work at all. I think you will agree with me that an insecure Web is intolerable, yet must be tolerated due to many factors, such as existing insecure websites, insecure links, and cheap devices that cannot afford security. So, yes, downgrading is a perilous but necessary step if we are to have a mixed Web. If we can assume that secure DNS servers will never be down (due to redundant servers), then it is safe. – David Spector Jan 03 '20 at 19:15
  • The only alternative I can think of is to have two entirely separate Webs, with the user responsible for knowing which to use. But this is the same overhead as the Secure Web that W3C and others want: the user has to use the proper protocol. While that sounds elegant, the Web is used by real people who simply cannot be burdened with such a task. Anyway, so long as there are insecure websites, people will try http if https fails, exposing them to malicious attacks. Real solutions are difficult, but HSTS simply won't scale up, and the DNS solution isn't perfect. – David Spector Jan 03 '20 at 19:20
  • Actually I can think of an alternative: the draconian approach of an appropriate agency declaring that after a certain date and time only https will be supported anywhere. The perfect solution. Except that it would create absolute chaos. – David Spector Jan 03 '20 at 19:22
  • @DavidSpector Mixed web and downgrade attacks are two separate things. HSTS prevents downgrade attacks under some realistic conditions. HSTS preload list make the conditions better achievable, although it does not scale. However, replacing HSTS by DNS and falling back to insecure DNS at the same time can hardly prevent a downgrade attack in typical scenarios. – v6ak Jan 03 '20 at 19:43
  • @DavidSpector Of course, it depends what scenarios you consider. I would typically care mostly about attacks at the “last mile”. If one can perform downgrade attack on HTTP, such attacker will likely be also able to perform a downgrade attack on DNS and to remove the secure flag. – v6ak Jan 03 '20 at 19:47
  • All quite true. As I said, there is no perfection possible in a mixed web. Are you ready to demand that all Web communications be secure, under all circumstances? Think of the burden on users and manufacturers alike. Because if you wish a transition period, what I've said is likely the only way. Or you can simply focus on the fact that so long as http is supported, there will always be attacks possible. Such a negative view promotes the current unsatisfactory situation and prevents a quick transition to the secure Web. It's your choice. – David Spector Jan 04 '20 at 20:14
  • @DavidSpector But switching from HSTS to your vision is not a step forward. It is a step backwards. Even pure HSTS (without HSTS preload) can guarantee TOFU security for websites you visit often enough. (TOFU = Trust On First Use) Your suggestion to use DNS without any effective protection against modification does not guarantee even TOFU. It guarantees nothing in usual conditions. – v6ak Jan 08 '20 at 00:10
  • I did not ever suggest using DNS without adding a secure DNS, if I understand your objection correctly. Also, please give an example of how the DNS solution fails to work in any conditions. – David Spector Jan 08 '20 at 16:30
0

Not a real full answer (because only browsers vendors could probably really answer you), but some perspective.

For whatever reasons there seems to always have been a "gap" (for whatever other name you can call it, my English is not sufficient to find the proper term here) between the "Web people" and the "DNS people".

It seems there always have been some misunderstandings between the two, which shows in multiple areas and produced strange outcomes.

For example, browsers vendors never wanted to use the DNS SRV records, while they are useful for many needs, and are used by other protocols. This among other things exacerbated the problem about "CNAME at apex", which is back on the drawing table this time with drafts specifically written/supported by the browsers people or some of them, to create SVCB/HTTPSSVC records. This may explain as well why DANE (TLSA records that basically allow to encode in the DNS some information about which certificates or certificates authority a given service use, in order for the client to check) is today more something existing in the SMTP world than the web world (but the Web PKI as it stands today is a big ecosystem that is difficult to move... look at what was needed to have something like Let's Encrypt, or how slowly things move at the CABForum to obsolete all algorithms or introduce new ones for example).

There is at least one actual technical fact that creates an impedance mismatch that remains open: from technical records in the DNS, it is difficult to defined administrative boundaries, and yet you need them for things like cookies related permissions. The best, but not perfect, solution for this right now is the Mozilla curated "Public Suffix List", and there were former attempts, like IETF DBOUND working group, that failed to produce any positive outcomes.

I guess the same issue could be at play in "HSTS preload vs data in the DNS" as any list such as the HSTS preloading is typically a list of hostnames, and then you have the problem of the "level". If example.com is preloaded, should other.example.com be too? And the opposite? How does it depend on the TLD? On the 2LD? Etc. If you have an exhaustive list, you do not have the issue (as you decide that everything is protected below and starting at one level; Google for example prelisted their newer TLDs like APP, DEV and NEW to the surprise of many domain name buyers not reading the fine print telling them that their website in those TLDs will not work on HTTP and hence that certificates are mandatory to acquire to have HTTPS). If instead you want to have records in the DNS for that, you need to define how records set at each level impacts those at level below and are themselves impacted at level above. This is one of the complex part that made endeavors like DBOUND fail.

There is also another pervasive issue at play here. For anything above to be useful in the DNS, you need DNSSEC. Otherwise you are absolutely vulnerable to a MiTM. But for both technical and non technical reasons DNSSEC is not deployed at 100% level, and never will but yet browsers want a solution for all cases.

Added to that there is an ongoing, sometimes veiled, belief by some people that if you have TLS, then you do not need DNSSEC (besides also those opposing DNSSEC per se, due to its complexity, the imperfect balance between benefits and drawbacks, etc.). This could be true on the paper, but certainly is not in practice as long as "most" TLS handshakes are authenticated by DV-issued certificates, and that BGP is not secured against hijacks. In fact, it is even worse now in the face of automated CAs.

You can see a huge example of this dichotomy with the recent DNS over TLS and even more so with DNS over HTTPS technologies, with some browsers having even decided to force DoH upon their users. This has only started to spark huge controversies and debates.

Note for example, that Mozilla "Trusted Recursive Resolver" program at https://wiki.mozilla.org/Security/DOH-resolver-policy does not require prospective applicants to be DNSSEC enabled. This speaks a lot. And of course, using a specific name in the DNS as canary (https://use-application-dns.net/) needs explicitly to NOT use DNSSEC so giving this signal to end-users: "DNSSEC is an important technology - supposedly - to protect your domains... but when it comes around the important part of configuring the DNS subsystem we have to use domain names explicitly without DNSSEC for things to work out". I can understand end users becoming completely confused there.

You can also count that nowadays both the web and the DNS are complex technologies reaching many parts and being discussed/specified in many documents and forums. It is probably mostly impossible for someone to be an expert in both at the same time and for a long time, which means there is unfortunately less opportunity for bridges and smart ideas that use the advantages of one side to cover the disadvantages of the other one and the opposite. Recently this was shown at the DNS side by the now famous "DNS camel".

Another issue coming to play is around "agility":

  • there are only a few major browsers vendors, and they kind of control all their clients (browsers installation) because the clients auto-update themselves (there are counter examples of course, many sites still have to battle around having to support old Internet Explorer versions for example)
  • while they are also only a few major DNS software implementations, they control far less their installations (both authoritative and recursive nameservers) which means some features may take a very long time to be actually deployed (and DNSSEC was in that case too), alongside all obsolete features that needs to be kept around (but recent "DNS flag days" options are trying to cut down this part). Nameservers software is typically not "auto-updated", or at least not without some careful monitoring and handling by a sysadmin, while browsers are automatically updated with many users not even being aware of changes (which is also why changes like "let us enable DoH automatically and by force" creep through and are discovered with anger)

So for the side (the web) that wants fast turnaround and features being delivered to their users, they may not want to rely on DNS features as those would take far longer to deploy, probably. All of this is of course highly subjective.

So I think some of the above can explain why the "web world" decided to go forward with a solution completely internal to their world (the HSTS preloading list), in order not t

Is this a good idea or not? Everyone can discuss and argue about that. At least there is both a scale problem (having an exhaustive list), a maintenance problem (updates of the list), and a delivery problem (including the list in browser software code). But at the same time, like explained above, there is no competing "equivalent" solution at the DNS level so it is not like you have two choices more or less at same stage (each one with its own benefits and drawbacks).

Like HPKP (HTTP Public Key Pinning) was the reference in the past but become obsolete, maybe once CT (Certificate Transparency) Logs really become mandatory everywhere for every case, a browser can switch to a simpler algorithm such as "if that hostname is covered by a certificate, as shown in CT Logs, then it means HTTPS only" and then HSTS Preloading is not needed anymore. Of course, this simple sentence have various technical consequences that are harder to solve than just the time to write the sentence.

Or you just decide that HTTP is dead and should never be used anymore and hence it is 100% HTTPS always for everyone, no need to maintain a list.

Browsers seem also to go towards that direction...

Patrick Mevzek
  • 1,748
  • 2
  • 10
  • 23
  • Patrick, I love your posting. You mention so many interesting and important issues all in one place. The depth of your understanding goes way beyond mine. From my limited point of view I can only say that the fundamental and obvious solution to most of these problems is simply to convert MOST of the Web (not all) to HTTPS. There will always be a need for HTTP for low-cost and home-brew devices, and as a transition strategy. But I believe a date should be set for HTTPS to be the default, with HTTP being an isolated exception. Look at the success of Let's Encrypt! The whole thing can be done. – David Spector Feb 29 '20 at 20:46
  • If it is not all HTTPS then you are vulnerable (hence HPKP, HSTS, etc.): if an attacker on the path is able to filter port 443 a browser may fall back to 80 if the browser is not instructed to never do this. Also, thanks for your kind words. – Patrick Mevzek Mar 05 '20 at 05:36
  • Patrick, yes. This is very important for standards organizations to understand. If HTTP is to continue, either for transition or for inexpensive devices or for any other reason, the HTTP world has to be isolated carefully, free of fallbacks. HTTP must be the exception only, and specially registered somehow. – David Spector Mar 06 '20 at 12:13