1

Maybe I'm just asking for the correct search terms.

mod_nss and the NSS information/design idea is that if the key storage device (in this case, an HSM) is capable of performing a DH/SSL handshake, then the device will be used (the key is not extracted from the HSM, and then used on the host for the handshake, rather it is left on the HSM, and the HSM is used to perform the handshake). The idea is that if you are paranoid (or regulated) enough that you need to use an HSM, you won't want your keys extracted into the host, as that exposes them to compromise.

HSM managed handshake too slow

The issue is the duration of the handshake. Even without a PCIe, direct-attached HSM, the length of time needed for a single TLS handshake is measurable. If you are suddenly trying to use an HSM for the handshakes, you are limited by the PCIe bottleneck and all the overheads involved -- HSMs are generally built for security, not for high-performance.

The HSM becomes a bottleneck for services that support "enough" connections/second, "enough" being defined as "sufficient to swamp the HSM".

To trade off between the security of the HSM and the performance of host-based TLS hand-shakes (which can benefit from multiple cores, etc), we need to store the key on the HSM, but use the key on the host.

NSS' worldview prevents this.

Question: Can I extract the private key?

Can OpenSSL or other crypto suite, which can be connected to an HSM, extract the key from the HSM, and cache it for use on the host?

Plan B

Plan B is to to get the (OpenSSL) OpenSC+OpenP11 engine or NSS source code, and hack it to extract and cache the private key on first request, and then just call back into openssl to do the actual TLS handshake rather than go out to the HSM for every new connection, allowing us to take advantage of the current (working) environment for https and ssh connections, while providing for security around the stored keys. But I'd rather take advantage of existing code.

Search engines just take me to the same pages, some relevant (the NSS design philosophy for example), some not. If you know of a good set of search terms that brings me to a page you know about :) that would be most help.

Thank you for your time.

Community
  • 1
rip...
  • 251
  • 1
  • 8

2 Answers2

2

When using an HSM with TLS all cryptographic operations involving the private key are executed on the HSM. This means the private key never leaves the HSM. In fact, the HSM is specifically designed to protect exfiltration of the private key using software based attacks and sometimes even hardware based attacks (i.e. tamper resistant). This exactly is the reason a HSM is used at all.

This means that an HSM would not be worth its money if it would be possible to extract the private key using OpenSSL or other tools or even by using specifically designed hacking tools. Thus the answer to your question for all existing HSM is hopefully: No, it is not possible to extract the private key using software.

Which means: if you need both more performance and the security of an HSM for every TLS connection then get a faster HSM.

grochmal
  • 5,677
  • 2
  • 19
  • 30
Steffen Ullrich
  • 184,332
  • 29
  • 363
  • 424
  • Gewiss! At the moment I work for an HSM vendor... but "the customer is always right" implies that I have to at least LOOK for an answer that solves the problem. In this case, they have a system that will need to support (claims not verified) 10k TLS handshakes /per second/. Since a TLS handshake takes approx 75mSec, give or take, that is ... what, 13 /minutes/ of work generated each second? They can do it with commodity servers, but /not/ when they use an HSM. Thus the: wrap the key, extract it from the HSM, and unwrap it only as part of the actual handshake on the commodity servers... – rip... Apr 27 '17 at 00:13
  • 1
    @rip...: again, a HSM is there to protect the private key and to not make it accessible from outside in any case. If a software would be able to retrieve the key from the outside and operate on the key just if the key would reside on the disk or in memory then system might be vulnerable to key exfiltration attacks - either malicious due to direct attacks on the system or due to bugs like [Heartbleed](https://en.wikipedia.org/wiki/Heartbleed). If they want huge performance and no HSM provides it then they simply cannot use a HSM but have to address the risk in a different way. – Steffen Ullrich Apr 27 '17 at 04:07
  • @rip...: and the customer is not always right. If the customer wants to buy a car which can transport as much as a truck, is fast as a race car, is economical like a bike and is cheap like a newspaper then there is simply no such thing and the customers needs to change its expectations. But maybe the customer is not explicitly looking for a HSM but only looks for a solution to protect the private key while keeping a big performance - in which case you might try to address the problem of the customer without HSM instead of focusing too much on selling what you have. – Steffen Ullrich Apr 27 '17 at 04:12
  • Yes to all of the above. But, whether a key can be extracted for use elsewhere depends on corporate Policy, and should be based on a clear understanding of the risks vs the requirements. They currently support the throughput they are asking for, but they are being asked by /their/ customers to move the tls handshakes into an HSM. In order to keep using the system that works now, they will need to convince others to let them do so, by providing benchmarks and investigatory/empirical data that shows that no HSM can provide what they need, barring a complete rewrite of the arch... – rip... Apr 27 '17 at 04:21
  • ...and a complete rewrite of the architecture is liable to cost as much as the 200+ high-end HSMs they would need to meet the requirements. – rip... Apr 27 '17 at 04:25
  • 1
    @rip...: If the customer really requires a HSM then he expects the security which comes with a HSM, i.e. that the private key never leaves the HSM. Maybe they could address the customers requirements by moving exactly this customer to a separate system which can handle the load of this single customer with a HSM. Or maybe they could explain to the customer that a real HSM is too costly or does not has the performance needed but that there might be other ways apart from a HSM to address the risk of exposing the private key. – Steffen Ullrich Apr 27 '17 at 04:30
  • 2
    @rip...: I think you and maybe your customer and their customer simply intervene the risk they want to address (exposing the private key) with the only risk mitigation they've heard of, i.e. a HSM. But there are other ways to mitigate this risk, for example by moving the essential parts to a different system which might not be tamper resistant against hardware attacks like a HSM but provides way more performance. Have a look at [Cloudflare keyless SSL](https://www.cloudflare.com/ssl/keyless-ssl/) for example. – Steffen Ullrich Apr 27 '17 at 04:36
  • I'm accepting this answer, but that might be simply confirmation bias :) I've just looked at Cloudflare, and I think that given the vertical I'm dealing with (see comment elsewhere re: 'regulatory' requirements, that will be a non-starter. – rip... Apr 28 '17 at 15:04
0

Yes, maybe.

Yes, you may be able to use/abuse (you choose what word fits best) the BACKUP feature of an HSM to extract the plain text version of a private key.

At least in cases where the backup/restore procedure is documented to allow this.

And at least in the case of the "Nitrokey HSM"/"SmartCard-HSM" this is in fact possible because everything about it is open source and there is no hardware to stop you from using/abusing that backup/restore feature.

Details here:

But...

Let me says this in summary: while this may be possible technically, it goes against the very reason of existence of an HSM. Namely: What's inside, STAYS inside! -- What you're trying to do sounds sketchy and dangerous. -- I agree with what Steffen said: If your HSM is too slow: get a bigger HSM. Either as a box, or as a service such as Cloudflare's "Keyless SSL". (Or get rid of your HSM altogether.)

Community
  • 1
StackzOfZtuff
  • 17,783
  • 1
  • 50
  • 86
  • 'Get rid of HSM' isn't an option (Regulatory). The HSM in use is protected against the nitrokey usage, key backups are assured using an internal backup key, and are only usable on HSM clusters that share that backup key. "A bigger HSM" in this use case would be 300x faster than the current speed demon in the HSM world (HSMs design goal of security rather than speed). I will look at CloudFlare, and if it makes sense, recommend it to the customer (yes, customer success is more important than the sale for this use case, because there are always other use cases that /can/ use an HSM). – rip... Apr 28 '17 at 14:46
  • @rip...: Can you name the exact regulatory requirements? Do you just need the HSM so you can tick the "Must have an HSM" box on some form? *Having* an HSM but then *extracting* the key and doing crypto off-HSM sounds like treating the HSM just like a very expensive USB stick. The security gains of an HSM (namely: protection even against a compromised root user) are lost. You get the same protection from just running full hard disk encryption. Also if you're the rightful owner of the cert in the first place then the whole extraction idea is a red herring anyway. You could just get another cert. – StackzOfZtuff Apr 29 '17 at 05:54
  • I'm pretty sure they (the upstream clients of the service) will be under either the various PCI (not PCIe :) ) requirements or CC. There is a lot of check-boxery in PCI, but if they are also looking at CC, that's gonna be a bear. – rip... Apr 29 '17 at 22:42