5
The rsautl OpenSSL utility does not appear to have a setting wherein the hash for OAEP can be changed. My testing suggests that it just defaults to sha1 and that's it.
My question is... is there a way to change the hash?
The command I used to test it out:
openssl rsautl -inkey publickey.txt -encrypt -oaep -in plaintext.txt -out ciphertext.txt
neubert
Posted 2015-12-21T13:50:44.237
Reputation: 3 991
11
rsautl which is ancient, no. pkeyutl which was supposed to supersede rsautl in 2010, yes. -pkeyopt rsa_oaep_md:name or rsa_mgf1_md:name where name is a hash as in your Q on PSS.
Note for pkeyutl (edit) before 1.1.0 the order of options matters; you must do -encrypt then -inkey rsaprivate then -pkeyopt (not currently documented AFAICS, though the analogous cases in genpkey are) and you need to do rsa_padding_mode:oaep before rsa_{oaep,mgf1}_md:hash .
Same possible issue with FIPS mode.
dave_thompson_085
Posted 2015-12-21T13:50:44.237
Reputation: 1 962
2I was trying this just now (maybe I tried it earlier too idk) and I had to do
-pkeyoptbefore each option. eg.openssl pkeyutl -in ciphertext.txt -decrypt -inkey test.pem -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256 -pkeyopt rsa_mgf1_md:sha1. So in that case I needed to do it once for each option - not once globally. – neubert – 2016-09-17T05:06:34.9201@neubert: yes you need
-pkeyoptfor each key-option; I thought that part was clear from the manpage and didn't repeat it. But the requirements on order were (and are) not all documented AFAICS, and such requirements are unusual foropenssl, so I called them out. – dave_thompson_085 – 2016-09-19T06:16:58.8001The ordering doesn't appear to matter with OpenSSL 1.1.0f, but I couldn't find these options documented. Although
-pkeyopt rsa_oaep_md:sha256works, it is not mentioned onman pkeyutl. – starfry – 2017-06-23T09:04:53.223@starfry: (don't know why I wasn't notified at the time) yes, 1.1.0, released 8 months after I wrote this answer, completely reimplemented option handling for all commandline operations; edited for anyone else who comes along. It's
rsa_mgf1_mdwhich applies to both OAEP and PSS -- and is documented in 1.1.1 which turnaboutly postdates your comment :-) – dave_thompson_085 – 2019-02-05T02:25:47.197