2

I'm trying to forward logs from rsyslog to graylog over tls.

rsyslog configuration:

# make gtls driver the default
$DefaultNetstreamDriver gtls
#
# # certificate files
$DefaultNetstreamDriverCAFile /etc/ssl/rsyslog/ca.pem
$DefaultNetstreamDriverCertFile /etc/ssl/rsyslog/rsyslog-cert.pem
$DefaultNetstreamDriverKeyFile /etc/ssl/rsyslog/rsyslog-key.pem
#
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer graylog.mydomain.com
$ActionSendStreamDriverMode 1 # run driver in TLS-only mode
*.* @@graylog.mydomain.com:6514 # forward everything to remote server

I generated the necessary certificates as described in the rsyslog documentation:

certtool --pkcs8 --generate-privkey --outfile ca-key.pem
certtool --pkcs8 --generate-self-signed --load-privkey ca-key.pem --outfile ca.pem

certtool --pkcs8 --generate-privkey --outfile graylog.key.pem 
certtool --pkcs8 --generate-request --load-privkey graylog.key.pem --outfile graylog.request.pem
certtool --pkcs8 --generate-certificate --load-request graylog.request.pem --outfile graylog.cert.pem --load-ca-certificate ca.pem --load-ca-privkey ca-key.pem

certtool --pkcs8 --generate-privkey --outfile rsyslog.key.pem 
certtool --pkcs8 --generate-request --load-privkey rsyslog.key.pem --outfile rsyslog.request.pem
certtool --pkcs8 --generate-certificate --load-request rsyslog.request.pem --outfile rsyslog.cert.pem --load-ca-certificate ca.pem --load-ca-privkey ca-key.pem

Graylog config

All certifcate files are in /etc/ssl/graylog/input-ca/

The graylog input is configured like this:

TLS cert file: /etc/ssl/graylog/input-ca/graylog-cert.pem
TLS private key file: /etc/ssl/graylog/input-ca/graylog-key.pem
TLS Client Auth Trusted Certs: /etc/ssl/graylog/input-ca

But when rsyslog pushes a log message to garylog I get this error:

2016-10-06T13:19:27.734+02:00 WARN  [AbstractNioSelector] Failed to initialize an accepted socket.
java.security.cert.CertificateParsingException: signed fields invalid
    at sun.security.x509.X509CertImpl.parse(X509CertImpl.java:1791) ~[?:1.8.0_91]
    at sun.security.x509.X509CertImpl.<init>(X509CertImpl.java:195) ~[?:1.8.0_91]
    at sun.security.provider.X509Factory.parseX509orPKCS7Cert(X509Factory.java:469) ~[?:1.8.0_91]
    at sun.security.provider.X509Factory.engineGenerateCertificates(X509Factory.java:354) ~[?:1.8.0_91]
    at java.security.cert.CertificateFactory.generateCertificates(CertificateFactory.java:462) ~[?:1.8.0_91]
    at org.graylog2.plugin.inputs.transports.util.KeyUtil.loadCertificates(KeyUtil.java:90) ~[graylog.jar:?]
    at org.graylog2.plugin.inputs.transports.util.KeyUtil.loadCertificates(KeyUtil.java:100) ~[graylog.jar:?]
    at org.graylog2.plugin.inputs.transports.util.KeyUtil.initTrustStore(KeyUtil.java:73) ~[graylog.jar:?]
    at org.graylog2.plugin.inputs.transports.AbstractTcpTransport$1.createSslEngine(AbstractTcpTransport.java:199) ~[graylog.jar:?]
    at org.graylog2.plugin.inputs.transports.AbstractTcpTransport$1.call(AbstractTcpTransport.java:186) ~[graylog.jar:?]
    at org.graylog2.plugin.inputs.transports.AbstractTcpTransport$1.call(AbstractTcpTransport.java:182) ~[graylog.jar:?]
    at org.graylog2.plugin.inputs.transports.NettyTransport$1.getPipeline(NettyTransport.java:110) ~[graylog.jar:?]
    at org.jboss.netty.channel.socket.nio.NioServerBoss.registerAcceptedChannel(NioServerBoss.java:134) [graylog.jar:?]
    at org.jboss.netty.channel.socket.nio.NioServerBoss.process(NioServerBoss.java:104) [graylog.jar:?]
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) [graylog.jar:?]
    at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42) [graylog.jar:?]
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [graylog.jar:?]
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [graylog.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_91]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_91]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]

Any ideas what might be the problem ?

Zombaya
  • 123
  • 1
  • 5

3 Answers3

1

You have to use keytool to import the certificate into the java keystore.

keytool -import -alias mygraylogcertificate -file mygraylogcertificate.cer -keystore examplekeystore

Keystore location is different depending on installation.

mzhaase
  • 3,778
  • 2
  • 19
  • 32
1

Ensure that your Graylog process can read the certificate files and key. (It's normally running as the "graylog" user, not root.) "Failed to initialize an accepted socket" is exactly the error that appears Graylog can't read these files.

You typically want your key, certificate and other files under /etc/graylog (I use /etc/graylog/keys) and owned by the graylog user. Make sure that the key is not readable to group or other.

cjs
  • 1,355
  • 1
  • 12
  • 23
0

We solved this in the end by having the client push logs with rsyslog over TLS to rsyslog on the collecting server and then letting rsyslog on the collecting server forward those to graylog2.

Schematicly: Dev-machine 1 -> rsyslog -> TLS -> collecting-server: rsyslog -> graylog2

Zombaya
  • 123
  • 1
  • 5