0

I have a virtual machine running on Ubuntu 20.04, I installed Telegraf, InfluxDB and Grafana for monitoring VMs and cluster's nodes.

Everythings work except when I try to make telegraf working in background.

The command to start manually Telegraf which is working :

telegraf --config http://XXX.XX.X.X:8086/api/v2/telegrafs/XXXXXXXXXXX

returns me :

2021-03-14T00:02:27Z I! Starting Telegraf 1.17.3
2021-03-14T00:02:27Z I! Loaded inputs: cpu disk diskio mem net processes swap system
2021-03-14T00:02:27Z I! Loaded aggregators:
2021-03-14T00:02:27Z I! Loaded processors:
2021-03-14T00:02:27Z I! Loaded outputs: influxdb_v2
2021-03-14T00:02:27Z I! Tags enabled: host=XXXXX
2021-03-14T00:02:27Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"XXXXX", Flush Interval:10s

So everythings good so far.

But impossible to run this service (/lib/systemd/system/telegraf.service):

[Unit]
Description=The plugin-driven server agent for reporting metrics into InfluxDB
Documentation=https://github.com/influxdata/telegraf
After=network.target

[Service]
EnvironmentFile=-/etc/default/telegraf
User=telegraf
ExecStart=/usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d $TELEGRAF_OP>
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartForceExitStatus=SIGPIPE
KillMode=control-group

[Install]
WantedBy=multi-user.target

It's starting well but after checking the service status I'm getting :

14T00:10:46Z E! [agent] Error writing to outputs.influxdb: could not write any address
14T00:10:56Z W! [outputs.influxdb] Metric buffer overflow; 25 metrics have been dropped
14T00:10:56Z E! [outputs.influxdb] When writing to [http://XXX.XX.X.X:8086]: 401 Unauthorized
14T00:10:56Z E! [agent] Error writing to outputs.influxdb: could not write any address
14T00:11:06Z W! [outputs.influxdb] Metric buffer overflow; 25 metrics have been dropped
14T00:11:06Z E! [outputs.influxdb] When writing to [http://XXX.XX.X.X:8086]: 401 Unauthorized
14T00:11:06Z E! [agent] Error writing to outputs.influxdb: could not write any address
14T00:11:16Z W! [outputs.influxdb] Metric buffer overflow; 25 metrics have been dropped
14T00:11:16Z E! [outputs.influxdb] When writing to [http://XXX.XX.X.X:8086]: 401 Unauthorized
14T00:11:16Z E! [agent] Error writing to outputs.influxdb: could not write any address

And InfluxDB does not get any data and so Grafana.

Can anyone help me solve this service problem and get Telegraf working in background pls ?

LuckyFr
  • 103
  • 1
  • 7
  • Hello, I can't add the tag InfluxDB because I don't have 300 of reputation. If someone can help please. Thank you very much. – LuckyFr Mar 14 '21 at 00:23
  • When you manually run telegraf, are you running as the telegraf user or as root? What's $TELEGRAF_OP? Why is there a ">" at the end of your ExecStart line? – Quetza Mar 15 '21 at 13:26
  • hello, ty for your comment. the telegraf --config command works well both in user and in root. it's just the telegraf service is not working. so can't run telegraf in background which is what I'm looking for. about the $TELEGRAF_OP it's an env variable about I don't know what but surely related to the config directory. – LuckyFr Mar 15 '21 at 13:40
  • What happens if you run the same command as in ExecStart as the telegraf user? Also, try set the WorkingDirectory to be the same as the directory you're running from when using the shell. – Quetza Mar 15 '21 at 14:04
  • If I enter : telegraf -config /etc/telegraf/telegraf.conf. It's working no matter where I am. I'm not sure the workign directory in the Exec start is useful. – LuckyFr Mar 15 '21 at 15:13
  • Change the ExecStart line to be `telegraf -config /etc/telegraf/telegraf.conf` – Quetza Mar 15 '21 at 15:37
  • Already did of course. It's the same error as shown above. – LuckyFr Mar 15 '21 at 15:43

1 Answers1

3

This information was hard to find! I had exactly the same problem and I found the solution here: https://github.com/influxdata/telegraf/issues/8395.

So all you have to do is to add the INFLUX_TOKEN variable to /etc/default/telegraf such as:

INFLUX_TOKEN=mytoken
Andrew Schulman
  • 8,561
  • 21
  • 31
  • 47
Dylan Do Amaral
  • 46
  • 1
  • Genius mate !!! the hell was this. there are 2 conf file. this one is completly lost. It's empty and not really shown. Thank you very much. Upvoted. – LuckyFr Mar 15 '21 at 22:28