Why is Uchiwa not able to monitor the datacenter on CentOS7?

Question

This post combined with Install Sensu using YUM was used in order to install, configure and start Sensu and related services:

sudo yum install -y erlang && \
sudo rpm -Uvh http://www.rabbitmq.com/releases/rabbitmq-server/v3.5.0/rabbitmq-server-3.5.0-1.noarch.rpm && \
sudo yum install -y redis && \
echo '[sensu]
name=sensu
baseurl=http://sensu.global.ssl.fastly.net/yum/$basearch/
gpgcheck=0
enabled=1' | sudo tee /etc/yum.repos.d/sensu.repo && \
sudo yum install -y sensu && \
sudo yum install -y uchiwa && \
for s in rabbitmq-server redis.service sensu-server sensu-api uchiwa; do sudo systemctl restart $s; done && \
sudo rabbitmqctl add_vhost /sensu && \
sudo rabbitmqctl add_user sensu secret && \
sudo rabbitmqctl set_permissions -p /sensu sensu ".*" ".*" ".*"

Analysis

Navigating to :3000 shows the uchiwa dashboard and indicates Connection error. Is the Sensu API running?
/var/log/sensu/sensu-api.log indicates

{"timestamp":"2016-07-03T22:58:58.532905+0000","level":"warn","message":"config file does not exist or is not readable","file":"/etc/sensu/config.json"}
{"timestamp":"2016-07-03T22:58:58.533069+0000","level":"warn","message":"ignoring config file","file":"/etc/sensu/config.json"}
{"timestamp":"2016-07-03T22:58:58.533137+0000","level":"warn","message":"loading config files from directory","directory":"/etc/sensu/conf.d"}
{"timestamp":"2016-07-03T22:58:58.712175+0000","level":"info","message":"api listening","protocol":"http","bind":"0.0.0.0","port":4567}

The config.json does not seems to exist (/etc/sensu/config.json: No such file or directory)
Downloading the example config.json and restarting the sensu-api did not solve the issue

{
  "rabbitmq": {
    "host": "localhost",
    "vhost": "/sensu",
    "user": "sensu",
    "password": "secret"
  },
  "redis": {
    "host": "localhost",
    "port": 6379,
    "password": "secret"
  }
}

the redis port is listening:

LISTEN     0      128    127.0.0.1:6379                     *:*

the rabbitmq port as well:

LISTEN     0      100          *:4567                     *:*

Uchiwa cannot connect to the datacenters

{
  "sensu": [
    {
      "name": "Site 1",
      "host": "localhost",
      "port": 4567,
      "timeout": 10
    },
    {
      "name": "Site 2",
      "host": "localhost",
      "port": 4567,
      "ssl": false,
      "path": "",
      "user": "",
      "pass": "",
      "timeout": 10
    }
  ],
  "uchiwa": {
    "host": "0.0.0.0",
    "port": 3000,
    "refresh": 10
  }
}

{"timestamp":"2016-07-03T23:34:32.990067621Z","level":"warn","message":"GET http://localhost:4567/stashes returned: 500 Internal Server Error"}
{"timestamp":"2016-07-03T23:34:32.990102095Z","level":"warn","message":"Connection failed to the datacenter Site 1"}
{"timestamp":"2016-07-03T23:34:32.990115588Z","level":"info","message":"Updating the datacenter Site 2"}
{"timestamp":"2016-07-03T23:34:32.991462585Z","level":"warn","message":"GET http://localhost:4567/stashes returned: 500 Internal Server Error"}
{"timestamp":"2016-07-03T23:34:32.991492978Z","level":"warn","message":"Connection failed to the datacenter Site 2"}

navigating to <IPADDRESS>:4567 results in:

{"error":"redis and transport connections not initialized"}

redis replies

sudo redis-cli ping
PONG

redis password was set, but the issue persists

sudo redis-cli
127.0.0.1:6379> auth secret
OK

Changing the `redis.json` to use IPv4 only (i.e. `127.0.0.1` instead of `localhost` fixed it for me...) (But my `sensu-api.log` showed return code 500) — Gert van den Berg, Nov 09 '17 at 16:01

score 3 · Answer 1 · answered May 17 '17 at 12:05

I first got RabbitMQ working when I set up correct SSL config in rabbitmq.json, it shows an example on the Sensu Website. After that it only said: "error": "not connected to redis"

I then got Redis working by changing from "localhost" in the sensu config to "127.0.0.1". Since Redis wasn't listening on IPv6 and "localhost" resolved both ::1 & 127.0.0.1 it didn't work properly.

This was a fresh install on Debian 8.

score 1 · Accepted Answer · edited Jul 06 '16 at 12:02

Concise

The sensu-client seems to require a running rabbitmq-server. As this message broker did not seem to be running neither the sensu-client was able to start, nor the Uchiwa dashboard was able to monitor it.

Verbose

If the following error:

{"error":"redis and transport connections not initialized"}

is shown and redis is listening, please check that rabbitmq is running and check the rabbitmq port using rabbitmq-status:

sudo systemctl status rabbitmq-server

Example

Today the above error was shown and when the rabbitmq port was checked it appeared that a certain port was not listening:

{listeners,[{clustering,25672,"::"},{amqp,5672,"::"}},

As SSL was configured, port 5671 should be configured in /etc/rabbitmq.config. Once the messagebroker server was restarted the ssl port 5671 started to listen and the issue was solved:

{listeners,[{clustering,25672,"::"},{amqp,5672,"::"},{'amqp/ssl',5671,"::"}]},

my rabbitmq config is :

[
{rabbit, [ 
{ssl_listeners, [5671]},
{ssl_options, [{cacertfile,"/etc/rabbitmq/ssl/cacert.pem"},
               {certfile,"/etc/rabbitmq/ssl/cert.pem"},
               {keyfile,"/etc/rabbitmq/ssl/key.pem"},
               {verify,verify_peer},
               {fail_if_no_peer_cert,true}]}
 ]}
  ].

score -1 · Answer 3 · edited May 09 '17 at 12:34

-1

Verify that sensu-api is configured and running. On centos-7.x systemctl status sensu-api
to start the servive systemctl start sensu-api

edited May 09 '17 at 12:34

HBruijn

72,524
21
127
192

answered May 08 '17 at 16:48

Dael

1

Why is Uchiwa not able to monitor the datacenter on CentOS7?

3 Answers3