0

I have installed Elasticsearch, Logstash, Kibana and Nginx following this guide, exept I used the latest version of everything.

https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-ubuntu-14-04

The problem is that when accessing the webserver it says:

Kibana is loading. Give me a moment here. I'm loading a whole bunch of code. Don't worry, all this good stuff will be cached up for next time!

But it never loads the Kibana start page.

When I'm trying http://server-ip:5601 I get a

ERR_CONNECTION_REFUSED

But it seems to load with curl:

elk-stack:/opt/kibana/bin$ curl -i 0:5601
HTTP/1.1 200 OK
X-App-Name: kibana
Accept-Ranges: bytes
Cache-Control: public, max-age=0
Last-Modified: Tue, 21 Jul 2015 09:46:14 GMT
ETag: W/"6f9-14eb001a270"
Content-Type: text/html; charset=UTF-8
Content-Length: 1785
Vary: Accept-Encoding
Date: Wed, 22 Jul 2015 16:07:09 GMT
Connection: keep-alive

<!DOCTYPE html>
  <!--[if IE 8]>         <html class="no-js lt-ie9" lang="en"> <![endif]-->
  <!--[if gt IE 8]><!--> <html class="no-js" lang="en"> <!--<![endif]-->
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
    <meta name="viewport" content="width=device-width">
    <link rel="shortcut icon" href="styles/theme/elk.ico">
    <title>Kibana 4</title>
    <link rel="stylesheet" href="styles/main.css?_b=7489">

  </head>
  <body kibana ng-class="'application-' + activeApp.id">

    <div class="col-md-offset-4 col-md-4 page-header initial-load">
      <center>
        <img width="128" alt="Loading Kibana" src="images/initial_load.gif">
        <h1>
          <strong>Kibana</strong>
          <small id="cache-message">is loading. Give me a moment here. I'm loading a whole bunch of code. Don't worry, all this good stuff will be cached up for next time!</small>
        </h1>
      </center>
    </div>

    <script>
      window.KIBANA_VERSION='4.1.1';
      window.KIBANA_BUILD_NUM='7489';
      window.KIBANA_COMMIT_SHA='bd4fcb654a2fa062c9a99853ac5f776965ab7637';
    </script>

    <script src="bower_components/requirejs/require.js?_b=7489"></script>
    <script src="require.config.js?_b=7489"></script>
    <script>
      var showCacheMessage = location.href.indexOf('?embed') < 0 && location.href.indexOf('&embed') < 0;
      if (!showCacheMessage) document.getElementById('cache-message').style.display = 'none';

      if (window.KIBANA_BUILD_NUM.substr(0, 2) !== '@@') {
        // only cache bust if this is really the build number
        require.config({ urlArgs: '_b=' + window.KIBANA_BUILD_NUM });
      }

      require(['kibana'], function (kibana) { kibana.init(); });
    </script>

  </body>
</html>

Elasticsearch seems to work.

curl http://localhost:9200
{
  "status" : 200,
  "name" : "Martinex",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "1.7.0",
    "build_hash" : "929b9739cae115e73c346cb5f9a6f24ba735a743",
    "build_timestamp" : "2015-07-16T14:31:07Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}

Kibana runs as a service (se the script in the guide). If i stop it and start Kibana manualy i get:

elk-stack:/opt/kibana/bin$ sudo ./kibana
{"name":"Kibana","hostname":"elk-stack","pid":1414,"level":30,"msg":"Found kibana index","time":"2015-07-22T16:10:31.805Z","v":0}
{"name":"Kibana","hostname":"elk-stack","pid":1414,"level":30,"msg":"Listening on 127.0.0.1:5601","time":"2015-07-22T16:10:31.823Z","v":0}
{"name":"Kibana","hostname":"elk-stack","pid":1414,"level":30,"req":{"method":"GET","url":"/","headers":{"connection":"upgrade","host":"X.X.X.X","accept":"text/html, application/xhtml+xml, */*","accept-language":"sv-SE,sv;q=0.8,en-GB;q=0.5,en;q=0.3","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko","accept-encoding":"gzip, deflate","if-modified-since":"Tue, 21 Jul 2015 09:46:14 GMT","if-none-match":"W/\"6f9-14eb001a270\"","authorization":"Basic YXRlYW9wOncwcms0bWU="},"remoteAddress":"127.0.0.1","remotePort":40809},"res":{"statusCode":304,"responseTime":8,"contentLength":0},"msg":"GET / 304 - 8ms","time":"2015-07-22T16:11:04.362Z","v":0}

I'm not realy sure if the Ngnix config is correct, it looks like this now:

server {
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    root /usr/share/nginx/html;
    index index.html index.htm;

            server_name elk-stack;

    auth_basic "Restricted Access";
    auth_basic_user_file /etc/nginx/htpasswd.users;

    location / {
            proxy_pass http://localhost:5601;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection 'upgrade';
            proxy_set_header Host $host;
            proxy_cache_bypass $http_upgrade;

            try_files $uri $uri/ =404;

Any ideas? I guess there is some logs somewhere, but I'm not sure which one to check. More info of the installation:

Ubuntu 14.04.2
Kibana 4.1.1
Elasticsearch 1.7.0

EDIT: GregL ask for the netstat output:

    $ netstat -lnp
(No info could be read for "-p": geteuid()=1000 but you should be root.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:5601          0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      -
tcp6       0      0 :::80                   :::*                    LISTEN      -
tcp6       0      0 127.0.0.1:9300          :::*                    LISTEN      -
tcp6       0      0 :::22                   :::*                    LISTEN      -
udp6       0      0 :::54328                :::*                                -
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node   PID/Program name    Path
unix  2      [ ACC ]     STREAM     LISTENING     9748     -                   /var/run/acpid.socket
unix  2      [ ACC ]     STREAM     LISTENING     7546     -                   @/com/ubuntu/upstart
unix  2      [ ACC ]     SEQPACKET  LISTENING     1667     -                   /run/udev/control
unix  2      [ ACC ]     STREAM     LISTENING     7921     -                   /var/run/dbus/system_bus_socket

Here is the kibana.yml config:

port: 5601

host: "localhost"

elasticsearch_url: "http://localhost:9200"

elasticsearch_preserve_host: true

kibana_index: ".kibana"    

default_app_id: "discover"

request_timeout: 300000

shard_timeout: 0

verify_ssl: true

bundled_plugin_ids:
 - plugins/dashboard/index
 - plugins/discover/index
 - plugins/doc/index
 - plugins/kibana/index
 - plugins/markdown_vis/index
 - plugins/metric_vis/index
 - plugins/settings/index
 - plugins/table_vis/index
 - plugins/vis_types/index
 - plugins/visualize/index

In the elasticsearch.yml it is bound to localhost:

network.host: localhost

EDIT2: Netstat after all the changes.

    $ netstat -lnp
(No info could be read for "-p": geteuid()=1000 but you should be root.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:9300          0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:5601            0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:9200          0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      -
tcp6       0      0 :::22                   :::*                    LISTEN      -
udp        0      0 0.0.0.0:54328           0.0.0.0:*                           -
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node   PID/Program name    Path
unix  2      [ ACC ]     STREAM     LISTENING     8764     -                   /var/run/acpid.socket
unix  2      [ ACC ]     STREAM     LISTENING     8013     -                   /var/run/dbus/system_bus_socket
unix  2      [ ACC ]     STREAM     LISTENING     1417     -                   @/com/ubuntu/upstart
unix  2      [ ACC ]     SEQPACKET  LISTENING     7856     -                   /run/udev/control
mfernholm
  • 3
  • 1
  • 3

2 Answers2

1

I have a feeling your problems are with the Nginx configuration.

Try commenting out host: "localhost" in kibana.yml, restarting the Kibana service and then trying server-ip:5601 again.

If that works, then you'll need to resolve the Nginx portion.

Changing the line proxy_pass http://localhost:5601 to proxy_pass http://<serverip>:5601 should fix the error about being unable to connect to the upstream server.

GregL
  • 9,030
  • 2
  • 24
  • 35
  • It removed the error message but still the same problem so there something wrong with the authentication. `2015/07/24 16:10:10 [error] 1371#0: *1 no user/password was provided for basic authentication, client: X.Y.Z.112, server: elk-stack, request: "GET / HTTP/1.1", host: "X.Y.Z.56"` – mfernholm Jul 24 '15 at 14:14
  • I disabled ipv6 also. I'm going to do some more troubleshooting on monday. – mfernholm Jul 24 '15 at 14:16
  • Does it work if you comment out the two lines related to authentication (`auth_basic ...` and `auth_basic_user_file ...`)? – GregL Jul 24 '15 at 14:28
  • There is no error message now, but still the same problem loading the page on port 80. – mfernholm Jul 27 '15 at 08:56
  • I see you've also got a `try_files` directive. That's not in the How-To you've linked, so maybe revisit that and see if you can fix it there. The other weird thing with the Nginx config provided adds a couple of headers when proxying, but Kibana doesn't do it when accessed directly. Not sure why they want you to add it. – GregL Jul 27 '15 at 11:35
  • The other thing that you can do is just run Kibana on port 80 (change `port: 5601` to `port 80` in kibana.yml and restart the service). That said, it will only work if a) you don't need authentication, and b) you don't already have something else listening on port 80, and do'n't need Nginx for anything else. – GregL Jul 27 '15 at 11:37
  • I removed the line about `try_files` and now it works. I have changed back to localhost in the kibana.yml and nginx config file also. So now you can only connect to port 80 externally and you have to authenticate. It still gives me those authentication errors in the log, but I can live with that. Thank you so much for your help @GregL – mfernholm Jul 27 '15 at 12:42
0

You shall connect to port http://server-ip:80 as your NGinx is configured to serve port 80 and proxy_pass from port 0.0.0.0:80 to localhost:5601

Dima Medvedev
  • 346
  • 1
  • 7
  • Now it works to connect to port 5601, but not port 80. So, when i connect on port 80 I get promted for username/password and then it loads the same page as before. When I connect to :5601 I don't need a password. But it's a bit strange beacause a get this errors in the log: – mfernholm Jul 24 '15 at 09:04
  • 2015/07/24 10:32:01 [error] 1288#0: *26 no user/password was provided for basic authentication, client: X.Y.Z.112, server: elk-stack, request: "GET / HTTP/1.1", host: "X.Y.Z.56" – mfernholm Jul 24 '15 at 09:04
  • `2015/07/24 10:32:06 [error] 1288#0: *26 connect() failed (111: Connection refused) while connecting to upstream, client: X.Y.Z.112, server: elk-stack, request: "GET / HTTP/1.1", upstream: "http://[::1]:5601/", host: "X.Y.Z.56"` – mfernholm Jul 24 '15 at 09:05