3

None of the existing answers helped, so here a new question.

Usecase: Redirecting syslog (or) monitoring static file.

I have successfully installed logstash (1.4.2), elasticsearch(1.1.1) and kibana(3.0.1) but struggling to get rid of error 

No results There were no results because no indices were found that match your selected time span

Sample logstash files used are as below. Please let me know, if anything else is required from my end.

  1. syslog (listening on port 9000, yes I have added ". @@localhost:9000" to /etc/rsyslog.d/50-default.conf and restarted rsyslog )

    sudo cat > /etc/logstash/conf.d/10-syslog.conf <<EOF
input {
  tcp {
    port => 9000
    type => syslog
  }
  udp {
    port => 9000
    type => syslog
  }
}
filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}
output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}
EOF

  2. Static file (file present with syslog type of data)

    input {
   file {
      path => "/var/log/awasthi.log"
      type => syslog
      start_position => "beginning"
      sincedb_path => "/dev/null"
   }
}

filter {
if [type] == "syslog" {
  grok {
    match => [ "message", "%{SYSLOGTIMESTAMP} %{NOTSPACE:hostname1}/%{NOTSPACE}"]
  }
}
}

output {
  stdout { codec => rubydebug }
}
user3247463
  • 31
  • 1
  • 3
  • Anything in the logstash or elasticsearch log files? Does anything appear in the elasticsearch directory? I'd assume not given what you get from _aliases. – Paul Haldane Nov 10 '14 at 08:11

3 Answers3

0

As mentioned in another thread:

"I had something similar, it sounds like you havent setup ACL's to allow the logstash user to view that log file.

Use 'setfacl -m u:logstash-:r-x /var/log' for example, and then test by editing /etc/passwd and giving the logstash user a shell temporarily. Then, su - logstash, and try and cd or cat that file. If it works, then the data should appear in your Kibana setup."

I use redis infront of elasticsearch, so i can test if logstash is working by simply running 'LLEN' and 'LPOP'.

  • Could you add a link to the other thread as reference please. – Reaces Dec 17 '14 at 13:14
  • Sorry, yep can do - http://serverfault.com/questions/586118/nothing-appearing-in-kibana-dashboard/653180#653180 –  Dec 17 '14 at 18:14
0

Documentation for elasticsearch 1.1 is no longer available (1.3 or 0.9 are), and ES's default network behaviour has changed (from originally listening on non-loopback IPs, to listening to the loopback by default).

Per https://www.elastic.co/guide/en/elasticsearch/reference/1.3/modules-network.html:

The network.publish_host setting allows to control the host the node will publish itself within the cluster so other nodes will be able to connect to it. Of course, this can’t be the anyLocalAddress, and by default, it will be the first non loopback address (if possible), or the local address.

Although the question is now largely academic, the most likely problem the asker was facing was due to trying to send data to localhost, instead of example.com (which per the question's info, is known to work).

With a more recent ES version, I suspect this issue would not have occured (because ES now listens only to localhost by default).

iwaseatenbyagrue
  • 3,588
  • 12
  • 22
0

List all indices with curl localhost:9200/_cat/indices?v to confirm your logs are getting to elasticsearch. Then make sure the index pattern configured in kibana (gear icon in the upper right, I think) matches the index naming pattern displayed in _cat/indicies (which should be set in the logstash config, but I don't use logstash anymore so I'm not sure. I hugely prefer rsyslog's om-elasticsearch module to logstash. It is much more energy efficient.)

Also make sure that the elasticsearch path configured in kibana's config file is 100% literally copy-and-paste accessible from your local web browser. If you said example.com, you better have example.com hostfiled to the correct server. localhost will not work at all, unless you have elasticsearch installed on your workstation and connected the the main server...

silver
  • 1