Problems configuring logstash for email output

Question

I'm trying to configure logstash to send email alerts and log output in elasticsearch / kibana.

I have the logs successfully syncing via rsyslog, but I get the following error when I run

/opt/logstash-1.4.1/bin/logstash agent -f /opt/logstash-1.4.1/logstash.conf --configtest

Error: Expected one of #, {, ,, ] at line 23, column 12 (byte 387) after filter { if [program] == "nginx-access" {

grok { match => [ "message" , "%{IPORHOST:remote_addr} - %{USERNAME:remote_user} [%{HTTPDATE:time_local}] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent}” ] } } }

output { stdout { } elasticsearch { embedded => false host => "

Here is my logstash config file

input {
 syslog {
 type => syslog
 port => 5544
}
}


filter {
if [program] == "nginx-access" {

grok {
match => [ "message" , "%{IPORHOST:remote_addr} - %{USERNAME:remote_user} \[%  {HTTPDATE:time_local}\] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent}” ]
}
}
}


output {
  stdout { }
 elasticsearch {
  embedded => false
  host => "localhost"
  cluster => "cluster01"
 }
email {
from => "logstash.alert@nowhere.com"
match =>  [
           "Error 504 Gateway Timeout", "status,504",
           "Error 404 Not Found", "status,404"
          ]
subject => "%{matchName}"
to => "you@example.com"
via => "smtp"
body => "Here is the event line that occured: %{@message}"
htmlbody => "<h2>%{matchName}</h2><br/><br/><h3>Full Event</h3><br/><br/><div align='center'>%{@message}</div>"
  }
}

I've checked line 23 which is referenced in the error and it looks fine....I've tried taking out the filter, and everything works...without changing that line.

Please help

Edit

I've now changed my config to this

input {
 syslog {
 type => syslog
 port => 5544
 }
}

filter {
grok {
type => "syslog"
match => ["syslog_program","nginx-access"]
match => [ "message","%{IPORHOST:remote_addr} - %{USERNAME:remote_user} \[%{HTTPDATE:time_local}\] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent}" ]
add_field => [ "nginx_response", "%{NUMBER:response}" ]
}
}

output {
stdout {}
elasticsearch {
embedded => false
host => "localhost"
cluster => "cluster01"
}
email {
match => [ "status", "status,304"]
to => "test@test.com"
from => "test@test.com"
options => [ "smtpIporHost", "",
         "port", "",
         "userName", "",
         "password", "",
         "starttls", "",
         "authenticationType", ""
       ]
via => "smtp" # or pop or sendmail
   subject => "Found %{IP:client} Alert on %{@source_host}"
   body => "Here is the event line %{@message}"
   htmlbody => "<h2>%{matchName}</h2><br/><br/><h3>Full Event</h3><br/><br/><div align='center'>%{@message}</div>"
}
}

This seems to work, in as much as I can see that it's now recognising things in logstash, and that there is an email plugin command there, but the match fails.....any ideas?

Thanks

Sobrique · Answer 1 · 2014-06-12T14:20:28.483

1

Do you not have to parse out [program] first? I don't think the 'input' field does any sort of filtering at all, so you might need to start with %SYSLOGBASE http://logstash.net/docs/1.4.1/filters/grok

You could try instead perhaps:

if [message] =~ /nginx-access/ {

Which'll keyword match your message field. That'll at least tell you if that is what's happening here.

edited Jun 12 '14 at 14:20

answered Jun 12 '14 at 14:08

Sobrique

3,697
2
14
34

score 0 · Answer 2 · answered Apr 30 '15 at 11:58

0

The email output doesn't support the match function. Instead what you could do would be to add_tag to a successful grok match, then put a conditional around your email output to only send certain emails depending on the tag you have defined.

answered Apr 30 '15 at 11:58

Rumbles

915
1
12
27

Problems configuring logstash for email output

2 Answers2