I'm setting up a generic Elasticsearch-Logstash-Kibana stack to deploy to a few of my clients. I'm trying to template some of the pipelines, so that we only need to deploy configs/pipelines as needed for each client.
Logstash refers to input{...}, filter{...}, and output{...} as sections and their content as plugins, their aggregation as a processing pipeline, and the file each pipeline is contained in as a config file.
With that in mind, is there a scope for sections and pipelines? That is, are the sections defined in a particular config file used only by the pipeline in that config file?
If I had 2 config files and 2 pipelines:
# my_apache_pipeline.config
input {
tcp {
port 5000
}
}
filter {
if [application == "httpd"] {
...
}
}
output {
elasticsearch {
...
}
}
and
#my_nginx_pipeline.config
input {
tcp {
codec => "json"
port => 6000
}
}
filter {
if [application == "nginx"] {
...
}
}
output {
elasticsearch {
...
}
}
Do those two above config files create the same 2 pipelines as the single below config file?
#my_merged_pipeline.config
input {
tcp {
codec => "json"
port => 6000
}
tcp {
port 5000
}
}
filter {
if [application == "nginx"] {
...
}
if [application == "httpd"] {
...
}
}
output {
elasticsearch {
...
}
}
That is, does it matter which config file a set of plugins/sections are in to get the pipelines? Or does an input {...} defined in a particular config file only apply to the filter {...} and output {...} in that config file?
