I am trying to apply an item template to my elasticsearch cluster, to deal with the issue of having fields with content longer than 32kb. I am using version 2.4.4, as this is the highest supported version in graylog.
See: https://github.com/Graylog2/graylog2-server/issues/873
Specifically the solution here: https://github.com/Graylog2/graylog2-server/issues/873#issuecomment-199898314
I am also running into an other problem that i am trying to fix with an item template. One of the fields can contain either a number or a string. But because elasticsearch maps the field based on the first occurance of a value in it, it will sometimes give me a MapperParsingException on the active index.
Based on the solution suggested in the linked github issue, i made my own item template and with support from the elasticsearch documentation i added a dynamic template.
This is the result:
{
"template": "graylog*",
"mappings": {
"_default_": {
"_all": {
"enabled": false
},
"dynamic_templates": [{
"entityid_as_string": {
"match": "EntityId",
"mapping": {
"type": "string"
}
}
},
{
"notanalyzed_string": {
"match_mapping_type": "string",
"mapping": {
"ignore_above": 32766,
"type": "string",
"doc_values": true
}
}
}]
}
}
}
The behaviour i expect, is that the field EntityId will always be mapped as a string. And that any string fields in a document, with content more than 32kb will not be indexed.
But that does not seem to be the case. Even after manually rotating the active write index, i am still getting the same errors. I have even rebooted the VM, and rotated active write index - with no effect.
Can anyone see an obvious mistake with my template? Specifically i am unsure if the _all section should be there.
I used this command to add it:
curl -XPUT 'localhost:9200/_template/loggingtemplate?pretty' -H 'Content-Type: application/json' -d'<template here'
And this command to verify that it has been added.
curl -XGET localhost:9200/_template/loggingtemplate