How to migrate graylog2 to new server with all saved data and logs?

Question

I need to change my graylog2 server to a new one with all data included. I've installed the new version, mongodb, elasticsearch and graylog2-web-interface.

I also copied all elasticsearch_data to the new server and checked all config files:

graylog2.conf

graylog2-elasticsearch.yml

indexer.yml

mongoid.yml

general.yml

email.yml

Anybody got the idea of the next steps? What else I need to do to have all previous logs and data in my new graylog2-server ?

You need to copy mongodb data in addition to elasticsearch data. — sciurus, May 18 '14 at 19:40
What version of graylog2 are you running? New 0.20.1 or 0.20.2? — cr0c, May 21 '14 at 13:25

score 1 · Answer 1 · answered May 21 '14 at 13:04

Not only configuration files you need. Dump elasticsearch and mongodb data and restore them on new servers. I don't know if copying elasticsearch data it will make elastic search work, but dump and restore definitively will work.

Personally I use es_dump_restore for elastic search and mongodump for mongodb.

Before all, stop the new servers (graylog rails interface) and restore the dumps, and only after that start the graylog interface.

score 1 · Answer 2 · answered Jan 18 '16 at 14:37

I assume you do have a lot of data on graylog2 but you want to avoid large downtime. This way, you need to copy the config (/etc/graylog and /etc/elasticsearch).

Then first you need to join the new elasticsearch to your existing elasticsearch cluster. Then you need to initiate data migration to new node. It can be done by setting:

# old node  /etc/elasticsearch/elasticsearch.yml
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low=0%
cluster.routing.allocation.disk.watermark.high=0%

Alternatively, you could initiate data migration with:

curl -XPUT localhost:9200/_cluster/settings -d '{
    "transient" : {
        "cluster.routing.allocation.exclude._ip" : "10.0.0.1"
    }
}'

Here "exclude" refers to node which should be excluded from storing the existing data (i.e. exclude the old node).

After you confirm data has moved (i.e. all shards on new node), you could stop everything on the old node and start graylog on new node.

How to migrate graylog2 to new server with all saved data and logs?

2 Answers2