0

I am running rsyslog on CentOS and logrotate to rotate my logs. All hosts write their logs to /var/log/syslog/ in their own separate directory in this manner:

/var/log/syslog/host1
/var/log/syslog/host2
/var/log/syslog/host3
/var/log/syslog/host4
/var/log/syslog/host5
/var/log/syslog/host6
/var/log/syslog/host7
/var/log/syslog/host8

Under each of those directories is a file like 'host1.log' that needs to be rotated. The only problem is that I have two hosts whose logs are REALLY big (host3 and host7), and need to be rotated with a different retention schedule. I want to keep logs for 45 days on these two specific hosts, but all other hosts should be kept for 120 days. The problem is that they are alphabetical, and don't process correctly. I have tried creating separate policies like this, in my /etc/logrotate.conf file:

/var/log/syslog/host3/*.log {
daily
rotate 45
maxage 45
compress
dateext
dateyesterday
}

/var/log/syslog/host7/*.log {
daily
rotate 45
maxage 45
compress
dateext
dateyesterday
}

# Everything else
/var/log/syslog/*/*.log {
daily
rotate 120
maxage 120
compress
dateext
dateyesterday
}

When I run this, it rotates host3 and host7 at 45 days, like it's supposed to. Then when it gets to the /var/log/syslog//.log section, it only processes down to host3, then stops. So basically, host4, host5, host6, and host8 are never considered for rotation.

I have tried changing the order in the /etc/logrotate.conf file to put the "everything" rule at the top, like this:

# Everything else
/var/log/syslog/*/*.log {
daily
rotate 120
maxage 120
compress
dateext
dateyesterday
}

/var/log/syslog/host3/*.log {
daily
rotate 45
maxage 45
compress
dateext
dateyesterday
}

/var/log/syslog/host7/*.log {
daily
rotate 45
maxage 45
compress
dateext
dateyesterday
}

When I run it like that, it sets all my hosts to 120 day rotation using the first rule, then ignores my specific rules for host3 and host7. When it gets to those directories, it says:

rotating pattern: /var/log/syslog/host3/*.log  after 1 days (45 rotations)
empty log files are rotated, old logs are removed
No logs found. Rotation not needed.

So, my question is, how do I setup my /etc/logrotate.conf file to allow for separate rules for separate directories? Is it even possible?

Jake
  • 25
  • 2
  • 8
  • Or, could I perhaps add a comment to my "everything" rule that tells it to ignore host3.log and host7.log? Or perhaps exclude the host3 and host7 directories from the first rule? – Jake Oct 10 '18 at 16:28

1 Answers1

0

I would explicitly include the paths for the hosts other than host3 and host7 instead of using a wildcard that includes them which make them configured twice and that means that they will only obey the first configuration present. You might want to try this:

/var/log/syslog/host3/*.log  var/log/syslog/host7/*.log {
    daily
    rotate 45
    maxage 45
    compress
    dateext
    dateyesterday
}

# Everything else
/var/log/syslog/*/*.log {
    daily
    rotate 120
    maxage 120
    compress
    dateext
    dateyesterday
    prerotate
        bash -c "[[ ! $1 =~ host3 ]] && [ ! $1 =~ host7 ]]"
    endscript
}
AHT
  • 166
  • 1
  • 7
  • That's a good suggestion. The only problem is I don't just have 8 hosts. I was simplifying for the sake of the question. I currently have 420 hosts in that folder, and that number will continue to increase as I continue to roll out the project. I will need to use wildcards one way or another. – Jake Oct 10 '18 at 19:20
  • I modified my answer to include a regex. hopefully this will do. – AHT Oct 10 '18 at 19:37
  • Thanks again for the response. My hostnames are not that clean. They are all a combination of ip addresses and naming schemes based on state, city, and device type, so it might be hard to catch them all in a regex express. I wonder if there is a way, instead, to say something like /var/log/syslog/*/*.log and exclude host3.log and host7.log explicitly. That would be much easier. I've been toying around with that, but haven't found a combination that works. – Jake Oct 10 '18 at 20:01
  • I think a prerotate script might do the job. I edited my answer but haven't tested it properly, please test it and let me know – AHT Oct 11 '18 at 09:32
  • Thanks for another tip. I found an article that suggested a prerotate script like that. I've already tried it once, but let me mess around with it a little more and see if I can get it to work. – Jake Oct 11 '18 at 15:08
  • I just tried your suggestion with the prerotate script. I added the missing bracket. I tried it with both 'host3' and 'host3.log' syntax. Both times, it stops processing when it gets to host3. Everything below it does not get rotated. Thanks for taking the time to look at this. – Jake Oct 11 '18 at 15:48
  • I've read a few posts that state overlapping is intentionally disallowed, but you can get around it by using globbing. Something like this: https://serverfault.com/questions/388105/how-can-i-specify-multiple-rules-for-a-particular-log-files-with-logrotate I've tried a few variations of that as well, but can't seem to get it to work either. I'm wondering if this is even possible. – Jake Oct 11 '18 at 15:50
  • You really need to have a certain naming convention to be able to flexibly tweak the log rotation configuration. Another option is to use size which will take precedence over daily rotation. – AHT Oct 11 '18 at 22:14
  • I have tried just about everything I can think of, including separate settings based on size. It all goes back to the original problem of not being able to have separate rules for different directories in the same /var/log/syslog directory. I am going to have to find a retention period that works for all devices and make it work. – Jake Oct 22 '18 at 22:15