Logstash can't read files it should have access too

Question

I've added user logstash into group adm using the command $ usermod -a -G adm logstash.

One of the files that the logstash agent is trying to read is /var/log/nginx/foo-access.log, which has the following permissions:

-rw-r----- 1 www-data adm 0 Jul 25 07:52 /var/log/nginx/foo-access.log

When I sudo su logstash I can read the file, BUT, when I $ sudo service logstash_agent restart (which the init script runs as the logstash user) it fills up the logstash logs with:

{:timestamp=>"2013-07-31T17:05:17.287000+0000", :message=>"failed to open /var/log/nginx/foo-access.log: Permission denied - /var/log/nginx/foo-access.log", :level=>:warn}

I can confirm the logstash user is in the adm group:

$ groups logstash
logstash : logstash adm

This file definitely has the right file access:

$ getfacl /var/log/nginx/foo-access.log
getfacl: Removing leading '/' from absolute path names
# file: var/log/nginx/foo-access.log
# owner: www-data
# group: adm
user::rw-
group::r--
other::---

I've tried turning it off and on again.

I assume you also checked the parent directory permissions (`/var/log/nginx`) — dawud, Jul 31 '13 at 17:48
The error message has `kapture-api3-access.log` while your examples are `foo-access.log`. I'm guessing the "foo" file vs error log file-name difference is just an oversight and not literally what the file name is? — fideloper, Jul 31 '13 at 17:57
lol, yeah this is just me not bothering to obscure all names. — Phil Sturgeon, Jul 31 '13 at 17:58
Your file doesn't seem to have the correct group write permissions. Chmod to fix — Nithin Meppurathu, Jul 31 '13 at 17:32
I don't want to write to the file, like I said I want to read it. — Phil Sturgeon, Jul 31 '13 at 17:37
I added the output of that command to the question. I'd ran it before with the results I expect, so didn't bother initially. — Phil Sturgeon, Jul 31 '13 at 17:56
Another quick test, you said you got it to work by su the user. Can you try giving the user just read access to the file and seeing if it works. I just don't feel that read access is all you need. — Nithin Meppurathu, Jul 31 '13 at 18:45

score 5 · Accepted Answer · answered Aug 04 '13 at 11:15

It turned out to be related to this bug in upstart:

https://bugs.launchpad.net/upstart/+bug/812870

Because adm was a secondary group it was not being applied to the process, which is broken as f**k and is apparently fixed in a later version.

My solution was to add setguid adm to the init file.

score 5 · Answer 2 · answered Jun 05 '14 at 09:32

5

Here is a working workaround:

sudo vi /etc/init.d/logstash

modify

    LS_GROUP=logstash

by

    LS_GROUP=adm

then

sudo /etc/init.d/logstash start

answered Jun 05 '14 at 09:32

David

151
1
1

1

Also works by adding "LS_GROUP=adm" to /etc/defaults/logstash - the nice part about that is that it's easier to manage using puppet . – thinice Apr 13 '15 at 19:47

Logstash can't read files it should have access too

2 Answers2