I have a process that's continuously writing files into a directory structure that looks like:
152527
├── 1525270000000
│ ├── 01be6dc0b6e3e087.json
│ ├── 067e2490bb7ee05c.json
│ ├── 15ec3513bf7deef6.json
├── 1525270010000
│ ├── 01be6dc0b6e3e087.json
│ ├── 067e2490bb7ee05c.json
│ ├── 15ec3513bf7deef6.json
...
The directories are written to a series of mounts, /mnt/fs_01, /mnt_fs_02, /mnt/fs_03, all combined into a read-only overlayfs of /mnt/fs_all.
A set of processes writes to a single mount at a time; the same set of processes reads from the combined /mnt/fs_all. (It's a uWSGI application.)
The problem is that fairly often, a new directory will be created in an underlying file system, but it doesn't show up on read.
The documentation on overlayfs mentions:
When a 'readdir' request is made on a merged directory, the upper and lower directories are each read and the name lists merged in the obvious way (upper is read first, then lower - entries that already exist are not re-added). This merged name list is cached in the 'struct file' and so remains as long as the file is kept open. If the directory is opened and read by two processes at the same time, they will each have separate caches. A seekdir to the start of the directory (offset 0) followed by a readdir will cause the cache to be discarded and rebuilt.
This means that changes to the merged directory do not appear while a directory is being read. This is unlikely to be noticed by many programs.
Unfortunately, this is noticed by my program.
I can implement the seekdir suggestion, but I'd prefer to disable the cache altogether if possible.
Is there a way to do that?
