What does Apache do with loaded modules when a file is modified?

Question

I am writing on an Apache module which currently runs on 2.2.22. The module runs scripts written in a new programming language, and for optimization, it caches the results of the parsing for successive uses.

Since the cached parsing should be deleted when the file is modified, I store the modification time and check it against the filesystem on each run.

This is some of the code with a debug message:

 if (file->is_modified (mtime)) {
     ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
          "P* removing cached file");
     files.erase(it);
 }

What I discovered when testing this (get a page, modify the file, get it again), is that this block never is reached. The file is parsed again after it's changed, but not because of this check. It looks to me like Apache itself is checking the modification time and is freeing all the memory in the module.

The parsed files are re-used correctly when left unmodified.

Is this intended behavior of the Apache server, and if so, what is this called and where is it documented?

Full code for the function which checks for cached programs

shared_ptr<pstar_file> pstar_pool::get_file_handle (
            request_rec *r,
            wpl_io &io,
            const char *filename,
            int mtime
            )
{
    pstar_map_t::iterator it;

    apr_proc_mutex_lock (mutex);

    // Check if a cached program exists
    it = files.find(filename);
    if (it != files.end()) {
            if (it->second->is_modified (mtime)) {
                    files.erase(it);
            }
            else {
                    apr_proc_mutex_unlock (mutex);
                    return it->second;
            }
    }

    apr_proc_mutex_unlock (mutex);

    shared_ptr<pstar_file> file_ptr(new pstar_file(io, filename, mtime));

    apr_proc_mutex_lock (mutex);
    files.insert(std::pair<string,shared_ptr<pstar_file>>(filename, file_ptr));
    apr_proc_mutex_unlock (mutex);

    return file_ptr;
}

Full code for the whole module on Github (multiple files): https://github.com/atlesn/P-star/blob/master/src/apache2/pstar_pool.cpp

score 5 · Accepted Answer · answered Dec 08 '13 at 08:11

5

There is quite a bit of context missing here. However, I think the answer might be quite simple.

Modules are loaded when a worker process is instantiated. When a module is loaded, typically, it gets copied into the process image in memory, and changes on disk don't affect it. Whether this is relevant depends on what you are doing in your custom module.

Certainly, apache does not exhibit strange or undefined behaviour when files accessed by modules are used and changed. If your module opens a file it would be very strange for a later modification to that file to do what you said, though I get the feeling there is a reason it might that isn't clear.

Check also that the underlying filesystem actually stores and updates mtime in the way you expect. It doesn't always. Your mtime check (which you've not included) also might be wrong.

Since you say the file is being re-loaded by some unknown means, it's quite possible that a new worker process is being instantiated for each request, or that you get a different worker each time you try.

Consider debug printing in your mtime check, to help isolate this.

answered Dec 08 '13 at 08:11

Falcon Momot

24,975
13
61
92

I added the code for the whole function. I used to have a `printf` right after the line `if (it != files.end()) {`. This block is entered when the filename exists in the hash map. After file modification, the map becomes empty and nothing is found. For the `mtime`, i just store the `mtime` in an integer and check `!=` with the new one. – Atle Dec 08 '13 at 15:22
1

How are you modifying the file? It's possible that perhaps when you modify it you are actually doing something like blanking it, removing it, and creating a new one... that would explain the behaviour. It might be wise to fseek(0) it, or better yet to close and reopen it entirely (in case the file was replaced). – Falcon Momot Dec 08 '13 at 19:58
1

This turned out to be very simple of course. For some reason, or by chance, the server began to use a different worker process when i modified the file (re-save in a text editor or with `touch`. I reduced the number of workers to two, and this made it easier to test the mtime-check, and it works perfectly. Apache didn̈́'t do anything funny, of course, I was just confused. Answer is in your 5th paragraph :) – Atle Dec 08 '13 at 21:07

What does Apache do with loaded modules when a file is modified?

1 Answers1