We have a Debian package which we want to upgrade. It installs a set of java jars, which run as a systemd service.
Can the package be upgraded while it is still running? My understanding is, yes, this is possible.
How does this work? Is the entire package loaded into memory and then the jars replaced while it is still running from memory?
Why can I upgrade the package while it is still running?
In the Unix world, file systems separate file names and file contents, and deleting a file name is always allowed provided you have the appropriate permissions. The file contents are removed when no more references (file names and file handles) remain.
The dpkg program doesn't know or care about running services. Upgrading a package works like this:
.dpkg-new to file names.dpkg-old suffix (this creates an additional reference).dpkg-old suffix. Any file contents that are then unreferenced are deleted here.This reduces the potential for errors, because the unsuffixed file names are only touched in the third step, and steps 1 and 2 are those with the highest potential for errors (e.g. disk full), the unsuffixed file names always refer to either the old or the new name, and the time where a package is inconsistent is kept to a minimum.
For services that are running and that are keeping the files open this means that they will keep using the old file contents, but if they close and reopen the file, they will get the new contents.
For traditional Unix services, that is not a big problem, because they mostly consist of executables and shared libraries, all of which are kept open because they are memory mapped into the process address space, so upgrading a package doesn't disturb these in the least.
For Java programs, whether they are affected by this depends on whether the Java interpreter closes and reopens the JAR files in between. Usually, packagers play it safe, shut down the service before the upgrade, and restart it afterwards rather than risk the service loading classes from different versions into the same service instance.
So the answer is: it depends on the service and the JVM.
Java code is generally loaded instead of memory mapped, so there is usually no need to keep file handles open, and the running service will use the code it has initially loaded, and not care about changed JAR files.
It will not generally load the entire JAR archive into memory though, but only copy the code for individual classes as they are used. Usually this code is subsequently run through a JIT compiler, and the service runs on the compiled version, so the in-memory code has no resemblance to the on-disk version anyway.
Ideally, JVMs would map the JAR files they use to their address space, which would keep a reference to the file open and associated with the process, which would both allow them to load classes from a consistent set of JAR files, and it would allow the post-upgrade summary of which services are using outdated files to work.
This report is a list of processes spawned from systemd service units that have handles to files that have no names left (so their contents will be removed once all of these handles are closed). Its accuracy depends on processes actually keeping file handles open (which for C programs is given because the files are mapped into process address space).
If you want to see what files are currently opened by the service process (or any process, for that matter), use the lsof(8) command to get a list.
tl;dr: For Java, it is mostly safe, if your service doesn't load and unload classes all the time (in which case, what is wrong with that service?).
When you upgrade a service that is currently running, the old version that is sitting in memory does not also get upgraded. The old version will continue to run in memory until it is stopped or restarted. The next time you run the service it will load the upgraded version into memory.
As to why you're allowed to do this, well.... that's a question for the devs. It certainly would be nice if dpkg/apt alerted users of which services are running when upgrading packages. I know it alerts the user if dpkg needs to restart the service, but not if it's already running.
