How to tell of a GZip file is still being written while being copied

Question

I am getting a gzip file in a path.

Input Folder Path:

<server>/input_gate

I want to check whether the GZIP file I am getting in this folder are valid? I have used the following command. It works good to find a gzip file valid is or not?

gzip -t -v <file_name>

But When the move is happening, the file is shown as invalid file. But actually the file is valid once the move is done. Is there any way to find whether the file which I get is a completely moved file or move is still in progress?

Answer 1

As per this answer on “Ask Ubuntu” you can use lsof (list open files) to check if the file is open:

lsof | grep <server>/input_gate

Answer 2

Using lsof

To test whether some process has a file open, say path/to/input_gate, run the command:

lsof path/to/input_gate

This will exit with code 0 if there is a process using the file and code 1 if no processing is using it (or an error occurred). This exit code can be used by shell commands.

Testing

To test this, let's create a process to open our file:

sleep 10 >>path/to/input_gate &

This will keep the file open for 10 seconds. To test lsof, quickly run the following command:

$ if lsof path/to/input_gate >/dev/null; then echo "In use"; else echo "Not in use"; fi
In use

Wait 10 seconds and try again:

$ if lsof path/to/input_gate >/dev/null; then echo "In use"; else echo "Not in use"; fi
Not in use

Waiting until the file is ready

If you want to wait until no other process is using the file:

while lsof path/to/input_gate >/dev/null; do sleep 1; done; echo "Finally, not in use"

Using inotifywait

If your system supports inotifywait (a linux system should), then it is possible to look for close events on the file of interest:

inotifywait  "path/to/input_gate" -e close

The above command will block until some process has closed the file.

The advantage of this approach is that it eliminates polling.

The disadvantage of this approach is that it will notify you the first time that some process closes the file and that might not be the same thing as all processes having closed the file.