GNU find .extension files and -exec, while new .extension files are being made

Question

Here is a problem, can someone please explain or provide some references. I've been trying to read since an hour but can not find the answer.

Suppose, I have process1 going on a pool of directories. process1 picks a directory and creates, say, .csv files inside it. The .csv files are being appended to, and are, therefore, incomplete while process1 is continuing on this, say, directory1. process1 then moves on to directory2 and so on.

Now, I want to do something with csvs in directories that process1 has finished with. Ideally, I should check which directories have been finished with and run process2 on those. But in my infinite wisdom, I do this:

find directories* -type f -name '*.csv' -exec process2

My assumption was that find will first find all csvs existing till the point of invocation of find and then -exec process2 on those only, ignoring any csvs that will have been made after the invocation of find.

It seems I was wrong (still not sure). Can someone explain, please?

Thanks.

Answer 1

The way find works with -exec is that the exec process is called as each file is found. If you want to work from a snapshot, you're probably better off doing something like this:

find directories* -type f -name '*.csv' > /tmp/filelist
cat /tmp/filelist | while read file; do 
  process2 $file # I'm assuming you want to pass the filename somehow...
done;

Again, you still have no idea which ones are actually being written to, so you have to solve that separately.