cp --update still copies unchanged files

1

I'm trying to copy only changed files using 'cp -pu' under Linux (Red Hat 6.1, source/destination FS are GlusterFS mounted partitions) but I found that it still tries to copy file seemingly old and unchanged. In fact, just running the same command again would skip to the next file. I can reproduce the problem in the test shown below. Notice that in the test, I hit control-C to stop the interactive cp command the each time it asks. The second run would skip the file asked in the first run. Why does this happen? And is there a better, fast standard tool to copy locally mounted directories? (I could probably try rsync but I don't need to be super accurate since the data are only for testing and I'm concerned about its speed.)

 [root@flex 6b42]# for d in *.story; do echo trying $d; ls -al $d
 /data/staging/./storage01/42/6b/6b42/$d; cp --parent -rpu -v -i $d
 /data/staging/./storage01/42/6b/6b42/; done trying
 0a65b244-f342-a120-9a47-62d0c6aa4a26.story
 0a65b244-f342-a120-9a47-62d0c6aa4a26.story: total 228 drwxrwx---    2
 somebody somebody     53 Jan  8 02:23 . drwxrwx--- 1280 somebody
 somebody 147456 Apr 30 11:07 ..
 -rw-r--r--    1 somebody somebody   2821 Nov 28  2011 data
 -rw-r--r--    1 somebody somebody      0 Nov 28  2011 images
 -rw-r-----    1 somebody somebody     17 Nov 28  2011 .timestamp

 /data/staging/./storage01/42/6b/6b42/0a65b244-f342-a120-9a47-62d0c6aa4a26.story:
 total 228 drwxrwx---    2 somebody somebody     53 Jan  8 02:23 .
 drwxrwx--- 1363 somebody somebody 155648 Apr 30 11:07 ..
 -rw-r--r--    1 somebody somebody   2821 Nov 28  2011 data
 -rw-r--r--    1 somebody somebody      0 Nov 28  2011 images
 -rw-r-----    1 somebody somebody     17 Nov 28  2011 .timestamp cp: overwrite
 `/data/staging/./storage01/42/6b/6b42/0a65b244-f342-a120-9a47-2d0c6aa4a26.story/.timestamp'?
 ^C [root@flex 6b42]# for d in *.story; do echo trying $d; ls -al $d
 /data/staging/./storage01/42/6b/6b42/$d; cp --parent -rpu -v -i $d
 /data/staging/./storage01/42/6b/6b42/; done trying
 0a65b244-f342-a120-9a47-62d0c6aa4a26.story
 0a65b244-f342-a120-9a47-62d0c6aa4a26.story: total 228 drwxrwx---    2
 somebody somebody     53 Jan  8 02:23 . drwxrwx--- 1286 somebody
 somebody 147456 Apr 30 11:07 ..
 -rw-r--r--    1 somebody somebody   2821 Nov 28  2011 data
 -rw-r--r--    1 somebody somebody      0 Nov 28  2011 images
 -rw-r-----    1 somebody somebody     17 Nov 28  2011 .timestamp

 /data/staging/./storage01/42/6b/6b42/0a65b244-f342-a120-9a47-62d0c6aa4a26.story:
 total 228 drwxrwx---    2 somebody somebody     53 Jan  8 02:23 .
 drwxrwx--- 1363 somebody somebody 155648 Apr 30 11:07 ..
 -rw-r--r--    1 somebody somebody   2821 Nov 28  2011 data
 -rw-r--r--    1 somebody somebody      0 Nov 28  2011 images
 -rw-r-----    1 somebody somebody     17 Nov 28  2011 .timestamp trying 0aee088d-af48-1236-bcae-6d2033a2acb7.story
 0aee088d-af48-1236-bcae-6d2033a2acb7.story: total 232 drwxrwx---    2
 somebody somebody     53 Jan  7 09:39 . drwxrwx--- 1286 somebody
 somebody 147456 Apr 30 11:07 ..
 -rw-r--r--    1 somebody somebody   5952 Dec 19 12:40 data
 -rw-r--r--    1 somebody somebody      0 Dec 19 12:40 images
 -rw-r-----    1 somebody somebody     17 Dec 19 12:40 .timestamp

 /data/staging/./storage01/42/6b/6b42/0aee088d-af48-1236-bcae-6d2033a2acb7.story:
 total 232 drwxrwx---    2 somebody somebody     53 Jan  7 09:39 .
 drwxrwx--- 1363 somebody somebody 155648 Apr 30 11:07 ..
 -rw-r--r--    1 somebody somebody   5952 Dec 19 12:40 data
 -rw-r--r--    1 somebody somebody      0 Dec 19 12:40 images
 -rw-r-----    1 somebody somebody     17 Dec 19 12:40 .timestamp cp: overwrite
 `/data/staging/./storage01/42/6b/6b42/0aee088d-af48-1236-bcae-6d2033a2acb7.story/.timestamp'?
 ^C

teddy

Posted 2012-06-07T17:58:23.610

Reputation: 111

I also have the impression that -u copies files it should not copy, e.g. files that have an older timestamp (changed) on the source than on the destination. – Ben – 2018-11-02T12:27:26.450

Answers

1

cp --update doesn't check if the files are different, it just compares timestamps.

Rsync, which you seem to know about already, sounds like a good fit for your situation.

Use rsync -av $d/ /data/staging/./storage01/42/6b/6b42/$d/

Ankur

Posted 2012-06-07T17:58:23.610

Reputation: 709

Yes, I know the difference between cp and rsync. And I just tried out rsync, it apparently got the same problem too. so I think this could be a file system issue? Also, I mistakenly said it is an NFS mount but it is in fact a Glusterfs mount. Maybe the bug is in gluster? – teddy – 2012-06-08T17:53:10.797

Maybe a timestamp issue? stat the files and compare.

Also try the -c flag with rsync which compares files based on checksums instead of size & modification time; this may be a bit slower, though. – Ankur – 2012-06-10T07:41:18.543

0

If you're only concerned with speed, rsync is not an option for a local backup, contrary to what the other answer says. I am confused too by -u, but in the end if it produces what you need on the destination there's probably nothing else as fast. If you need to do frequent updates and always use the same command, hopefully next time the timestamps will be as cp -u expects them.

Nemo

Posted 2012-06-07T17:58:23.610

Reputation: 1 050

Does the timestamp have time shift info in folder entry? Just curious... – pbies – 2014-11-02T19:08:03.740