How to copy files on unix shell using wildcards

I'm trying to copy some files into existing folders. This is an example of the schema

# origin 
/home/user/myfiles/folder1/group1/1.jpg
/home/user/myfiles/folder1/group1/2.jpg
/home/user/myfiles/folder1/group2/1.jpg
/home/user/myfiles/folder1/group2/2.jpg
/home/user/myfiles/folder2/group1/1.jpg
/home/user/myfiles/folder2/group1/2.jpg
/home/user/myfiles/folder2/group2/1.jpg
/home/user/myfiles/folder2/group2/2.jpg

The destination folder schema already exists, its just the origin files (jpgs) that are not there.

My question is how to do this?

I would like to end up with something like below, but this is not working. Maybe I should include regex in the query to separate asterisk into values and use as origin/destination?

 cp /home/user/myfiles/*/*/*.jpg  /home2/user/myfiles/*/*/*.jpg

My original intend is to use this copying schema on my unanswered question: s3cmd: how to copy files from local to S3 bucket with existing structure as a possible solution.

Martin

Posted 2012-07-25T23:26:15.143

Reputation: 339

Are the destination folders completely empty? If so just delete them and copy files and directory structure. Also, the shell expansion of the second set of asterixes is your problem. – Hennes – 2012-07-25T23:34:51.037

They're not. They have other type of files that I still need. – Martin – 2012-07-25T23:42:39.397

Can you elaborate on your last sentence? – Martin – 2012-07-25T23:43:09.660

I can, but I am still trying to wrap my head around what your precise goal is. The cp command has the same source and destination. The s3cmd question identified local dev and remote s3 site, that I could understand. But the current question seems to be a NOP. (Granted, that might be because I am sleepy). – Hennes – 2012-07-26T00:04:23.277

@Hennes Sorry for misleading you, just fixed the question. The dest should be different than origin! – Martin – 2012-07-26T01:42:22.480

Answers

With GNU cp you can use the --parents option, like this:

cd /home/user/myfiles ; cp --parents */*/*.jpg /home2/user/myfiles/

You have to cd first because if you do cp --parents /home/user/myfiles/... it will create those parents too, and you'll end up with /home2/user/myfiles/home/parents/myfiles/.

If you need a non-GNUish answer, the traditional unix method would be tar over a pipe.

cd /home/user/myfiles ; tar cf - */*/*.jpg | ( cd /home2/user/myfiles ; tar xf - )

The second cd is in a subshell with the second tar, so that's where the files get extracted. The tarball itself is created, sent over the pipe, and extracted all at the same time. Add a v flag to one of the tars if you want progress reporting.

Alan Curry

Posted 2012-07-25T23:26:15.143

Reputation: 2 354

The following should do the trick:

for i in `find /home/user/myfiles/folder1 -print`; do 
  if [ ! -d $i ]; 
    then filePath=`echo $i | cut -f 4- -d"/"`; 
         cp $i /home/user/myfiles/folder2/$filePath; 
  fi; 
done

Basically, I am extracting the part of the filename that is going to change between the two folders, and then issuing a copy command.

Notice the "cut -f 4-" above - it gets the fields from the 4th one onwards. You can change the number as per your wish

Perhaps even basename or a simpler command exists... but this will do for now :)

Sudipta Chatterjee

Posted 2012-07-25T23:26:15.143

Reputation: 308