Copy all files and folders excluding subversion files and folders on OS X

14

5

I'm trying to copy all files and folders from one directory to another, but exclude certain files. Specifically, I want to exclude subversion files and folders. However, I'd like a general yet concise solution.

I imagine I'll find the need to exclude several types of files in the near future. For example, I might want to exclude .svn, *.bak, and *.prj.

Here is what I've put together so for, but it is not working for me. The first part, find works, but I'm doing something wrong with xargs and cp. I tried cp with and without the -R. Also, I'm using OS X and it appears to have a less featured version of xargs than linux systems.

find ./sourcedirectory -not \( -name .svn -a -prune \)
     | xargs -IFILES cp -R FILES ./destinationdirectory

Michael Prescott

Posted 2009-09-28T14:23:16.367

Reputation: 3 351

I may be wrong, but I think that this is trickier than you think. Even if your find command properly uses -prune to exclude .svn items, you then pass the -R flag to cp which tells that command to be recursive. When it does so, you lose whatever granularity you had in the find command. I'm going to tinker with this for a minute, but I think the answer is not to use -R in the cp command. – Telemachus – 2009-09-28T14:34:35.100

It seems to work for me on a Linux system. Can you be more specific as to what "it is not working" means? Any error messages? Files getting/not getting copied that you expect? – Paused until further notice. – 2009-09-28T15:03:17.890

It's the -R flag, I'm pretty sure. Remove that and you should be fine (though you might want to add -mindepth 1 to ignore the toplevel directory folder which you don't want to copy, I assume) – Telemachus – 2009-09-28T15:22:42.227

I usually do these things by copying everything, and then deleting the unwanted files in the target directories. That's often much simpler. – Jan Doggen – 2014-05-19T19:52:45.087

Answers

22

(Edited after re-reading the question. Questioner says rsync is not installed)

A possible problem with your find/xargs solution is spaces in the filenames. To get around that, tell find and xargs to use a null character (ASCII 0) to separate the found files:

find ./sourcedirectory -not ( -name .svn -a -prune ) -print0 | xargs -0 -IFILES cp FILES ./destinationdirectory

If you find that rsync is available, I still think that rsync is the far better solution:

Use rsync with the -C option. From the rsync man page:

This is a useful shorthand for excluding a broad range of files that you often don't want to transfer between systems. It uses a similar algorithm to CVS to determine if a file should be ignored.

This will tell rsync to ignore these patterns:

RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS .make.state .nse_depinfo *~
#* .#* ,* _$* *$ *.old *.bak *.BAK *.orig *.rej .del-* *.a *.olb *.o *.obj 
*.so *.exe *.Z *.elc *.ln core .svn/ .git/ .bzr/

For example:

rsync -avC /path/to/source/directory /path/to/destination/directory

(note: If you're not too familiar with rsync yet, be sure to read on that man page about how rsync deals with a trailing slash in the source path. It behaves differently if you include the slash than if you don't. Search for 'trailing slash')

Doug Harris

Posted 2009-09-28T14:23:16.367

Reputation: 23 578

-C flag solved so much problems i was having... nothing like reading the whole man page. thanks – gcb – 2011-11-09T08:25:27.347

Oh rats, I just re-read your question and saw that you said you don't have rsync installed. On my MacBook Pro (OS X 10.6.1) it's in /usr/bin/rsync. It was installed for me under Tiger (10.4) and Leopard (10.5) also. – Doug Harris – 2009-09-28T15:06:16.047

I'm pretty sure that you (and the OP) don't want the -R flag in the xargs part of the command. – Telemachus – 2009-09-28T15:13:44.987

Good catch, I copied that from the original question. Will edit now. – Doug Harris – 2009-09-28T15:18:49.893

1Thanks Doug! One thing I preach, "use the right tool for the job". I've recently come from Windows world to OSX and am still fumbling around in ignorance. I posted this same question in a linux channel and someone quickly said, just use "rsynch" I typed "rsynch" in the terminal and saw it didn't exist and continued investigating the find | xargs approach. I'm a bit stubborn like that. Anyhow, OSX does have "rsync" by default, and your post was very helpful. I don't have enough experience here yet to know which is better, but rsync sure is much more concise. Thanks! – Michael Prescott – 2009-09-29T13:50:59.023

If you'll be doing any work with linux machines in addition to your work on OS X, I think you'll find it worth your time to learn how to use rsync. Its primary functionality is intelligent copying of just the changed stuff (like robocopy on windows, if you're familiar with that). Because it copies just the delta, it's a great way to handle (non-Time Machine) backups, code deployments and the like. – Doug Harris – 2009-09-29T14:53:55.243

2

Not a general solutions but... you can use the svn export command to create a copy of the workspace without the .svn metadata folders.

Chris Nava

Posted 2009-09-28T14:23:16.367

Reputation: 7 009

Not to be offensive, but I'm not sure why this answer is getting up-voted. I'm aware of svn's capabilities, but "I'd like a general yet concise solution. I imagine I'll find the need to exclude several types of files in the near future" – Michael Prescott – 2009-09-29T13:55:29.127

2

%> mkdir -p FOLDER_OUT && ( tar cf - FOLDER_OR_FILES_IN --exclude=.svn  | tar xvf - -C FOLDER_OUT )

if you want you can even put 'pv' or something similar in between the 2 tar processes.

akira

Posted 2009-09-28T14:23:16.367

Reputation: 52 754

2

A dirty but quick and concise way:

cp -r source destination
find destination -iname .svn |xargs rm -rf

This copies one directory to another (thus the recursive option -r) and then recursively wipes out anything named .svn (ignoring case).

michele

Posted 2009-09-28T14:23:16.367

Reputation: 21

1

I would go about it a different way, using tar and its exclude mechanism.

From in the destination directory:

tar -X excludefile -C source -f - . | tar xf -

This will cd to source, tar up the contents, excluding what is listed in excludefile, and then untar it to the current directory.

KeithB

Posted 2009-09-28T14:23:16.367

Reputation: 8 506

Elegant solution indeed. – Nick Stinemates – 2009-09-28T20:59:37.820

well, it's the same answer i gave, just later. plus you have to be in the destination directory... :) – akira – 2009-09-30T05:08:11.543

0

Edited answer: The problem is that -R makes your copying recursive and so you end up copying the hidden files. Here is what I would use:

find source/  -mindepth 1 -not \( -name .svn -prune \) | xargs -Iitem cp item target/

The -mindepth 1 flag tells find to ignore the toplevel directory. Since you want to copy all of the content of that directory into a new toplevel directory, I assume you don't want that.

As Chris Nava says in his answer, there's already a built-in way to do this if we're talking about SVN folders, but since you asked for a more general solution, this may help a bit.

Telemachus

Posted 2009-09-28T14:23:16.367

Reputation: 5 695

Thanks Telemachus, it is helpful. I don't have the experience to critique, but I'll repeat what I've been told since my original post. "xargs is broken" The unknown irc commenter that told me that inspired me to look around a bit more and I think Doug Harris's answer addresses the problem. That tell xargs to use null characters. I think that is the -0 switch? – Michael Prescott – 2009-09-29T14:08:02.040

@Michael: xargs isn't broken, but on Unix-like systems, the default is not to use filenames (or directory names) with spaces in them. If you have spaces or "funny" characters in your filenames (or directory names), then you have to do extra work to deal with that. (In GNU find, there is a whole section in the man page called "UNUSUAL FILENAMES" because of this issue.) The -0 flag in xargs and -print0 for find help to handle these problems. I promise you, however, that you don't want to use -R in your copy command. It will undo whatever you do to avoid the SVN directories. – Telemachus – 2009-09-29T14:42:21.547

0

I suppose it depends how big your tree is, but why not just copy everything first, then trim the .svn folders after:

find /dest-dir -type d -name .svn -exec rm -rf {} \;

?

Steve Folly

Posted 2009-09-28T14:23:16.367

Reputation: 5 293

Without any benchmarking, my initial thought is that this is a double waste of CPU cycles: the copying and the removing. It may take a little extra human time to craft the right find command, but you only do that once. You may use the command hundreds of times, once you get it right. – Telemachus – 2009-09-28T16:38:40.500

@Telemachus - true, but that's what computers are (supposed to be) good at - doing the tricky stuff so we don't have to! Really - what harm is there in copying some files only to have them deleted soon after, if it means the commands you invent to do it are very simple? – Steve Folly – 2009-09-28T22:30:59.150

@Steve: there's no harm, really. As far as it goes, this is a fine solution. It follows one principle I like: "Do the simplest thing possible that works." On the other hand, it violates another principle I like even more: "Learn your tools." I would prefer to learn how to use find itself better, so that I didn't have to do this. But you're right: there's nothing really wrong with this solution. – Telemachus – 2009-09-29T12:01:20.877

@Telemachus: I do agree with "Learn your tools". When I started out, I'm sure my find command above would have looked very cryptic to me then :-) – Steve Folly – 2009-09-30T18:44:21.837

0

Don't forget grep -v

find . | grep -v .svn

Nick Stinemates

Posted 2009-09-28T14:23:16.367

Reputation: 388

This command doesn't do what you think it's doing. – Juan A. Navarro – 2011-04-05T15:28:18.633

0

You can also do the opposite. Copy everything then delete .svn folders using the below command:

find . | grep ".svn" | xargs rm -rf

devXen

Posted 2009-09-28T14:23:16.367

Reputation: 101