UNIX shell scripting: how to recursively move files up one directory?

I have a large number of small files, f, arranged in a directory structure like so:

/A/B/C/f

There are 11 directories at the level of A, each with about 100 directories at the level of B, each with about 30 directories at the level of C, each with one file f.

How do I move all files f up one level? For example, given this set of files...

/A/B/C/f1
/A/B/C/f2
/A/B/C/f3
/A/B/C/f4

I want the directory /A/B/ to contain 4 files, f1 through f4. Removing directories C is not necessary.

I'm hoping this is a solved problem, possibly involving find, xargs, and whatnot. Any ideas?

Cheers,

James

jl6

Posted 2010-09-12T22:02:09.770

Reputation: 1 025

Answers

It's fairly simple with GNU find (as found on Linux) or any other find that supports -execdir:

find A -type f -execdir mv -i {} .. \;

With a standard find:

find A -type f -exec sh -c 'mv -i "$1" "${1%/*}/.."' sh {} \;

With zsh:

zmv -Q -o-i 'A/(**/)*/(*)(.)' 'A/$1$2'

If the directory structure always has the same nesting level, you don't need any recursive traversal (but remove empty directories first):

for x in */*; do; echo mv -i "$x"/*/* "$x"/..; done

Gilles 'SO- stop being evil'

Posted 2010-09-12T22:02:09.770

Reputation: 58 319

For that set of files, this would do:

$ cd /A/B/C/
$ mv ./* ../

But I'm expecting that your problem is somewhat more complicated... I can't answer to this... I'm not quite sure how your dir structure is... Could you clarify?

BloodPhilia

Posted 2010-09-12T22:02:09.770

Reputation: 27 374

This will obviously work for any one directory /A/B/C, but as I have about 30000 such directories, I am hoping for a command that can be executed at the top of the hierarchy to take care of them all at once. – jl6 – 2010-09-12T22:20:14.580

1@jl6 I see, and only files need moving? So C doesn't have to move to B, and B not to A, etc... – BloodPhilia – 2010-09-12T22:32:43.953

That's correct - the operation should only be performed on files, at the lowest level of the hierarchy, and they should only be moved one level up. – jl6 – 2010-09-12T23:00:54.107

My first guess is

$ find A -type f -exec mv {} .. \;

as long as you don't specify -depth it should be ok. I have NOT tried it so test it first before you commit to it.

hotei

Posted 2010-09-12T22:02:09.770

Reputation: 3 645

What does find do when there are two files wit the same name? – None – 2010-09-12T22:26:44.463

Find isn't the problem, the imbedded "mv" command is. Dupe names would not be good as mv is not going to rename, it's going to overwrite. You could use mv -i to help a little, but with 30,000+ files that might be painful. If you need to have that much control over the situation you may need a program written in a scripting language (my choice would be python, YMMV). – hotei – 2010-09-12T22:30:35.373

I haven't used "backups" with mv, but looking at the man page it looks like that might be a solution to your problem of dupe names. "find A -type f -exec mv --backup numbered {} .. ;" might work (note that's a double dash in --backup) – hotei – 2010-09-12T22:37:41.637

The find example given will move all files at the lowest level to a directory one above A (I believe the ".." is getting interpreted by the shell before being passed to find/mv?). I need the files to move just one level up in the hierarchy. One further clarification: there will be no duplicate filenames, as long as the files only move ONE level up. – jl6 – 2010-09-12T22:57:01.140

@hotei: The mv commands are executed in the current directory, so .. doesn't do what you hope. See my answer for solutions. @jl6: the shell has nothing to do with .., that's handled by the kernel.

– Gilles 'SO- stop being evil' – 2010-09-12T23:03:46.667

@Gilles: Good catch. I learn something every time I visit this site! – hotei – 2010-09-12T23:25:07.987

If you only want to move files that are in leaf directories (i.e. you don't want to move /A/B/file to /A if B contains subdirectories) then here are a couple of ways to do that:

Both require this

leaf ()
{
    find $1 -depth -type d | sed 'h; :b; $b; N; /^\(.*\)\/.*\n\1$/ { g; bb }; $ {x; b}; P; D'
}
shopt -s nullglob

This one works:

leaf A | while read -r dir
do
    for file in "$dir"/*
    do
        parent=${dir%/*}
        if [[ -e "$parent/${file##*/}" ]]
        then
            echo "not moved: $file"
        else
            mv "$file" "$parent"
        fi
    done
done

This will be faster, but it doesn't like empty source directories:

leaf A | while read -r dir
do
    mv -n "${dir}"/* "${dir%/*}"
    remains=$(echo "$dir"/*)
    [[ -n "$remains" ]] && echo "not moved: $remains"
done

Paused until further notice.

Posted 2010-09-12T22:02:09.770

Reputation: 86 075