scp all files ending '-123.jpg' - recursion, spaces, wildcards …

Question

There's too much going on for me to get my noobie head around here. I'm wanting to download all files that end -123.jpg from a multitude of nested directories on a remote server. Some of these directories have spaces in their names. I'm thinking that the command should be along the lines of:

scp -r user@server:/path/to/parent\ directory/*/*/*123\.jpg ./

… where "parent\ directory" is a directory name with a space, and the specified path goes as deep as it can before it splits off to various sub-directories, for example dir/sub dir/[uniquely_id]-123.jpg file. (Note that these sub-directories often contain spaces too, should that affect the * wildcard)

I'm getting 'no match' returned for this, or 'no such file or directory' if I meddle with the space escaping. I'm thinking therefore that it's the recursion or the wildcard that I've got wrong.

Thanks in advance.

Answer 1

Lacking your server layout for testing I don't know if the following command actually works, but I think rsync would be better than scp in that case, as you can specify in- and exclude patterns. Something like:

rsync -arvzSH  --include "*123.jpg" --exclude "*" user@server:/path/to/parent\ directory/ /path/to/target directory/

may work. But you better double check the rsync documentation.

Answer 2

Filename escaping with scp is tricky; your arguments get (re)expanded on the remote side. "Interesting" characters have to be double quoted, to avoid special handing by your local shell and by the remote. I would expect both the following to work for you:

scp user@server:'/path/to/parent\ directory/*/*/*123.jpg' ./
scp user@server:'"/path/to/parent directory"/*/*/*123.jpg' ./

If that's confusing, rsync is pretty easy to use (as lothar mentions), and piping tar or cpio through ssh is pretty easy too.

ssh user@server 'cd /path/to/parent\ directory;
                 find -name "*123.jpg" -print0 | cpio -0 -o' |
    cpio -i -d -v

Answer 3

This properly should be done in two parts:

1. first find all needed files and copy them to one folder (this does not work if you have the same filenames) - you may do it with one find and -exec
2. then send a bunch of files using scp

scp is not enough advanced to copy folders structure and there is no single command to make a copy of a folders tree.

Also, if you need all files in folders, as you will not be able to select them by wildcards, and you want to preserve folders structure - you may want to use mc = Midnight Commander and its "Shell link".

Answer 4

A work around for scp with wildcards:

for i in "*-123.jpg"
do 
    scp $i <user>@<server>:/path/to/destination/
done

Refer: https://community.hpe.com/t5/System-Administration/How-to-scp-mutiple-files-using-wild-chars-or-in-a-single-scp/m-p/3639440/highlight/true#M238959