scp a single file to multiple locations

Can you scp a file to multiple locations in the same command?

For example:

scp file.txt user@ip-1.com:, user@ip2.com:

Or would it be more practical to create a bash script that has all the hosts already in it and it would just take a file as the argument?

Andrew

Posted 2011-09-22T20:29:48.967

Reputation: 405

Answers

Let's say you have a file (destfile.txt) with user@host-values, one on each line. Then you could do like this:

while IFS= read -r dest; do
  scp ourfile.txt "$dest:remote/path/"
done <destfile.txt

Kusalananda

Posted 2011-09-22T20:29:48.967

Reputation: 1 941

I'm doing the former, but with rsync and some of my nodes are not updating. Any idea why? – Soubriquet – 2018-09-22T17:50:01.413

This works! Still a handful to type out though. Perhaps there's a way to simplify that. – Andrew – 2011-09-22T21:02:08.503

2@Andrew, if you change the for-loop into while read dest; do, it will read from standard input. Put it in a script and feed the destfile.txt into it (e.g., ./thescript.sh <destfile.txt). – Kusalananda – 2011-09-22T21:19:16.507

Looks like a job for parallel-scp(n)(t) - this implements a set of commands that allow for scp commands to be run on multiple systems at once. It will allow for the copying of files in parallel to a set of machines.

AnonymousLurker

Posted 2011-09-22T20:29:48.967

Reputation: 775

cat file.txt | tee >(ssh user@ip1.com "cat > file.txt") \
                   >(ssh user@ip2.com "cat > file.txt")

tar cz file1 file2 file3 | tee >(ssh user@ip1.com "tar xz") \
                               >( ... )

user1686

Posted 2011-09-22T20:29:48.967

Reputation: 283 655

Another alternative (and a one-liner) would be to instead use pdsh to connect to each target node and trigger a get from there:

pdsh -w^destfile.txt scp hostname:/path/to/file /path/to/destfile

This of course requires one more information (the local host) and different user rights, but you avoid looping in bash and using indirection to read the file.

Rick Moritz

Posted 2011-09-22T20:29:48.967

Reputation: 101

Here's another option, with a one line shell script.

cscp.sh 20337.patch < hosts.txt

It uses two files, one for the loop, and one for the server host list. It reads $1 for the first parameter from the CLI as the filename to SCP

cscp.sh

#!/bin/bash
while read host; do
  scp $1 ${host}:
done

hosts.txt

project-prod-web1
project-prod-web2
project-prod-web3

Usage

Copy file to multiple hosts:

cscp.sh file < hosts

David Thomas

Posted 2011-09-22T20:29:48.967

Reputation: 181

If you have a consistent naming convention going on for multiple servers, you can do something neat like:

for x in st1-abc-{11..20}.acme.com; do scp filez.tgz user@$x; done

deed02392

Posted 2011-09-22T20:29:48.967

Reputation: 2 662

You can also use xargs.

hosts.txt :

user@ip-1.com:
user@ip2.com:

Then run this command :

cat hosts.txt | xargs -I HOST scp file.txt HOST

Dgief

Posted 2011-09-22T20:29:48.967

Reputation: 1