How do I get an entire directory in sftp?

201

42

I'm just learning to use sftp, and I want to copy a directory from the remote computer to my local computer. If I try

get [directory]

it gives me this error:

Cannot download non-regular file: /home/mpirocch/Documents

get -R doesn't work, either.

Matthew

Posted 16 years ago

Reputation: 11 686

Wow, sftp is basically useless as is. lftp helps a lot.

– Eric Duminil – 6 years ago

2Recursive put/get was added to OpenSSH in 5.4, which was released in 2010. You may consider upgrading your servers as it also adds a lot of bug fixes and better security features. – Lèse majesté – 13 years ago

Answers

282

Use the -r (recursive) flag:

get -r *

mani-fresh

Posted 16 years ago

Reputation: 2 944

For some reason uploading recursively does not work with put -r * it can't create directories. However for me the lftp solution below worked just fine. – Ciantic – 11 years ago

Anyone know the way to get -r to give directories write permissions? Everytime I try -r on a read only directory, it will not download it because it creates the directory locally read only as well. – Rahly – 10 years ago

Not working with openssh-clients-5.3p1-111.el6.x86_64 on 2.6.32-573.12.1.el6.x86_64 #1 SMP Tue Dec 15 08:24:23 CST 2015 x86_64 x86_64 x86_64 GNU/Linux – user637338 – 10 years ago

1

more specifically mget -r ./* to get all files and folders in the current directory. works in MINGW64 too. https://kb.wisc.edu/cae/page.php?id=32991

– SashikaXP – 8 years ago

18Thank you for putting an answer that actually answers the question. – aVeRTRAC – 14 years ago

30Gives me: get: Invalid flag -r – ghbarratt – 13 years ago

2This also gives me Invalid flag. I'm not sure this is supported on all systems. – Ben – 13 years ago

@Ben: When I asked this question in 2009, the -r flag did not exist. Is your system up to date? – Matthew – 13 years ago

It works on all current systems. Just tested it. – buschtoens – 12 years ago

15@silvinci not all current systems. Illegal option with CentOS 6.4 openssh-clients-5.3p1-84.1.el6.x86_64 – CrackerJack9 – 12 years ago

52

Use:

scp -r mpirocch@my-server:/home/mpirocch/Documents Documents

Cristian Ciupitu

Posted 16 years ago

Reputation: 4 515

6Some systems may allow sftp, but not scp access. – Turion – 8 years ago

This doesn't answer the question. – Eric Duminil – 6 years ago

@EricDuminil have you seen the date? Unfortunately back then this was a good alternative, but if you want to be pedantic, yeah it doesn't answer the question and maybe deserves a downvote. – Cristian Ciupitu – 6 years ago

@CristianCiupitu: It's been a while indeed, but many shared hosting servers still only allow sftp. scp simply isn't an alternative in this case. – Eric Duminil – 6 years ago

@EricDuminil then take the neutral approach and don't upvote or downvote? – Cristian Ciupitu – 6 years ago

Please don't take downvotes personally. The description on the downvote icon is "This answer is not useful". And well, scp isn't useful for a question asking specifically about sftp. – Eric Duminil – 6 years ago

4This works, but it would be nice to be able to do this from SFTP (so I can see what I copy before I copy it). Is this possible? – Matthew – 16 years ago

@Matthew: I don't think so. I have been asking myself the same question for some time and the best answer that I have for the moment is scp -r. Or you could use an advanced SFTP client like Filezilla. – Cristian Ciupitu – 16 years ago

1scp -r will follow symlinks... – jsleuth – 13 years ago

42

Use lftp:

lftp sftp://user@host

Then, within lftp, cd into the directory you want to copy, and use the mirror command to recursively download the selected directory, like this:

mirror

This command accepts options and arguments:

mirror [OPTIONS] [source [target]]

For example, the -R (or --reverse) option will cause it to upload the local directory tree to the remote directory:

mirror -R

See the lftp(1) man page at the project’s site or at Debian.org for other commands and options.

bshanks

Posted 16 years ago

Reputation: 549

5+1 lftp is like a little bundle of magic and joy! – WCWedin – 14 years ago

17

well this little guide should help, mirror a remote server to local folder with lftp

lftp sftp://user:password@server.org:22 -e 'mirror --verbose --use-pget-n=8 -c /remote/path /local/path'

  • sftp:// = uses SFTP protocol
  • mirror = mirror mode
  • verbose = shows the files being downloaded
  • use-pget-n = number of segments, realy useful to speed up big files
  • parallel = downloads multiplier files at the same time

if you want to download files in parallel switch out use-pget-n=8 with --parallel=8

hope this helps anyone needing to mirror a remote folder to a local folder

nwgat

Posted 16 years ago

Reputation: 961

Thanks, this worked for me, after the other methods failed (in particular no "-r" option on sftp). The first part of the command in my case needed to be quoted, due to dollar characters in the password lftp 'sftp://user:pa$$word@server.org:22' -e 'mirror --verbose --use-pget-n=8 -c /remote/path /local/path' – Jason – 11 years ago

Excellent, thanks. I wanted to copy from local to server, so I used mirror -R ... – Eric Duminil – 6 years ago

15

Don't use the sftp program directly if you can find something better. For Linux, many file managers (at least Nautilus and Dolphin, the GNOME and KDE ones) support sftp natively, and there's always sshfs. For windows, there's WinSCP, and probably others. The point of all of these is to let you access files over sftp as if they were on a regular filesytem, so you don't have to care that you're accessing them over sftp.

Ryan C. Thompson

Posted 16 years ago

Reputation: 10 085

1I've had performance issue with Nautilus SFTP. Yeah, it's really convenient but I wasn't able to saturate 1 Gbit/s network via Nautilus, while with plain sftp/ssh command line interface I was (cca 100 MB/s file transfers). So the issue was Nautilus on GNU/Linux Mint. – stamster – 8 years ago

Wow, nautilus does work very well with SFTP. I added a bookmark in Nautilus, and now it's all ridiculously easy. Thanks! – Matthew – 16 years ago

14

Try mget instead of get.

Clarification: mget will work if you are inside the directory you want to copy; if you do something like this:

sftp> cd dir_to_get
sftp> mget *

it will get all the files in that directory. However, it will not recursively get the contents of any subdirectories.

Ken Keenan

Posted 16 years ago

Reputation: 305

1getting Cannot download non-regular file with this – Dchris – 10 years ago

1This worked for me on a system where 'get -r' didn't work. – Tchotchke – 9 years ago

1mget doesn't work any better than get for me. – Matthew – 16 years ago

10

get -r [directory]

gets [directory] and everything under it, where r stands for recursive. I found this just by typing help from sftp.

drkvogel

Posted 16 years ago

Reputation: 271

1IMO this is the best answer to this question. :) – ramrunner – 13 years ago

3

This is a duplicate of an answer from Nov 29 '10 (almost one year and a half ago).

– Cristian Ciupitu – 12 years ago

4

As with cp:

scp -rp user@host:/path/to/dir dir

The above will preserve times and modes of the original files and subdirectories. This is especially useful for the retrieval of backups.

Aziraphale

Posted 16 years ago

Reputation: 159

This is especially useless if only sftp is allowed. – Eric Duminil – 6 years ago

This is basically a duplicate of an answer from Nov 29 '10 (almost two years ago).

– Cristian Ciupitu – 12 years ago

0

I have Java dist folder in remote server, where i have following tree:

- dist
--- Audio.jar
--- README
--- lib
----- lib.jar

Goal is: I want to use SFTP? And put them in /tmp/<>

Step 1. sftp remoteuser@ip

Step 2. cd /var/tmp

Step 2. lmkdir /tmp/dist; lmkdir /tmp/dist/lib

Step 3. lcd /tmp/dist

Step 4. mget *

Step 5. lcd /tmp/dist/lib

Step 6. mget *

Step 7. finally i have my goal

$ ls
Audio.jar  lib  README.TXT

YumYumYum

Posted 16 years ago

Reputation: 1 399