How do I get an entire directory in sftp?

201

42

I'm just learning to use sftp, and I want to copy a directory from the remote computer to my local computer. If I try

get [directory]

it gives me this error:

Cannot download non-regular file: /home/mpirocch/Documents

get -R doesn't work, either.

Matthew

Posted 2009-09-12T21:39:59.043

Reputation: 11 686

Wow, sftp is basically useless as is. lftp helps a lot.

– Eric Duminil – 2019-12-10T14:40:25.440

2Recursive put/get was added to OpenSSH in 5.4, which was released in 2010. You may consider upgrading your servers as it also adds a lot of bug fixes and better security features. – Lèse majesté – 2012-09-28T04:50:01.127

Answers

282

Use the -r (recursive) flag:

get -r *

mani-fresh

Posted 2009-09-12T21:39:59.043

Reputation: 2 944

For some reason uploading recursively does not work with put -r * it can't create directories. However for me the lftp solution below worked just fine. – Ciantic – 2014-12-22T08:24:09.647

Anyone know the way to get -r to give directories write permissions? Everytime I try -r on a read only directory, it will not download it because it creates the directory locally read only as well. – Rahly – 2015-11-05T00:52:44.027

Not working with openssh-clients-5.3p1-111.el6.x86_64 on 2.6.32-573.12.1.el6.x86_64 #1 SMP Tue Dec 15 08:24:23 CST 2015 x86_64 x86_64 x86_64 GNU/Linux – user637338 – 2016-02-26T10:54:37.493

1

more specifically mget -r ./* to get all files and folders in the current directory. works in MINGW64 too. https://kb.wisc.edu/cae/page.php?id=32991

– SashikaXP – 2018-01-25T04:09:47.373

18Thank you for putting an answer that actually answers the question. – aVeRTRAC – 2012-01-19T19:56:47.713

30Gives me: get: Invalid flag -r – ghbarratt – 2012-07-18T21:21:11.917

2This also gives me Invalid flag. I'm not sure this is supported on all systems. – Ben – 2012-08-28T07:35:05.720

@Ben: When I asked this question in 2009, the -r flag did not exist. Is your system up to date? – Matthew – 2012-12-10T16:24:50.920

It works on all current systems. Just tested it. – buschtoens – 2013-07-25T10:03:32.410

15@silvinci not all current systems. Illegal option with CentOS 6.4 openssh-clients-5.3p1-84.1.el6.x86_64 – CrackerJack9 – 2013-11-05T01:25:07.643

52

Use:

scp -r mpirocch@my-server:/home/mpirocch/Documents Documents

Cristian Ciupitu

Posted 2009-09-12T21:39:59.043

Reputation: 4 515

6Some systems may allow sftp, but not scp access. – Turion – 2017-08-04T09:41:48.703

This doesn't answer the question. – Eric Duminil – 2019-12-10T14:20:24.350

@EricDuminil have you seen the date? Unfortunately back then this was a good alternative, but if you want to be pedantic, yeah it doesn't answer the question and maybe deserves a downvote. – Cristian Ciupitu – 2019-12-10T15:22:44.753

@CristianCiupitu: It's been a while indeed, but many shared hosting servers still only allow sftp. scp simply isn't an alternative in this case. – Eric Duminil – 2019-12-10T15:38:37.520

@EricDuminil then take the neutral approach and don't upvote or downvote? – Cristian Ciupitu – 2019-12-10T17:24:21.807

Please don't take downvotes personally. The description on the downvote icon is "This answer is not useful". And well, scp isn't useful for a question asking specifically about sftp. – Eric Duminil – 2019-12-10T17:54:38.020

4This works, but it would be nice to be able to do this from SFTP (so I can see what I copy before I copy it). Is this possible? – Matthew – 2009-09-12T22:53:20.017

@Matthew: I don't think so. I have been asking myself the same question for some time and the best answer that I have for the moment is scp -r. Or you could use an advanced SFTP client like Filezilla. – Cristian Ciupitu – 2009-09-12T23:53:18.867

1scp -r will follow symlinks... – jsleuth – 2013-02-13T15:57:06.877

42

Use lftp:

lftp sftp://user@host

Then, within lftp, cd into the directory you want to copy, and use the mirror command to recursively download the selected directory, like this:

mirror

This command accepts options and arguments:

mirror [OPTIONS] [source [target]]

For example, the -R (or --reverse) option will cause it to upload the local directory tree to the remote directory:

mirror -R

See the lftp(1) man page at the project’s site or at Debian.org for other commands and options.

bshanks

Posted 2009-09-12T21:39:59.043

Reputation: 549

5+1 lftp is like a little bundle of magic and joy! – WCWedin – 2011-04-27T13:32:24.823

17

well this little guide should help, mirror a remote server to local folder with lftp

lftp sftp://user:password@server.org:22 -e 'mirror --verbose --use-pget-n=8 -c /remote/path /local/path'

  • sftp:// = uses SFTP protocol
  • mirror = mirror mode
  • verbose = shows the files being downloaded
  • use-pget-n = number of segments, realy useful to speed up big files
  • parallel = downloads multiplier files at the same time

if you want to download files in parallel switch out use-pget-n=8 with --parallel=8

hope this helps anyone needing to mirror a remote folder to a local folder

nwgat

Posted 2009-09-12T21:39:59.043

Reputation: 961

Thanks, this worked for me, after the other methods failed (in particular no "-r" option on sftp). The first part of the command in my case needed to be quoted, due to dollar characters in the password lftp 'sftp://user:pa$$word@server.org:22' -e 'mirror --verbose --use-pget-n=8 -c /remote/path /local/path' – Jason – 2014-11-09T22:57:00.630

Excellent, thanks. I wanted to copy from local to server, so I used mirror -R ... – Eric Duminil – 2019-12-10T14:29:14.773

15

Don't use the sftp program directly if you can find something better. For Linux, many file managers (at least Nautilus and Dolphin, the GNOME and KDE ones) support sftp natively, and there's always sshfs. For windows, there's WinSCP, and probably others. The point of all of these is to let you access files over sftp as if they were on a regular filesytem, so you don't have to care that you're accessing them over sftp.

Ryan C. Thompson

Posted 2009-09-12T21:39:59.043

Reputation: 10 085

1I've had performance issue with Nautilus SFTP. Yeah, it's really convenient but I wasn't able to saturate 1 Gbit/s network via Nautilus, while with plain sftp/ssh command line interface I was (cca 100 MB/s file transfers). So the issue was Nautilus on GNU/Linux Mint. – stamster – 2018-01-25T22:15:14.947

Wow, nautilus does work very well with SFTP. I added a bookmark in Nautilus, and now it's all ridiculously easy. Thanks! – Matthew – 2009-09-13T14:13:29.413

14

Try mget instead of get.

Clarification: mget will work if you are inside the directory you want to copy; if you do something like this:

sftp> cd dir_to_get
sftp> mget *

it will get all the files in that directory. However, it will not recursively get the contents of any subdirectories.

Ken Keenan

Posted 2009-09-12T21:39:59.043

Reputation: 305

1getting Cannot download non-regular file with this – Dchris – 2015-08-24T09:04:07.887

1This worked for me on a system where 'get -r' didn't work. – Tchotchke – 2016-09-20T19:47:28.097

1mget doesn't work any better than get for me. – Matthew – 2009-09-12T22:53:57.893

10

get -r [directory]

gets [directory] and everything under it, where r stands for recursive. I found this just by typing help from sftp.

drkvogel

Posted 2009-09-12T21:39:59.043

Reputation: 271

1IMO this is the best answer to this question. :) – ramrunner – 2013-01-24T18:28:00.900

3

This is a duplicate of an answer from Nov 29 '10 (almost one year and a half ago).

– Cristian Ciupitu – 2014-01-04T19:43:55.210

4

As with cp:

scp -rp user@host:/path/to/dir dir

The above will preserve times and modes of the original files and subdirectories. This is especially useful for the retrieval of backups.

Aziraphale

Posted 2009-09-12T21:39:59.043

Reputation: 159

This is especially useless if only sftp is allowed. – Eric Duminil – 2019-12-10T14:19:44.933

This is basically a duplicate of an answer from Nov 29 '10 (almost two years ago).

– Cristian Ciupitu – 2014-01-04T19:47:22.160

0

I have Java dist folder in remote server, where i have following tree:

- dist
--- Audio.jar
--- README
--- lib
----- lib.jar

Goal is: I want to use SFTP? And put them in /tmp/<>

Step 1. sftp remoteuser@ip

Step 2. cd /var/tmp

Step 2. lmkdir /tmp/dist; lmkdir /tmp/dist/lib

Step 3. lcd /tmp/dist

Step 4. mget *

Step 5. lcd /tmp/dist/lib

Step 6. mget *

Step 7. finally i have my goal

$ ls
Audio.jar  lib  README.TXT

YumYumYum

Posted 2009-09-12T21:39:59.043

Reputation: 1 399