0

I use wget -m to mirror my webspace to my nas. So far it's working great, but I noticed that when a file on the server got deleted, it will be still there locally.

Is there any way to delete the local files that aren't existing anymore on the server?

In case of the backup, that won't be a problem, because after wget I zip the files with a timestamp with tar.

I found some old forum posts online talking about this missing feature, but those are 15 years old. Is there any possible way to do it now?

If it's not possible, is there a way to delete the directory after it got downloaded and zipped? --delete-after deletes the files before they got zipped. Anyway it would be pretty inefficient to download everything every time, but better than having old files around.

jona
  • 113
  • 4

1 Answers1

0

Your requested task is a type of "sync". I believe wget may not be the solution for your task as wget (and curl) are primarily "get" or "download" tools.

Alas, I am not familiar with a syncing tool which will work with an HTTP endpoint. I typically will use rsync or similar tool.

As you mentioned, the easiest way to accomplish the task is to always start with an empty directory.

There may be a "downloader" application which supports a local cache to hit before re-downloading a file, but afaik wget isn't one of them.

Adam Chasen
  • 46
  • 4
  • Thanks. But `rsync` is only available with ssh, which I can't use because I don't have access to ssh keys. lftp would be an option as far as I can see that my some research, but could't make it to work in the first place. – jona Mar 02 '21 at 17:06
  • @jona You don't have access to your own web site?!? What is going on here? – Michael Hampton Mar 02 '21 at 20:28
  • @MichaelHampton The hosting support told me that it's not possible to create ssh keys in my plan. If I understand this correctly, I need those to use `rsync`. Although I have access to SFTP/ SSH via shell – jona Mar 03 '21 at 09:23
  • If you have access to SSH/SFTP, then you *may* have `rsync` access. It actually uses SSH as a transport, but requires the `rsync` tool to be available on both systems. Here is a quick search result: https://phoenixnap.com/kb/how-to-rsync-over-ssh With SFTP, there should be several older tools which allow "sync". Back in my windows days I would use WinSCP. – Adam Chasen Mar 03 '21 at 18:14