0

for file in $(find . -type f -mmin -240) do filename=$(echo "$file" | cut -d'.' --complement -s -f1) s3cmd put $file s3://${bucketname}/$filename done

I am trying to do the above in order to upload all files that have changed in the last 4 hours to a bucket in amazon but because many of the files have spaces in them, this is not working.

I keep getting the following error:

ERROR: Parameter problem: Nothing to upload.

I know I could use sync instead of put but I don't want to do that because this folder has over 30K files and that would mean that sync has to check and forth in the server and it would take ages and cost extra money.

Anyone knows how to pass variables that contain spaces to this to make it work?

Ulukai
  • 829
  • 2
  • 10
  • 28
  • Try putting `"` around the variable - `"$file` – user9517 Sep 12 '16 at 14:24
  • I tried that. I also tried putting it around the second argument. No joy. – Ulukai Sep 12 '16 at 14:55
  • Try running `s3cmd` in debug mode (by adding the `-d` flag). That should give more information about what's happening. Uploading files with spaces in the name of the local file works fine for me (with s3cmd version 1.6.1 and quotes around the file name. The message you're getting implies that the command can't find the file it has been asked to transfer. – Paul Haldane Sep 12 '16 at 20:13
  • I think I found what the issue is. Some filenames contain hyphens, and you can't store - in a variable... now I need to think how to get around this without complicating my life too much. – Ulukai Sep 13 '16 at 08:29

1 Answers1

1

The problem is that bash for loop is using $IFS as field separator, which defaults to space. So each part of filename separated by a space will come as a different iteration in loop.

IFS can be changed to match newline instead of space and allow correct processing of find output.

For example: 

# save and set IFS
OIFS=$IFS
IFS=$'\n'
# do your processing
for file in $(find . -type f); do echo '>>'${file}"<<"; done
# restore IFS
IFS=$OIFS
tonioc
  • 1,017
  • 8
  • 11
  • I had thought about this but because I had used a similar loop before somewhere else and it recognised the line break without having to do this, I decided not to. Thank you so much, brilliant - it works! – Ulukai Sep 16 '16 at 13:03