Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [s3cmd]

s3cmd is a simple command-line interface to storing and retrieving data to/from Amazon's S3 cloud storage service.

43 questions
0
votes
1 answer

Where is the AWS S3 metadata stored written by s3cmd?

I was looking for a AWS S3 solution for backing up a Linux file system so that it would preserve the timestamp and permissions. It appears that s3cmd allows this by writing this information to the metadata for the object (file) on AWS S3 in the…
Edward_178118
  • 895
  • 4
  • 14
  • 30
0
votes
1 answer

How to upload multiple files from directory to S3?

I have a directory on an Ubuntu, with 340K images, 45GB of total size! Is there an efficient way to transfer them all to an S3 of DigitalOcean? I thought of using s3cmd put or s3cmd sync but I'm guessing that would perform the put operation on every…
Sotiris Kaniras
  • 198
  • 1
  • 10
0
votes
1 answer

Transfer image files from MongoDB to S3

I have a NodeJS app that stores all of its data, even images, to MongoDB. Now I want to use an S3 to store my app's files (images, 43gb) and so, I need to transfer there all files that I have in my mongo. I found the s3cmd tool that can transfer my…
Sotiris Kaniras
  • 198
  • 1
  • 10
0
votes
1 answer

run s3cmd on Synology NAS

I'm trying to use s3cmd on a Synology NAS with DSM 6.0, and I'm getting this error. I've installed the python-magic libs but still. [root@OFFICE_01 arc]# s3cmd --version !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An unexpected error has…
brunobhr
  • 33
  • 9
0
votes
1 answer

can I attach IAM role to instance after launching it?

Getting error : root@omp_affiliate_backend ~]# s3cmd get s3://admarvel-noc/all-users/nrpe-plugins.tar.gz . s3://admarvel-noc/all-users/nrpe-plugins.tar.gz -> ./nrpe-plugins.tar.gz [1 of 1] ERROR: S3 error: 403 (Forbidden): Is it due to iam role ?…
Ashish Karpe
  • 277
  • 2
  • 5
  • 19
0
votes
1 answer

How to use s3cmd put when there are spaces in the filename?

for file in $(find . -type f -mmin -240) do filename=$(echo "$file" | cut -d'.' --complement -s -f1) s3cmd put $file s3://${bucketname}/$filename done I am trying to do the above in order to upload all files that have…
Ulukai
  • 829
  • 2
  • 10
  • 28
0
votes
1 answer

s3cmd restore recursive not working right

I am running the following command to restore the data from Amazon S3 to the web server. s3cmd restore --recursive s3://my-bucket/Weekly-Sunday/dbexports/ and getting this error: Restore is not allowed, as object's storage class is not GLACIER On…
prosti
  • 328
  • 1
  • 6
  • 17
0
votes
0 answers

s3cmd 403 Access Denied error on put but not ls

I can access s3 using s3cmd on an ec2 instance to do s3cmd ls s3://myc bucketname but not using 'put' to add a file. It gives a 403 access denied error. I am using IAM and have group setup to access on the single bucket in question. Is there…
Tofuwarrior
  • 229
  • 1
  • 3
  • 10
0
votes
1 answer

How can I control a gradual file upload with cron or similar?

I have several terabytes of data I need to back up to S3/Glacier over a slow link. I want to upload from, say 9pm to 9am and resume from where it left off. Is there free software that would do that easily? I could use s3cmd/s3sync for the uploading,…
sventechie
  • 119
  • 1
  • 9
0
votes
1 answer

How continue sync with S3CMD after broken connection?

I'm using s3cmd to backup some logs into Amazon S3 buckets. Which is a great tool. But I notice that, very often, my sync's are broken, they just cancel and try to upload again, until they can upload the file, but the problem is that they don't…
Valter Silva
  • 155
  • 1
  • 4
  • 14
0
votes
0 answers

Hardware Clock Drift in AWS CentOS 6.8 instance

I have a CentOS 6.8 EC2 instance on AWS that is up and running since 1232 days. On 11th October 2021, I found that the command 's3cmd' failed due to hardware clock drift. I debugged it and found hardware clock drift. I, then, synced the hardware…
Manoj Agarwal
  • 101
  • 2
0
votes
1 answer

cannot Delete objects on s3 ceph bucket

I have a ceph cluster and running a few s3 bucket on it, 'gitlab-s3-api' user has full permission on everything on ( users=;buckets=;metadata=;usage=;zone=* ) but very oddly it cannot delete any file on its own bucket. { "user_id":…
AliReza NaSRi
  • 11
  • 2
-1
votes
1 answer

Python error when s3cmd is executed within bash script

I'm creating a bash script of which will run on a cron, it's purpose is to backup a directory to Amazon S3 using the s3cmd command. I get the following error when I run the script: /usr/bin/env: python: Not a directory When I comment out the s3cmd…
Ben Everard
  • 569
  • 3
  • 7
  • 21
1 2
3