Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [archive]

148 questions
81
votes
11 answers

Transfer 15TB of tiny files

I'm archiving data from one server to another. Initially I started a rsync job. It took 2 weeks for it to build the file list just for 5 TB of data and another week to transfer 1 TB of data. Then I had to kill the job as we need some down time on…
lbanz
  • 1,579
  • 4
  • 18
  • 30
31
votes
8 answers

Is there a smarter tar or cpio out there for efficiently retrieving a file stored in the archive?

I am using tar to archive a group of very large (multi-GB) bz2 files. If I use tar -tf file.tar to list the files within the archive, this takes a very long time to complete (~10-15 minutes). Likewise, cpio -t < file.cpio takes just as long to…
Alex Reynolds
  • 453
  • 2
  • 9
  • 20
20
votes
14 answers

Email Archiving in an Exchange Environment

We're looking into a solution to do email archiving and near-line backup of our Exchange Server 2007 environment. We currently have about 200 mailboxes and the Mailstore is approx 150 GB. We need to be able to archive email for several years for…
BillN
  • 1,503
  • 1
  • 13
  • 30
20
votes
4 answers

Creating a tar file with checksums included

Here's my problem : I need to archive to tar files a lot ( up to 60 TB) of big files (usually 30 to 40 GB each). I would like to make checksums ( md5, sha1, whatever) of these files before archiving; however not reading every file twice (once for…
wazoox
  • 6,782
  • 4
  • 30
  • 62
19
votes
3 answers

wheezy-updates on archive.debian.org returns 404 Not Found

I have an old test machine on Debian Wheezy that I can't really update right now. Debian Wheezy repositories were archived in February. So I want ahead, and updated my /etc/apt/sources.list from deb http://ftp.fr.debian.org/debian wheezy main deb…
Zoddo
  • 194
  • 1
  • 1
  • 8
16
votes
6 answers

Cost effective, long term archival of video and image data? ~50 TB

My lab is in the process of setting up a small server that holds data (mostly video and image data, plus a few documents) for the project our group is working on at a moment in time. Historically, after a research project ends, the data haphazardly…
hpy
  • 835
  • 3
  • 18
  • 28
14
votes
9 answers

Home-brew automatic tiered storage solutions with Linux? (Memory -> SSD -> HDD -> remote storage)

Related to this question about using an SSD for system and HDD for data, except I would like my system to do this automatically... Is it possible to have several layers of storage and push items automatically between them, using preferably free,…
David Gardner
  • 1,499
  • 2
  • 13
  • 25
12
votes
3 answers

Tar: avoid archiving of files larger than certain size

I want to archive files (with tar) which are below 3 MB in size. But I also want to retain the directories in which those files exist. (so I cannot use find command). I just want to avoid the files which are above 3 MB in size. How can this be done?
nixnotwin
  • 1,513
  • 5
  • 34
  • 54
11
votes
9 answers

Looking for an open source email archiving application

I'm looking for an open source application that will archive my email. It might do this by logging in to my POP3 account on a regular basis and copying the emails across, or it might just read my Unix mbox/maildir file/directory directly on the mail…
Joel
  • 439
  • 2
  • 5
  • 11
11
votes
2 answers

How can I list the content of a zip archive, but only the first level?

I have a big zip file and I want to know what it's contain. I know I can run: zipinfo file.zip but the output is too verbose and there are a lot of files in the sub-directories. I want to see a list of the files in the top level. Example If the…
Sapir
  • 113
  • 1
  • 6
11
votes
2 answers

How to exclude files from TAR archive using regular expressions?

I have a simple question, yet I can't find or solve the answer. I want to make a tar archive, but I want to exclude some files from it using regular expression. Example of the file to exclude is this: 68x640X480.jpg I have tried this with no…
Frodik
  • 273
  • 1
  • 3
  • 14
9
votes
3 answers

General Advice about an archive solution. ~15tb and growing.

I need a better storage and archive system for my small business's files. Specifically the files are completed video projects. Beyond time and cost limitations what is holding me back is I don't believe in any of the solutions I have pondered.…
maestrojed
  • 221
  • 1
  • 1
9
votes
4 answers

Which tar file format should I use?

tar can create the archive in different formats. GNU tar, ustar, pax, v7. What would be the best for long time archiving? Is there significant differencies in these formats? I would use the best format for general backups, and I don't want that…
Glendyr
  • 601
  • 2
  • 7
  • 12
8
votes
7 answers

Which archive/compression format?

Zip, Rar, 7z, Gzip, BZip2, Tar etc. I'm hearing 7z is the flavor of the month, why? Is it best for all situations or are there better choices for specific situations. Or maybe the actual file archiver ie WinZip, WinRar, 7Zip etc (as opposed the…
Ash
  • 273
  • 4
  • 7
7
votes
4 answers

Backup is not Archiving - But what requirements does archiving have?

There's related thread on how to handle archiving. I'm asking on how to find the requirements in a company on the way archiving should be done, and also the requirements that need to be fullfilled so that archiving is actually done instead of just…
Martin M.
  • 6,428
  • 2
  • 24
  • 42
1
2 3
…
9 10