Synchronization and backup programs

This page lists and compares applications that synchronize data between two or more locations, and those that build on top of such functionality to make incremental copies of important data for backup purposes. Because of their relationship, the two groups share several traits that justify describing them in the same article.

Related articles

Important considerations

In order to choose the best program for one's own needs, the following aspects should be considered:

  • The type of backup medium that is going to store the data, e.g. CD, DVD, remote server, external hard drive, etc.
  • The planned frequency of backups, e.g. daily, weekly, monthly, etc.
  • The features expected from the backup solution, e.g. compression, encryption, handles renames, etc.
  • The planned method to restore backups if needed.

Data synchronization

These applications simply keep directories synchronized between multiple locations/machines, in a "mirror" fashion. Nonetheless, most of them still allow storing and reverting to old revisions of modified or deleted files.

See also:

Legend
Name
The application name, linking to the ArchWiki article or the official website.
Package
A link to the package.
Implementation
The programming language, library, or utility that the application is based on.
Delta transfer
Only the modified parts of files are transferred.
Encrypted transfer
Data is encrypted by default when transferred over the network.
FS metadata
File system permissions and attributes are synchronized.
Resumable
The synchronization can be resumed if interrupted.
Handles renames
Moved/renamed files are detected and not stored or transferred twice. It typically means that a checksum of files or its chunks is computed. Applications missing this functionality can be supplemented by combining with hsyncAUR, which only synchronizes renames.
Version control
The old version of files are backed up (reverse incremental backup).
Change propagation
Specifies in how many directions changes can be propagated.
  • unidirectional means one-way synchronization of two locations,
  • bidirectional means two-way synchronization of two locations and
  • multidirectional means full synchronization of more than two locations.
Conflict resolution
The application handles file conflicts, either automatically or interactively, i.e. it does not silently discard conflicting files. This attribute does not apply to applications that only propagate changes in one direction.
FS monitoring
The application listens to file system events to trigger the synchronization.
CLI
The application provides a command-line interface.
Other interfaces
The application has the specified user interfaces, e.g. GUI, TUI, or web-based.
License
The license of the server and client applications.
Other platforms
Supported operating systems other than Linux.
Maintained
The project is maintained.
Specificity
Brief notes about special features that notably set the application apart from the others.

Table
Name Package Implementation Delta transfer Encrypted transfer FS metadata Resumable Handles renames Version control Change propagation Conflict resolution FS monitoring CLI Other interfaces License Other platforms Maintained Specificity
FreeFileSync freefilesyncAUR C++ ? ? ? unidirectional / multidirectional Yes ? Yes Windows, macOS Yes
git-annex Haskell, git Yes Yes multidirectional; with git remotes optional, git-annex assistant Yes macOS, Android (beta), Windows (beta) Yes
osync.sh Bash, based on rsync ? Yes Yes bidirectional Yes BSD Yes
rclone Go ? ? ? ? ? unidirectional ? ? Yes Web (experimental add-on) *BSD, Plan9, Solaris, Windows, macOS Yes
rdiff-backup Python, librsync Yes ? Yes unidirectional Yes Win32 Yes
Resilio Sync C++ Yes Yes ? Yes ? Yes multidirectional ? ? FreeBSD, Windows, macOS, Android, iOS, Windows Phone, Amazon Kindle Fire Yes P2P sync
rsync C Yes Yes Yes unidirectional No Yes Win32 Yes
SparkleShare C#, git Yes ? ? Yes Yes ? ? ? No Yes Windows, macOS Yes It can sync with any Git server over SSH.
Syncany Java ? ? ? ? ? ? ? ? ? Yes Yes No
Syncthing Go Yes Yes ? , previous versions moved to archive folder multidirectional Yes BSD, Windows, macOS, Android, Kindle Paperwhite Yes P2P sync
Synkron C++ ? ? ? ? ? ? multidirectional ? ? No Qt Windows, macOS
taskd C++, Python Yes Yes ? Yes ? ? multidirectional ? No Yes No Android Yes
Unison unison OCaml Yes Yes No bidirectional No Yes FreeBSD, Windows, macOS, Android Yes
yarsync Python, based on rsync Yes Yes Yes Yes, for repository snapshots (each file has a single version) unidirectional / multidirectional No Yes No Yes UNIX-like systems or backup drives, CLI like git.
Zaloha2.sh bash No No No No bidirectional No Yes No Windows under Cygwin Small and simple

Incremental backups

Applications that can do incremental backups remember and take into account what data has been backed up during the last run (so-called "diffs") and eliminate the need to have duplicates of unchanged data. Restoring the data to a certain point in time would require locating the last full backup and all the incremental backups from then to the moment when it is supposed to be restored. This sort of backup is useful for those who do it very often.

See also:

Legend:

  • Name: the application name, linking to the ArchWiki article or the official website.
  • Package: a link to the package.
  • Implementation: the programming language, library, or utility that the application is based on.
  • Compressed storage: compression is used for storage.
  • Encrypted storage: encryption is used for storage.
  • Delta transfer: only the modified parts of files are transferred.
  • Encrypted transfer: data is encrypted by default when transferred over a network.
  • FS metadata: file system permissions and attributes are backed up.
  • Easy access: the backup is stored plainly in the file system, or is mountable as such.
  • Resumable: the backup can be resumed without restarting it if interrupted.
  • Multithreading: the backup can be done in multiple threads of execution concurrently.
  • Handles renames: moved/renamed files are detected and not stored or transferred twice; it typically means that a checksum is computed for files or chunks thereof.
  • CLI: the application is command-line driven, i.e. it is scriptable.
  • Other interfaces: the application has the specified user interfaces, e.g. GUI, TUI, or web-based.
  • Licence: the licence of the server and client applications.
  • Other platforms: supported operating systems other than Linux.
  • Maintained: whether the project is maintained.
  • Specificity: brief notes about special features that notably set the application apart from the others.

Single machine

These applications are aimed at backing up data from the machine they are installed on, although the backup destination can be located on an external machine or storage media.

Chunk-based increments

If a file is modified, these applications store only its changed parts at the next snapshot. Compared to #File-based increments applications, these are more space-efficient, especially when large files receive small modifications; on the other hand, the archived snapshots have to be opened with the backup application that created them, since the files have to be reconstructed from the stored binary diffs.

Name Package Implementation Compressed storage Encrypted storage Delta transfer Encrypted transfer FS metadata Easy access Resumable Multithreading Handles renames CLI Other interfaces Licence Other platforms Maintained Specificity
Areca Backup Java AES128, AES256 Yes Yes Yes No No No Yes Yes GPLv2 Windows Yes
BorgBackup Python, C (Cython) Yes Yes Yes Yes No Yes Yes BSD *BSD, macOS, Windows (Cygwin / WSL) Yes Deduplication based on variable length chunks; support both local and SSH-based remote backup destination.
bup C, Python, git Yes No Yes Yes Yes pick up where you left off No Yes Yes GPLv2 NetBSD, Windows, macOS Yes
Duplicacy Go Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Custom (non-free) FreeBSD, macOS, Windows Yes
Duplicati C# Yes Yes Yes Yes Yes No Yes No Yes Yes LGPL Windows, macOS Yes
Duplicity librsync gzip Yes Yes ? No Yes No No Yes GPL Yes
Kopia kopiaAUR Go, Javascript front-end Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes APACHE Windows, macOS, OpenBSD Yes
Kup Backup System rsync, bup front-end Yes Yes Yes Yes Yes No ? Yes Qt GPLv2 Yes
restic Go Yes Yes Yes Yes Yes Yes Yes Yes Yes No BSD OpenBSD, Windows, macOS Yes
ZBackup C++ LZMA, LZO Yes Yes ? No ? Kinda through tar Yes No GPLv2 No

File-based increments

If a file is modified, these applications store its new version entirely at the next snapshot. Compared to #Chunk-based increments applications, these are less space-efficient, especially when large files receive small modifications; on the other hand, often the archived snapshots can be opened without the need to have the backup application installed.

Specific legend:

  • Hard links: whether unmodified files are stored as hard links to previous versions.
Name Package Implementation Compressed storage Encrypted storage Delta transfer Encrypted transfer FS metadata Easy access Resumable Handles renames Hard links CLI Other interfaces Licence Other platforms Maintained Specificity
Back In Time Python, rsync, diff No Yes Yes No No Yes Qt GPLv2 Yes
DAR (Disk ARchive) C++ Yes Yes Yes ? ? ? ? No Yes GPL FreeBSD, NetBSD, Windows, macOS Yes
Link-Backup Python 2 No No ? ? ? Yes Yes No [dead link 2022-09-23 ⓘ] Yes No MIT No
rdup C ? ? ? Yes ? No Yes Yes No GPLv3 Yes
rsnapshot rsync No No Yes Yes ? ? ? ? Yes No GPLv2 Win32
timeshift rsync No No ? ? ? ? Yes Yes GTK GPLv3 Designed for full-system backups to dedicated devices.

Network oriented

These applications have been designed to centralize the backup of several machines connected to a network, through a server-client model. In general they are more complicated to deploy, compared to #Single machine solutions.

Specific legend:

  • Control direction: Pull: server logs into client. Push: client initiates backup session.
  • Increment type: the strategy used to reduce used space by deduplicating data (i.e., besides compression).
    • file-based: if a file is modified, the entire new version is stored at each snapshot.
      • hard-links: whether unmodified files are stored as hard links to previous versions.
    • chunk-based: only the modified parts of files are stored at each snapshot.
Name Package Implementation Control direction Compressed storage Encrypted storage Delta transfer Encrypted transfer FS metadata Easy access Resumable Handles renames Increment type CLI Other interfaces Licence Other platforms Maintained Specificity
BackupPC Perl Pull Yes No Yes Yes Yes No Yes ? file-based, hard links No Web GPLv2 Any (no client needed) Yes
Bacula bacula* in AUR C++ Pull Yes Yes ? Yes ? ? Yes ? file-based Yes AGPLv3 Windows, macOS Yes
Bareos bareos* in AUR C++ (Bacula fork) ? ? ? ? ? ? ? ? ? ? ? ? AGPLv3 Yes
burp librsync Push Yes Yes Yes Yes Yes ? Yes ? chunk-based Yes AGPLv3 Windows, macOS Yes
SafeKeep rdiff-backup Pull No No ? Yes ? ? ? ? chunk-based Yes Yes GPL No
Synbak Multitool wrapper ? Yes No Yes Yes Yes ? ? ? ? No Web GPLv3 Yes Unifies several backup methods.
UrBackup urbackup* in AUR C++ Pull No No Yes Yes Yes Yes Yes file-based,hard-links and symlinks/chunk-based CoW-Snapshots Yes (client) AGPLv3+ Windows, macOS Yes

Version control systems

While version control systems are mostly used for source code, they can track any files in a directory.

See List of applications/Utilities#Version control systems and dotfiles.

gollark: We have plenty of spare computation power at GTech™'s hyperscale computing arrays.
gollark: Yes. I am using several exaflops of computation power to process this thought.
gollark: GTech™ law #918383.
gollark: This is illegal.
gollark: ++remind 10h fix thing

See also
This article is issued from Archlinux. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.