What files, file types, and folders to exclude from user folder backup?

Question

When backing up a user and shared users folder using the backup software of your choice on the operating system of your choice what files, file types, and folders do you exclude from the backup?

Answer 1

In Java Regular Expression format (some data from CrashPlan Excludes and other unlisted excludes from CrashPlan's my.service.xml file):

All:

Default from CrashPlan: 
.*/(?:42|\d{8,}).*/(?:cp|~).*
(?i).*/CrashPlan.*/(?:cache|log|conf|manifest|upgrade)/.*
.*\.part
.*/iPhoto Library/iPod Photo Cache/.*
.*\.cprestoretmp.*
.*/Music/Subscription/.*
(?i).*/Google/Chrome/.*cache.*
(?i).*/Mozilla/Firefox/.*cache.*
.*/Google/Chrome/Safe Browsing.* 
.*/(cookies|permissions).sqllite(-.{3})?

Extra not in CrashPlan by default:
(?i).*\.(class|obj)
(?i).*\.(tmp|temp|cache|part|rbf|chk|dov) <-- temporary files
(?i).*\.(bac|bak|bkp|bup|back|dmp|dump|err|log|old|prv) <-- backup files
(?i).*\.(asd|wbk|xlk|bde|bdi|bdm|bmb|bom|crc|m\_p|bdb|bks|bps|mch|sol|idb|nch) <-- program specific backup/temporary files
(?i).*\.(bmk|chw|ftg|fts|gid|hhc|hhk) <-- help index files
(?i).*\.cache.*
:/Config\\.Msi.*
.*/Google/Chrome/.* <-- if using Chrome Sync
.*/Google/(Drive|CrashReports)/.*
(?i).*/(Google Drive|LocalGoogle)/.*
.*/Dropbox/.*
.*/.*SkyDrive.*/.*
.*/(My Cubby|cubby)/.*
(?i).*/Cookies/.*
(?i).*/Downloads/.*
(?i).*/.*cache.*/.*
(?i).*/.*log.*/.*

Windows:

Default from CrashPlan:
.:/ProgramData/
.*Thumbs\.db
.*/Local Settings/History/
.*/NetHood/
.*/PrintHood/
.*/Cookies/
.*/Recent/
.*/SendTo/
.*/LocalService/
.*/NetworkService/
.*/Start Menu/
(?i).*/NTUSER.*
(?i).:/WINDOWS/
(?i).:/autoexec.bat
.*\$RECYCLE\.BIN/.*
.*/System Volume Information/.*
.*/RECYCLER/.*
.*/I386.*
.*/pagefile.sys
.*/MSOCache.*
.*UsrClass\.dat\.LOG
.*UsrClass\.dat
.*/Temporary Internet Files/.*
(?i).*/ntuser.dat.*
.*/Local Settings/Temp.*
.*/AppData/Local/Temp.*
.*/AppData/Temp.*
.*/Windows/Temp.*
(?i).*/Microsoft.*/Windows/.*\.log
.*/Microsoft.*/Windows/Cookies.*
.*/Microsoft.*/RecoveryStore.*
(?i).:/Config\\.Msi.*
(?i).*\\.rbf
.*/Windows/Installer.*
.*/Application Data/Application Data.*
(?i).:/Config\.Msi.*
(?i).*\.rbf
(?i).*/Microsoft.*/Windows/.*\.edb 
(?i).*/Google/Chrome/User Data/Default/Cookies(-journal)?
(?i).*/Safari/Library/Caches/.*
(?i).*/(Application Data|AppData)/Apple Computer/Mobile Sync/.*
.*\.tmp
.*\.tmp/.*

Extra not in CrashPlan by default:
.*/hiberfil.sys
.*/swapfile.sys
.*/Local Settings/History/.*
(?i).*\$RECYCLE\.BIN/.*
(?i).*/Windows/.*
.*/(Program Files|Program Files \(x86\))/.*
.*/AppData/Local/Packages/.* <- this really still needs to be figured out as per http://superuser.com/questions/490925/explain-windows-8-windows-store-appdata-packages-and-what-to-backup
.*/AppData/Local/Microsoft/Windows Store/.*
.*/NetHood/.*
.*/PrintHood/.*
.*/Cookies/.*
.*/Recent/.*
.*/SendTo/.*
.*/LocalService/.*
.*/NetworkService/.*
.*/AppData/LocalLow/.*
.*/Internet Explorer/.*

Mac:

Default from CrashPlan:
/bin/
/etc/
/sbin/
/usr/
/sw/
/System/
/private/
/dev/.*
.*\.Trash.*
/\.vol/.*
/mach.sym
/mach_kernel
.*\.hotfiles\.btree.*
.*/Trash/.*
.*/Network Trash Folder/.*
.*/VM Storage
.*\.fseventsd.*
.*/Library/Caches/.*
.*\.Spotlight-.*/.*
/Network/.*
/tmp/.*
/cores/.*
/afs/.*
/automount/.*
/private/Network/.*
/private/tmp/.*
/private/var/tmp/.*
/private/var/folders/.*
/private/var/run/.*
/private/var/spool/postfix/.*
/private/var/vm/.*
/Previous Systems.*
.*/lost\+found/.*
.*/Microsoft User Data/Entourage Temp/.*
.*/iP.* Software Updates/.*
.*/Library/Application Support/SyncServices/.*
.*/Library/Logs/.*
.*/Library/Mail/Envelope Index
.*/Library/Mail/AvailableFeeds/.*
.*/Library/Mirrors/.*
.*/Library/PubSub/Database/.*
.*/Library/PubSub/Downloads/.*
.*/Library/PubSub/Feeds/.*
.*/Library/Safari/Icons.db
.*/Library/Safari/WebpageIcons.db
.*/Library/Safari/HistoryIndex.sk
.*/Library/Calendars/Calendar Cache
.*/iTunes/Album Artwork/Cache/.*
.*/iPhoto Library/iPod Photo Cache.*
(?i).*/backups.backupdb/.*
/Desktop DB
/Desktop DF
/Network/Servers.*
/Users/Shared/SC Info.*
/net/.*
/private/var/automount/.*
/private/var/db/dhcpclient/.*
/private/var/db/fseventsd/.*
/System/Library/Extensions/Caches/.*
.*Mobile.*Backups/.*
.*/Library/Saved Application State/.*
.*/Library/Mail/.*/Info.plist
.*/Library/Calendars/.*/Info.plist 
.*/Application Support/Google/Chrome/Default/Cookies(-journal)?
.*/Library/Cookies/(Cookies.binarycookies|com.apple.appstore.plist)

Linux:

Default from CrashPlan:
/bin/a
/boot/
/etc/
/initrd/
/lib/
/opt/
/sbin/
/usr/
/var/
/srv/
/sys/.*
/dev/.*
/proc/.*
/tmp/.*
.*/lost\+found/.*
/selinux/.* 
.*/.config/google-chrome/Default/Cookies(-journal)?

Extra not in CrashPlan by default:
(?i).*\~

Solaris:

Default from CrashPlan:
/bin/
/boot/
/etc/
/initrd/
/lib/
/opt/
/sbin/
/usr/
/var/
/srv/
/sys/.*
/dev/.*
/proc/.*
/tmp/.*
.*/lost\+found/.*
.*/.config/google-chrome/Default/Cookies(-journal)?

Answer 2

Linux: Everything ending with a ~ (backups from several editors). And to "optimize" your regexps, just a short example:

(?i).*\.(tmp|temp|part|bak|\~)

Answer 3

After careful examination I also exclude files under %APPDATA% that match (ignoring case) any of these regular expressions:

• Cache files

/Mozilla/.*/Cache
/MicrosoftEdge/Cache
/Chrome/.*/Cache
/Default/Media Cache
/GoogleEarth/unified_cache
/AppData/Local/Temp
/MicrosoftEdge/.*/Cache
/Skype/.*/emo_cache
/Device Metadata/dmrccache$
/cache$
/INetCache$
/LocalCache$
/Package Cache$
/WebCache$
/OfficeFileCache$
/CryptnetUrlCache$
/AppCache$
/IECompatCache$
/WebServiceCache$
/IECompatUaCache$

• log files

/logs$
/log$

• System files

/AppData/Local/Microsoft/Windows$
/AppData/Local/Microsoft/Device Stage$

• Temporary files

/temp$
/temporary[^/]*$

There's also discussion about this topic on https://forum.duplicati.com/t/common-exclusions-and-inclusions-from-appdata/807 . These lists appear there. CAUTION It seems like they use common regex but without taking care to escape literal dots):

.*/AppData/Local/Temp.*
.*/AppData/Temp.*

.*/AppData\Local\Microsoft\Windows\Notifications\WPNPRMRY.tmp
.*/AppData\Local\Microsoft\Windows\UsrClass.dat.LOG1
.*/AppData\Local\Microsoft\Windows\UsrClass.dat.LOG2
.*/AppData\Local\Microsoft\Windows\WebCache\WebCacheV01.dat
.*/AppData\Roaming\Skype\your.skypename\main.lock
.*/AppData\Local\Google\Drive\lockfile
.*/AppData\Local\Microsoft\Internet Explorer\Recovery\High\Active*.dat
.*/AppData\Roaming\Mozilla\Firefox\Profiles\<profile>.default\parent.lock
.*/AppData\Local\Google\Chrome\User Data\Default\Current Session
.*/AppData\Local\Google\Chrome\User Data\Default\Current Tabs
.*/AppData\Local\Google\Chrome\User Data\lockfile

*.part
*.tmp
*.ost
hiberfil.sys
pagefile.sys
swapfile.sys
*\UsrClass.dat*
*\ntuser.dat*
$Recycle.Bin\
Windows\
MSOCache\
*System Volume Information*\
*\AppData\*\Temp\
*\AppData\*\Internet Explorer\
*\AppData\*\Microsoft\*\WebCache\
*\AppData\*\Microsoft\*\INetCache*\
*\AppData\*\Microsoft\Office\*\OfficeFileCache\
*\Temporary Internet Files\
*\Firefox\Profiles\*\*cache*\
*\Microsoft\Windows Defender*\