How can I delete Time Machine files using the commandline

I want to delete some files/directories from my Time Machine Partition using rm, but am unable to do so. I'm pretty sure the problem is related to some sort of access control extended attributes on files in the backup, but do not know how to override/disable them in order to get rm to work. An example of the error I'm getting is:

% sudo rm -rf Backups.backupdb/MacBook/Latest/MacBook/somedir
rm: Backups.backupdb/MacBook/Latest/MacBook/somedir: Directory not empty
rm: Backups.backupdb/MacBook/Latest/MacBook/somedir/somefile: Operation not permitted

There are a number of reasons I do not want to use either the Time Machine GUI or Finder for this. If possible, I'd like to be able to maintain the extended protection for all other files (I'd like not to disable them globally, unless I can re-enable once I've done my work).

Tim

Posted 2010-07-12T15:50:37.047

Reputation: 1 267

It seems that the accepted answer is quite dangerous when deleting snapshots, or folders in snapshots, like your rm -r Backups.backupdb/MacBook/Latest/MacBook/somedir; if you agree, then please accept Arne's answer? – Arjan – 2015-08-26T16:44:16.740

I've gotten closer.

First, I needed to disable ACLs on my Time Machine disk. Previously, one would use fsaclctl to do this, but Snow Leopard does not include this. I grabbed a copy of the binary from an older version of OSX and ran this:

% sudo fsaclctl -p /Volumes/tmvol -d

I then tried to use "sudo rm -rf" to remove a directory, but still ran into issues with some files (though others went away fine). Specifically, it failed on soft links. Very strange. On the plus side, the links take up almost no space. On the negative side, I still have the directories sticking around. – Tim – 2010-07-14T16:26:05.333

Answers

117

To work around "operation not permitted" errors, use the Time Machine Safety Net "bypass" program:

sudo /System/Library/Extensions/TMSafetyNet.kext/Contents/MacOS/bypass rm -rfv /Volumes/[disk]/Backups.backupdb/[path]

In 10.8 Mountain Lion, bypass moved into 'Helpers':

/System/Library/Extensions/TMSafetyNet.kext/Helpers/bypass

In 10.10 Yosemite, bypass moved here:

/System/Library/Extensions/TMSafetyNet.kext/Contents/Helpers/bypass

Beware when using this to delete specific snapshots: as Time Machine uses hard links, using rm -r on folders might also affect older and newer snapshots of the same machine. (See other answers referring to tmutil delete to safely delete a specific snapshot.) Using rm to delete all snapshots for a single machine is okay though. And so is using rm to delete a specific file, which will only remove that hard-linked file from the snapshot(s) you specify, assuming the file is not in a hard-linked directory as then you'd actually remove the file from all those hard-linked directories.

Eric W

Posted 2010-07-12T15:50:37.047

Reputation: 1 306

THX -- that saved me. What's the use of a feature to allow Apple to bypass the underlying UNIX safety just to provide a bypass to the bypass? What for? Just to do the same thing you intended only delayed by a 10minutes google search? – georg – 2014-12-11T14:06:07.073

Though I hear the danger of how using this method to remove an individual snapshot is likely to also remove files within other snapshots, it was very effective at removing an entire backup -- all snapshots -- from another machine inside Backups.backupdb. – Ivan X – 2015-04-17T03:45:01.830

To extend on Haravikk's comment a bit: using rm for hard-linked files is okay, but for hard-linked folders it's not. Kent's answer mentions the same problem. And in What is the Unix command to create a hardlink to a directory in OS X? someone wrote for 10.5 in 2010: "Deleting is a different story: if you go about it the usual way to delete directories, you'll delete the contents. So you must "unlink" the directory: unlink new_hard_link". So only use this to delete all backups (snapshots) for a given machine.

– Arjan – 2015-08-26T16:24:42.173

bypass apparently only works for rm, what's the workaround for mv? – Michael – 2017-01-23T03:07:29.733

@Michael I've added an example above – Ryan Schumacher – 2017-12-03T06:59:58.230

+1!! This helped me. I didn't even try the others because this looked like "the right way" and indeed, it allowed me to resolve the issue with no further questions. Thanks! – tripleee – 2012-02-28T13:59:23.330

I created an account just to +1 you. I have been looking for a better solution to this, and this was it. It had been driving me crazy. Thank you. – CWSpear – 2012-05-13T21:46:48.547

3Awesome. This works. (although I leave out the 'v' option to avoid seeing an entry for each of a few hundred thousand files) i.e.: sudo ...bypass rm -rf /Volumes/... – Brent Faust – 2012-11-13T01:54:49.387

Fantastic! Saved me hours of hair pulling. Thanks! – jsd – 2013-02-27T23:30:13.887

6This is an extremely dangerous method for managing Time Machine files; Time Machine uses hard-linked directories to reference folders that haven't changed since the previous backup, however the rm process doesn't understand these and will follow these links and remove files from them. This means you could potentially delete files that are also in older and newer backups than the one that you selected, potentially causing irreparable damage to the backup(s). Arne Stenström's recommendation of using tmutil is by far the superior solution. – Haravikk – 2013-12-03T18:59:51.267

@Haravikk What you described, however, was exactly what I came here looking to do: target specific files but erase them throughout Time Machine, not delete entire snapshots. – silvamerica – 2014-04-02T07:42:50.237

BLUF (bottom line up-front):

sudo tmutil delete snapshot-dir

Using sudo chmod -R -N folder to remove all ACLs from a folder hierarchy doesn't work on the files and folders in Time Machine's Backups.backupdb, because of the TM Safety Net mechanism and the criteria described in this 318 Tech Journal post (but possibly not exactly as described).
(Before learning this from looking up the Safety Net mentioned in Eric W's answer (which works), I had only tested on a folder cloned from a subfolder of a TM backup, and there chmod worked. But trying chmod on a folder in an actual TM backup gives the "Operation not permitted" error.)

Of possible use:
In Mac OS 10.7+, there's a tmutil command (which I haven't tried, since I'm still on Snow Leopard). It has a delete verb, which according to the description "can delete snapshots from backups that were not made by, or are not claimed by, the current machine" (where a "snapshot" is a dated folder representing a single incremental backup). It isn't clear to me if this means that it can't delete snapshots which are made by or claimed by the current machine.(?)

Arne Stenström

Posted 2010-07-12T15:50:37.047

Reputation: 1 620

Is it possible to use tmutil to delete just some files from a snapshot? It didn't work for me (Invalid deletion target (error 22)) so I went with bypass instead. – RTS - read about Monica Cellio – 2014-12-06T14:39:48.253

+1 for BLUF. Did you get that from (the totally awesome) Manager-Tools? :) – Olie – 2015-12-04T20:18:13.950

Also: I keep finding references to a Backups.backupdb directory that I'm not seeing on my backup drive (a Time Capsule). I just have things of the form MachineName.sparsebundle. Did the format change? I'm trying to use TM's delete backup command, but it's been stuck on 99.99% progress bar for several hours. – Olie – 2015-12-04T20:20:59.130

@Olie, http://pondini.org/TM/Works3.html

– Arjan – 2015-12-19T13:28:12.310

2Indeed, tmutil is useful, and will allow you to delete backups (tmutil delete /Volumes/DISK/Backups.backupdb/HOST/DATE_FOLDER). However, you still need Eric's bypass trick to delete the "Backups.backupdb" folder itself. – mivk – 2012-06-30T13:27:29.400

My OSX 10.8.3 (Mountain Lion) did not require bypass. Just sudo tmutil delete <snapshot-dir>. The popular bypass rm answer is outdated. – John Mee – 2013-05-09T22:33:10.063

A warning about using the bypass command to remove an old backup: if the deleted backup has folders that are exactly the same in earlier or later backups, then files might be deleted from earlier or later backups as well!

Time Machine not only uses hard links for unchanged files, but also uses hard links for folders in which no files were added, changed or deleted at all. This results in something like:

/2014-11-06/folder/file1
                  /file2
                  /file3
/2014-11-13/folder/file1 = hard link to file /2014-11-06/folder/file1
                  /file2 (changed; new inode)
                  /file3 = hard link to file /2014-11-06/folder/file3
/2014-11-20/folder/ = hard link to folder /2014-11-13/folder/
/2014-11-27/folder/ = hard link to folder /2014-11-20/folder/

With the above, deleting any file from /2014-11-06/folder/ is fine, and only affects the backup for that date. The hard link reference counts are decreased, so the "inode" for file2 will be removed, but inodes for file1 and file3 will still have a reference count of 1 due to the later backups. Hence, rm -R /2014-11-06 is fine too.

However, removing any file from either /2014-11-13/folder/, /2014-11-20/folder/ or /2014-11-27/folder/ will effectively remove it from all those 3 folders.

The problem is that rm -R does not care about hard-linked folders. It just recurses into any hard-linked folder it finds, boldly deletes all its files, and then removes the empty folder.

So: when removing an old backup one should not recurse into a hard-linked folder and delete its contents. Instead, one should only remove the hard link for the folder itself. So, rather than rm -R use tmutil delete as explained in Arne's answer.

As an aside, it seems that the OS X unlink command cannot be used on folders: "only one argument, which must not be a directory, may be supplied". The OS X API can remove hard-linked folders, and so can GNU Coreutils, like installed using Homebrew.

Finally, to prove all of the above, a test-case (OSX 10.6.8):

sh-3.2# ls -lFa 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist 
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-06-012454/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-13-024438/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-20-014044/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-27-025033/Users/USERNAME/Library/Safari/TopSites.plist

Note that the number of links for each occurrence is 2 (second column). Let's remove the first occurrence:

sh-3.2# /System/Library/Extensions/TMSafetyNet.kext/Contents/MacOS/bypass unlink 2014-11-06-012454/Users/USERNAME/Library/Safari/TopSites.plist

sh-3.2# ls -lFa 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist 
-rw-r--r--@ 1 USERNAME  staff  1551 10 30  2014 2014-11-13-024438/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 1 USERNAME  staff  1551 10 30  2014 2014-11-20-014044/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 1 USERNAME  staff  1551 10 30  2014 2014-11-27-025033/Users/USERNAME/Library/Safari/TopSites.plist

So, after unlinking one of the files, the number of links dropped to 1 for each occurrence, though the file is still shown 3 times. No problems yet. Remove the first occurrence again:

sh-3.2# /System/Library/Extensions/TMSafetyNet.kext/Contents/MacOS/bypass unlink 2014-11-13-024438/Users/USERNAME/Library/Safari/TopSites.plist

sh-3.2# ls -lFa 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist 
ls: 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist: No such file or directory

Now all are gone. Apparently the file TopSites.plist was last changed 2014-11-06 and hard-linked on 2014-11-13 as then some other files were added, changed or removed in the Safari folder. Next, the contents of the Safari folder did not change in the subsequent two backups, so on 2014-11-20 and 2014-11-27 the Safari folder was hard-linked to the previous backup.

Indeed, the 4 folders only use 2 inodes (first column):

sh-3.2# ls -lFaid 2014-11*/Users/USERNAME/Library/Safari/
648651968 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:06 2014-11-06-012454/Users/USERNAME/Library/Safari//
650804457 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:07 2014-11-13-024438/Users/USERNAME/Library/Safari//
650804457 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:07 2014-11-20-014044/Users/USERNAME/Library/Safari//
650804457 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:07 2014-11-27-025033/Users/USERNAME/Library/Safari//

Kent

Posted 2010-07-12T15:50:37.047

Reputation: 1 354

Some background: for files, hard links work as expected; only the hard link you want to delete is removed. Like: touch file1; ln file1 file2; ln file2 file3; ls -li; rm file2; ls -li will only remove a single hard link. But for folders, in What is the Unix command to create a hardlink to a directory in OS X? someone wrote for 10.5 in 2010: *"Deleting is a different story: if you go about it the usual way to delete directories, you'll delete the contents. So you must "unlink" the directory: unlink new_hard_link".* This is probably still valid.

– Arjan – 2015-08-26T16:39:00.713

The man page for unlink (in 10.6.8) says that it cannot be used on directories: When the utility is called as unlink, only one argument, which must not be a directory, may be supplied. – Kent – 2015-08-26T22:31:19.430

Hmmm, mystery. Even more so, your answer is important: don't use [bypass] rm -r on hard-linked directories. (But I don't have to explain that to you.) – Arjan – 2015-08-27T07:08:37.087

I just experimented a little more. Running bypass unlink FILE has the same (unintended?) consequence as bypass rm FILE. The identical FILE is removed from all backups, not just the single location that was specified. And, unlink will NOT take a directory or more than one file as an argument (10.6.8 server; but, I don't think this will have changed through the newer OS versions) – Kent – 2015-09-10T00:00:16.827

Wow, I'm quite surprised by your bypass rm FILE and bypass unlink FILE removing all identical files, which does not match what one sees for touch file1; ln file1 file2; ln file2 file3; ls -li; unlink file2; ls -li, nor for touch file1; ln file1 file2; ln file2 file3; ls -li; /System/Library/Extensions/TMSafetyNet.kext/Contents/Helpers/bypass unlink file2; ls -li. I'm never going to remove something from a backup... – Arjan – 2015-09-10T06:47:05.367

I was (and am) surprised too. I added some terminal output to the answer above, but I still don't understand exactly what is going on. The link count being "1" when there are obviously more than one links is fairly disturbing. It's possible (nee, likely?) that previously mucking about in that directory has broken something... Your last sentence is probably the best advice in this thread. ;-) – Kent – 2015-09-10T07:20:30.997

I guess count "1" indicates those files are actually in hard-linked directories. So, all files in that folder were exactly the same as in the same folder on an older backup, so Time Machine just created a hard-linked directory. So then that file has no hard links itself, and that also explains why removing one actually removes the original? (I don't know if you can use ls to tell if directories are hard-linked, but adding a file into one folder should then also show in the others.) – Arjan – 2015-09-10T09:04:58.113

Brilliant! I think that's exactly what's going on; one less mystery in the world. – Kent – 2015-09-10T11:29:27.987

I did a major rewrite for future reference. Please see if that's okay. – Arjan – 2015-12-19T12:26:42.200

Does that mean that deleting it in descending time order is faster? – Lenar Hoyt – 2017-09-25T13:19:39.367

Note: Due to the "TM Safety Net" mentioned by Eric W, this answer doesn't work for the case of a Time Machine backup, which the question specifically pertains to. But for most every other case, the information about how to get rid of ACLs is relevant.

There's no need to use ACL tools copied from an older OS.

Use ls -le to view ACLs and chmod to alter them.

For more info, type man chmod and look under ”ACL Manipulation Options” .

The command to remove all ACLs from a folder hierarchy is:

chmod -R -N foldername

Ram

Posted 2010-07-12T15:50:37.047

Reputation: 977

Time machine works like rshapshot. It creates a tree of hard links for each new backup. Hard links to files already existing in a previous backup use very little additional space. Only when the last hard link to a file is removed is the file actually deleted from the filesystem.

Removing an entire individual backup won't hurt. You're just removing hard links. No other backup will be affected. But that can be accomplished via tmutil.

One scenario in which it may be necessary to bypass protection is to remove a specific file from all backups (and the reason why I ended up on this post).

My backup disk is full. I have a very large file (many gigabytes) that has been backed up for months. There is one physical copy of it, but many snapshots with hard links to that copy. To actually get rid of that file, I need to remove the hard link from every backup.

Note that the inode number is the same for all hard links to the same file.

% cd /Volumes/WD\ 500G\ USB/Backups.backupdb/csm-laptop
% ls -li */Macintosh\ HD/Users/csm/vm.img
...
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-005636/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-015812/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-030036/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-041307/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 Latest/Macintosh HD/Users/csm/vm.img

(Latest is just a symlink to the last dated directory)

% sudo bypass rm -f */Macintosh\ HD\Users\csm\vm.img

The file is removed from all backups, and space is returned. If the file has been changing over time, each backup will have a full copy and the space returned will be enormous.

Curt

Posted 2010-07-12T15:50:37.047

Reputation: 216

As for "Removing an entire individual backup won't hurt. You're just removing hard links. No other backup will be affected.": if this refers to snapshots within backups for a single machine, then given Kent's (old) answer using rm -r might be (or: have been) dangerous. Better be safe than sorry and use tmutil for that.

– Arjan – 2015-08-26T16:41:25.847

If you want to delete all files in a folder and not just specific files, you can accomplish this by adding the folder to Time Machine's exclusion list. (System preferences -> Time Machine -> Options. Drag the folder here.)

Next time you perform a backup, copies of that folder will be removed from previous backups.

Now, if you really want to do this from a CLI, there's a way, albeit a little cumbersome.

Make a backup of /Library/Preferences/com.apple.TimeMachine.plist
Copy /Library/Preferences/com.apple.TimeMachine.plist somewhere where you can toy with it.
Cd into whereever you put it.
Execute
plutil -convert xml1 com.apple.TimeMachine.plist
to convert it from binary form.
Open the converted plist in preferable text editor, search for "skippaths"
Insert a new line in that section, formatted as <string>/Path/To/Exclude</string>
Save and exit, convert back by executing
plutil -convert binary1 com.apple.TimeMachine.plist
Copy your edited plist back into /Library/Preferences/
Start a backup by executing
/System/Library/CoreServices/backupd.bundle/Contents/Resources/backupd-helper -auto

Edit: When you perform step 9, all copies of the newly excluded folder will be erased from previous backups.

To remove the exception, copy your backup back into /Library/Preferences.

Frost

Posted 2010-07-12T15:50:37.047

Reputation: 268

I'm not trying to exclude a directory. I'm trying to delete a directory from an existing backup and to do some from the commandline. Really, I'd just like to figure out how to allow rm to work within a Backups.backupdb directory. – Tim – 2010-07-12T20:30:01.683

OK, perhaps it wasn't clear from my instructions that the folder is actually deleted from the backups when performing the above steps. See my edit.

But if it's not the solution you are looking for, then I suppose you could always su - root and then rm -rf the folders, but I think fiddling with something as precious as backups in that way is something one should generally try to avoid. – Frost – 2010-07-13T06:27:06.490

But adding it as an exclusion would delete all copies of it from all the backups. Also, though this is a TimeMachine backup, it is not from the machine that I'm doing this work on, so it is no longer an active TimeMachine. sudo has the same effect as "su - root" and then "rm -rf". I'm pretty sure this is failing because Apple has added a level of security to the filesystem beyond simple *nix permissions. – Tim – 2010-07-13T15:11:20.980

This might sound stupid, but I'm not entirely sure that sudo has the same effect as su - root on a mac. I seem to recall an occasion when I was trying to delete something where sudo didn't suffice, but sudo - root did the trick. – Frost – 2010-07-14T12:58:27.873

For completeness sake, I did "su -" and confirmed that the files still could not be deleted. And considering the number of times that the "stupid" idea turned out to be correct, I'd never discount them without trying. – Tim – 2010-07-14T15:41:14.163

A number of days later, I realise that i meant that you should try deleting as root, by executing sudo su. This works for me in cases where a simple sudo doesn't do it. I simply remembered the command wrong in my earlier comments. – Frost – 2010-08-11T17:58:00.577

3@Frost- I don't think Time Machine works the way you suggest. I just tried excluding a folder, then running TM, and the older backups of that folder are still present in TM. Or perhaps its behavior has changed since you posted this nearly a year ago. – Caffeine Coma – 2011-06-01T16:20:36.037

If you're not executing the command as the user that "owns" the backup, you're going to have a hard time deleting from the command line. I just had this issue with a migration, and we had to ditto the entire Time Machine backup (1tb+) and format the drive before we could get any sort of access to it - and trust me, I tried everything to override the permissions.

Nic

Posted 2010-07-12T15:50:37.047

Reputation: 966

Nic, a few years late, but in case you missed it: see Eric's answer.

– Arjan – 2015-08-26T15:13:01.940

@Tim: this assertion would even be untrue on other unixoid systems, given the immutable attribute (chattr). What root will give you theoretically is to circumvent any security net. – 0xC0000022L – 2015-11-19T17:37:13.423

2sudo should give me access to all files as I'm running as administrator. I'm pretty sure this is an ACL issue and I'm working on that. – Tim – 2010-07-13T15:09:25.103

You can make ls list extended attributes in a long view using the -@ flag. It will list ACLs when you supply the -e flag. So, you can find out what you're dealing with by using ls -lea@ DIR.

Judging by my local Time Machine backups, it looks like Time Machine applies extended attributes with metadata about the newest and oldest snapshots. The data stored by the xattrs looks to be a binary plist. These seem innocuous.

Time Machine also looks to apply ACLs to certain directories it knows about, such as the ones placed in a standard user directory. There are two kinds of ACLs potentially standing in your way: those applied directly to the file or directory that deny delete, and those applied to a parent of the file that deny delete_child.

Unfortunately, Mac OS X does not supply the user utilities getfacl and setfacl specified by POSIX.2c to view and manipulate ACLs. To mess with ACLs, you'll have to do some programming; see the acl(3) manpage.

Jeremy W. Sherman

Posted 2010-07-12T15:50:37.047

Reputation: 606