Removing an (apparently) infinitely recursive folder

Question

Somehow, one of our old Server 2008 (not R2) boxes has developed a seemingly infinitely-recursing folder. This is playing havock with our backups, as the backup agent tries to recurse down into the folder and never returns.

The folder structure looks something like:

C:\Storage\Folder1
C:\Storage\Folder1\Folder1
C:\Storage\Folder1\Folder1\Folder1
C:\Storage\Folder1\Folder1\Folder1\Folder1

... and so on. It's like one of those Mandelbrot sets we used to all play with in the 90's.

I've tried:

Deleting it from Explorer. Yeah, I'm an optimist.
RMDIR C:\Storage\Folder1 /Q/S - this returns The directory is not empty
ROBOCOPY C:\temp\EmptyDirectory C:\Storage\Folder1 /PURGE - this spins through the folders for a couple of minutes before robocopy.exe crashes.

Can anyone suggest a way to kill this folder off for good?

Have you tried `rmdir` without the quiet switch? `rmdir C:\storage\folder1 /s` — Mathias R. Jessen, Feb 12 '15 at 11:53
I'd try `/MIR` instead: `ROBOCOPY /MIR C:\temp\EmptyDirectory C:\Storage\Folder1` also may be worth running a `chkdsk` just for giggles. — jscott, Feb 12 '15 at 14:16
`/MIR` seemed to last longer, but eventually bombed out too ("robocopy has stopped working"). I'm a bit scared to do a `chkdsk`; this is a pretty old server and I'm worried that this problem is indicative of bigger file system problems... — KenD, Feb 12 '15 at 15:38
Somewhat related, hopefully some useful info for you: http://superuser.com/questions/620442/how-can-one-delete-recursive-directories-in-windows — chue x, Feb 12 '15 at 15:58
And also this one: http://superuser.com/questions/416351/how-to-remove-an-infinitely-recurring-directory-tree — Calimo, Feb 12 '15 at 16:14
Try booting from a Linux (Ubuntu/Centos/Fedora/...) desktop trial CD and removing the folder from there. — Guntram Blohm, Feb 12 '15 at 17:00
@KenD If you suspect file system corruption issues, you should certainly try repairing the filesystem first. Trying directory removal tricks might make things worse. — jscott, Feb 12 '15 at 17:05
Since (from your answer below), the directory was not infinite, just very deep, if you had [CygWin](https://en.wikipedia.org/wiki/Cygwin) or [UnxUtils](https://en.wikipedia.org/wiki/UnxUtils) installed, you could use `find` to do a depth first directory removal: `find Storage/Folder1 -depth -exec rmdir {} \;` — Johnny, Feb 13 '15 at 01:25
@Johnny I imagine the same would work on [MSYS2](http://sourceforge.net/projects/msys2/). — jpmc26, Feb 13 '15 at 20:10
If long pathnames are the problem, `find -exec` won't help. Use `-execdir` to use a relative path while CDed to the right place. — Peter Cordes, Feb 15 '15 at 05:05

KenD · Accepted Answer · 2015-02-13T20:23:37.457

45

Thanks to everyone for the useful advice.

Straying well into StackOverflow territory, I've solved the problem by knocking up this snippet of C# code. It uses the Delimon.Win32.IO library that specifically addresses issues accessing long file paths.

Just in case this can help someone else out, here's the code - it got through the ~1600 levels of recursion I'd somehow been stuck with and took around 20 minutes to remove them all.

using System;
using Delimon.Win32.IO;

namespace ConsoleApplication1
{
    class Program
    {
        private static int level;
        static void Main(string[] args)
        {
            // Call the method to delete the directory structure
            RecursiveDelete(new DirectoryInfo(@"\\server\\c$\\storage\\folder1"));
        }

        // This deletes a particular folder, and recurses back to itself if it finds any subfolders
        public static void RecursiveDelete(DirectoryInfo Dir)
        {
            level++;
            Console.WriteLine("Now at level " +level);
            if (!Dir.Exists)
                return;

            // In any subdirectory ...
            foreach (var dir in Dir.GetDirectories())
            {
                // Call this method again, starting at the subdirectory
                RecursiveDelete(dir);
            }

            // Finally, delete the directory, and any files below it
            Dir.Delete(true);
            Console.WriteLine("Deleting directory at level " + level);
            level--;
        }
    }
}

edited Feb 13 '15 at 20:23

answered Feb 12 '15 at 18:44

KenD

1,127
2
17
35

Any reason not to just use `Directory.Delete(..., true)`? – Cole Tobin Feb 12 '15 at 22:55
2

I tried, even using the Delimon version of `.Delete` (instead of the normal `System.IO` version), and although it didn't throw an exception it didn't seem to do anything. Certainly the recursion using the method above took ages, and `.Delete` only chewed on things for 5-10 seconds. Maybe it chipped away at a few directories and then gave up? – KenD Feb 12 '15 at 23:01
4

Did you ever figure out how that happened to begin with? Sounds like some really badly written userland program's fault. – Parthian Shot Feb 13 '15 at 00:30
I have *no* idea - this is a very old and crufty server, that's had literally hundreds of different tools installed and subsequently removed over the years, and all sorts of strange people accessing via RDP. I suspect some kind of NTFS weirdness; the server is due to be retired in the coming months, so before it's reassigned to propping-up-a-wobbly-table duty, I'll run `CHKDSK` on it and see what it finds. – KenD Feb 13 '15 at 08:14
8

Recursing into a function 1600 times? [Stack overflow](http://en.wikipedia.org/wiki/Stack_overflow) territory indeed! – Aleksandr Dubinsky Feb 13 '15 at 11:53
2

Just as an aside, were the folders populated by anything? If you can determine at what intervals the recursive folders were created and multiply that by the number of recursions, you will (hopefully) get a rough timeframe of when this problem started... – Get-HomeByFiveOClock Feb 13 '15 at 14:18
1

@Get-HomeByFiveOClock: nothing that I could see in the folders, but I only went down a hundred or so levels :) I wish I'd taken note of the timestamps of the folders - you're right, it would have been interesting to see if they were all the same timestamp, or over a period of time ... – KenD Feb 13 '15 at 15:06
8

Wow, glad you ended up solving this. FYI, the official Microsoft supported fix to this kind of situation is "reformat the volume." Yeah, seriously. :/ – HopelessN00b Feb 13 '15 at 15:56
1

@AleksandrDubinsky: My old laptop still has a near-infinitely deep Eclipse workspace on it. I'm fairly sure running this program *will actually overflow* if I tried running it. – Lilienthal Feb 13 '15 at 19:02
1

The fact that it wasn't actually infinately recursive but just very deep is significant and was a major red herring assumption. – JamesRyan Feb 14 '15 at 16:31
1

This took 20mins to run because you're passing the ever-growing path to the OS every time you list a directory, so it has to check every component of the path every time. And again when actually removing each directory. `O(sum(1..n)) = O(n^2)`. I posted an answer with 2 methods that don't have this problem. – Peter Cordes Feb 15 '15 at 05:45

Brian · Answer 2 · 2015-02-12T13:52:22.980

25

Could be a recursive junction point. Such a thing can be created with junction a file and disk utility from Sysinternals.

mkdir c:\Hello
junction c:\Hello\Hello c:\Hello

And you can now go endlessly down c:\Hello\Hello\Hello.... (well until MAX_PATH is reached, 260 characters for most commands but 32,767 characters for some Windows API functions).

A directory list shows that it is a junction:

C:\>dir c:\hello
 Volume in drive C is DR1
 Volume Serial Number is 993E-B99C

 Directory of c:\hello

12/02/2015  08:18 AM    <DIR>          .
12/02/2015  08:18 AM    <DIR>          ..
12/02/2015  08:18 AM    <JUNCTION>     hello [\??\c:\hello]
               0 File(s)              0 bytes
               3 Dir(s)  461,591,506,944 bytes free

C:\>

To delete use the junction utility:

junction -d c:\Hello\Hello

edited Feb 12 '15 at 13:52

answered Feb 12 '15 at 13:28

Brian

3,386
17
16

4

Unfortunately, a `DIR` just shows me plain ole directories - no sign of any junctions I'm afraid – KenD Feb 12 '15 at 15:36
2

Can you do a quick double check with `junction -s C:\Storage\Folder1 ` ? – Brian Feb 12 '15 at 17:31
3

`No reparse points found` :( – KenD Feb 12 '15 at 18:31
3

I wonder what created such a mess of actual sub directories. – Brian Feb 12 '15 at 18:51
2

Use `dir /a` to see '' without specifying the specific name. – Chloe Feb 13 '15 at 00:55
@Brian: Simple: a batch file that 1. creates `Folder1` and 2. recursively copies an entire directory tree to that `Folder1\ `. Now watch what happens if `Folder1` is created in the tree to be copied. – MSalters Feb 13 '15 at 13:36

score 17 · Answer 3 · answered Feb 13 '15 at 14:06

17

Not an answer, but I don't have enough rep for a comment.

I once fixed this problem on a then-huge 500MB FAT16 disc on an MS-DOS system. I used DOS debug to manually dump and parse through the directory table. I then flipped one bit to mark the recursive directory as deleted. My copy of Dettman and Wyatt 'DOS Programmers' Reference' showed me the way.

I am still inordinately proud of this. I would be amazed and terrified if there is any general-purpose tool that has such power over FAT32 or NTFS volumes. Life was simpler back then.

answered Feb 13 '15 at 14:06

Richard

179
2

8

I would say that you are *justifiably* proud of this. – mfinni Feb 13 '15 at 14:37
3

+1 have some rep on me. Nice solution. – Chris Thornton Feb 13 '15 at 15:02
3

You can now delete the first sentence of your answer. – A.L Feb 14 '15 at 22:36
This is not an answer. It's a story about a different operating system and a different filesystem. In addition to being of no help for this (NT, NTFS) problem, it wouldn't even help someone with the same problem (DOS, FAT16) because it doesn't actually contain any details. – nobody Feb 16 '15 at 01:12
@Andrew Medico: I agree with you, hence my first sentence. But I do tell you where to find the info to solve the slightly-related problem. – Richard Feb 16 '15 at 10:33
@All: Thank you for your kind comments. With the information to hand, the most tricky bit was calculating the next sector number to dump as I parsed the directory tree. It only took about an hour and a half in all. Life emphatically was easier then. – Richard Feb 16 '15 at 10:41
You could probably do something like this for XFS or EXT4, but then you'd want to run `fsck` to update the free-lists, or else the filesystem would never be able to reuse the metadata space (inodes) for any of the directories in that orphaned/dangling subtree. Actually, `fsck` would probably link the dangling subtree back into `/lost+found`. – Peter Cordes Jul 26 '17 at 23:45

SpiderPig · Answer 4 · 2015-02-14T17:43:17.223

Java can also deal with long file paths. And it can do it a lot faster too. This code (which I copied from the Java API documentation) will delete a 1600 level deep directory structure in about 1 second (under Windows 7, Java 8.0) and with no risk of stack overflow since it doesn't actually use recursion.

import java.nio.file.*;
import java.nio.file.attribute.*;
import java.io.*;

public class DeleteDir {

  static void deleteDirRecur(Path dir) throws IOException {
    Files.walkFileTree(dir, new SimpleFileVisitor<Path>() {
         @Override
         public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
             throws IOException
         {
             Files.delete(file);
             return FileVisitResult.CONTINUE;
         }
         @Override
         public FileVisitResult postVisitDirectory(Path dir, IOException e)
             throws IOException
         {
             if (e == null) {
                 Files.delete(dir);
                 return FileVisitResult.CONTINUE;
             } else {
                 throw e;
             }
         }
     });
  }

  public static void main(String[] args) throws IOException {
    deleteDirRecur(Paths.get("C:/Storage/Folder1"));
  }
}

I hate you. You've forced me to upvote an answer that involves using Java, and now I feel all dirty. I need a shower. — HopelessN00b, Feb 14 '15 at 19:37
My apologies. I hope you will recover eventually from your trauma. But is a language developed by Microsoft (c#) really much better? — SpiderPig, Feb 14 '15 at 19:49
I'm primarily a Windows guy these days, so yes, yes it is. I may also be bitter and biased against Java on account of having to maintain 5 specific, different versions of JRE across almost 1,000 clients in my employer's environment, one of which dates back to 2009, on account of the crap... er, malware... uh, "enterprise-y" software suites that we use for business critical applications. — HopelessN00b, Feb 14 '15 at 19:55

Peter Cordes · Answer 5 · 2015-02-15T06:02:14.853

You don't need long pathnames if you chdir into the directory and just use relative paths to rmdir.

Or, if you have a POSIX shell installed, or port this to the DOS equivalent:

# untested code, didn't bother actually testing since the OP already solved the problem.

while [ -d Folder1 ]; do
    mv Folder1/Folder1/Folder1/Folder1  tmp # repeat more times to work in larger batches
    rm -r Folder1     # remove the first several levels remaining after moving the main tree out
    # then repeat to end up with the remaining big tree under the original name
    mv tmp/Folder1/Folder1/.../Folder1 Folder1 
    rm -r tmp
done

(Using a shell variable to track where you renamed it for the loop condition is the other alternative to unrolling the loop like I did there.)

This avoids the CPU overhead of KenD's solution, which forces the OS to traverse the tree from the top to the nth level every time a new level is added, checking permissions etc. So it has sum(1, n) = n * (n-1) / 2 = O(n^2) time complexity. Solutions that pare off a chunk from the start of the chain should be O(n), unless Windows needs to traverse a tree when renaming its parent directory. (Linux/Unix doesn't.) Solutions that chdir all the way down to the bottom of the tree and use relative paths from there, removing directories as they chdir back up, should also be O(n), assuming the OS doesn't need to check all your parent directories every system call, when you do things while CDed somewhere.

find Folder1 -depth -execdir rmdir {} + will run rmdir while CDed to the deepest directory. Or actually, find's -delete option works on directories, and implies -depth. So find Folder1 -delete should do the exact same thing, but faster. Yeah, GNU find on Linux descends by scanning a directory, CDing to subdirectories with relative paths, then rmdir with a relative path, then chdir(".."). It doesn't rescan directories while ascending, so it would consume O(n) RAM.

That was really an approximation: strace shows it ACTUALLY uses unlinkat(AT_FDCWD, "tmp", AT_REMOVEDIR), open("..", O_DIRECTORY|...), and fchdir(the fd from opening the directory), with a bunch of fstat calls mixed in, too. But the effect is the same if the directory tree isn't getting modified while find is running.

edit: Just for kicks, I tried this on GNU/Linux (Ubuntu 14.10, on a 2.4GHz first-gen Core2Duo CPU, on an XFS filesystem on a WD 2.5TB Green Power drive (WD25EZRS)).

time mkdir -p $(perl -e 'print "annoyingfoldername/" x 2000, "\n"')

real    0m1.141s
user    0m0.005s
sys     0m0.052s

find annoyingfoldername/ | wc
   2000    2000 38019001  # 2k lines / 2k words / 38M characters of text


ll -R annoyingfoldername
... eventually
ls: cannot access ./annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername/annoyingfoldername: File name too long
total 0
?????????? ? ? ? ?            ? annoyingfoldername

time find annoyingfoldername -delete

real    0m0.054s
user    0m0.004s
sys     0m0.049s

# about the same for normal rm -r,
# which also didn't fail due to long path names

(mkdir -p creates a directory and any missing path components).

Yes, really 0.05 seconds for 2k rmdir ops. xfs is quite good at batching metadata operations together in the journal, since they fixed meta data ops being slow like 10 years ago.

On ext4, create took 0m0.279s, delete with find still took 0m0.074s.

got curious and tried it on Linux. Turns out the standard GNU tools are all fine with long paths, because they recurse down the tree instead of trying to make system calls with giant long paths. Even mkdir is fine when you pass it a 38k path on the command line! — Peter Cordes, Feb 15 '15 at 06:05

score 1 · Answer 6 · answered Mar 31 '17 at 16:24

I too had this, on a standalone Windows 10 system though. C:\User\Name\Repeat\Repeat\Repeat\Repeat\Repeat\Repeat\Repeat seemingly to infinity.

I could navigate using Windows or Command Prompt to about the 50th one and no further. I could not delete it, or click on it, etc.

C is my language so eventually I wrote a program with a loop of system calls, which repeat until they fail. You could do this in any language though, even DOS batch. I made a directory called tmp and moved Repeat\Repeat into that, deleted the now-empty Repeat folder, and moved tmp\Repeat back into current folder. Over and over again!

 while (times<2000)
 {
  ChkSystem("move Repeat\\Repeat tmp");
  ChkSystem("rd Repeat");
  ChkSystem("move tmp\\Repeat Repeat");
  ++times;
  printf("Removed %d nested so far.\n", times);
 }

ChkSystem just runs a system() call and checks the return value, stopping if it failed.

Importantly, it failed a number of times. I thought perhaps my program wasn't working, or that it was infinitely long after all. However, I have had this before with system calls, with things not syncing up, so I just ran the program again and it carried on from where it left off, so don't immediately think your program isn't working. So in total, after running it about 20 times, it cleared them all. In total, it was about 1280 folders deep originally. No idea what caused it. Crazy.

Imanol Barba Sabariego · Answer 7 · 2015-08-08T22:35:12.310

0

I did run into the same issue with a 5000+ directory-deep folder mess that some Java application did and I wrote a program that will help you remove this folder. The whole source code is in this link:

https://imanolbarba.net/gitlab/imanol/DiREKT

It removed the whole thing after a while, but it managed to do the job, I hope it helps people who (as I), run into the same frustrating issue

edited Aug 08 '15 at 22:35

answered Aug 08 '15 at 18:35

Imanol Barba Sabariego

3
7

Please don't post link-only answers. You should put the most important info from the link in the post itself and provide the link for reference. – Frederik Aug 08 '15 at 22:14
Oh sorry, it's a program, and I really did not want to post ALL the source code here... I think it's **pretty clear** that I wrote a program and I'm hosting it following this link, and the motivation and all is in the answer, so it looks pretty clear to me its not a link-only answer, nontheless, I'll specify (more clearly) that it is a software meant to be run to solve this problem – Imanol Barba Sabariego Aug 08 '15 at 22:33

Removing an (apparently) infinitely recursive folder

7 Answers7