In a recent pen test of a web application one of the issues found was a 'backup file'. This was a javascript file that was renamed to filename.js1 when an updated version of filename.js was uploaded.
The 'backup file' lives in a directory with forbidden listing and is not referenced or used anywhere in the application.
How did they find this file?
Many automated scanners get around banned directory listings by "bruteforce" searching for files. This means that they will check for additional files with names similar to files that do exist (i.e. filename.js1 as well as files that aren't referenced at all (aka secret.txt). If you happen to have a file whose name is on the bruteforced list and is in an accessible directory, then it will be found regardless of whether or not "directory listing" is enabled
It's worth pointing out that hackers do this same thing, so this is a real issue. In general, if something is in a publicly accessible directory, then you should assume it will be found. So if you don't want it to be public then you need to keep it out of public directories - disabling directory listing provides very little security.
Finally, this may not seem like a large issue (and it probably isn't), but leaving backups of javascript files in public directories is actually a bad idea in general. When it comes to XSS an attacker will generally have the most success if they are able to exploit a javascript file hosted on the same domain. This is because doing so gives opportunities to bypass a CSP or other security "firewalls". As a result, if an older javascript file happened to have a security vulnerability in it that was fixed in a later version, and an attacker found a way to force the user's browser to load the older javascript file, they may chain their way to a more damaging vulnerability. This may seem far fetched, but chaining together many small vulnerabilities into a larger one is how many of the worst breaches happen.
tl/dr: If something is hosted by your website but doesn't have a reason to be there, then it is a liability. Kill it with prejudice.
There are many tools available which brute-force filenames. Some of these are more intelligent than others.
For instance, a "dumb" tool may just have a word list, containing probable names for files and directories, such as
/admin/wp-admin.phplogin.phpA more intelligent tool may look at the files which it already knows about (e.g. by crawling the application) and try to find similarly-named files. In your case, there was a file named filename.js, so the application likely tried to mangle the name, as TripeHound pointed out in a comment:
filename.js1filename.js.bakfilename.bak.js.filename.jsOne might be tempted to think that an unreferenced file is "safe", because it's not a part of the application. However, the file is still accessible, and depending on the contents of the file, this may allow an attacker to do various things:
In general, it's best to avoid having unreferenced files in your webroot. As the name implies, they are not used by the application and thus are only a source of problems.
The real problem here is that you have a deployment / production environment that is not controlled (and thus replicable) through an automated source control and deployment system.
This means that, if you find some new file in your system, you don't know if that's some kind of backdoor dropped by a root kit, or some innocuous renamed file your colleague left behind.
In general, a best security practice is to only ever have files on a server that are put there by an automated script that clones some kind of build artifacts, and to have that automated process also delete files that should no longer be there. Then you can run audits for "are the files in production what the build system says they should be?"
And if you think that "bad deployment practices can't possibly be a life threatening problem for my business," then I invite you to google "Knight Capital Group."
The same way an attacker would: by guessing.
That's why you have pen testers: to test things that you may not have thought of.
Remove the backup file from your application so that it is not accessible.
How did they find this file?
By very simple "brute force" guesswork, as others have already mentioned.
You will be able to see this as it happened, the request for this file along with the other guesses that were made, in your web server's logs. Unless, of course, the testers managed to find a hold that allowed them to reset your logs (which would be listed in your test report) or you didn't have sufficient logging enabled.
but being a less-predictable-renamer is probably not a bad idea either
It is good practise to avoid old files being in your application directory at all. Even in your source directories, especially if your deployment model is "copy from source".
Instead of keeping old versions of files around for reference or other reasons, make use of the features offered by your version control arrangement. All VCSs will let you retrieve older versions of files, some will let you shelve intermediate versions without properly checking them in, you can use branching to separate out experimental work, etc.
