An application has a file extension validation at the server side and rejecting the request if the Extensions are not matching to (.pdf/.docx). But if i modify an exe file's extension to .pdf and upload it, its processing the request. Is this a security vulnerability?? If so, how an attacker can take advantage of?
PS: Application is using this feature to attach some support pdf documents where an approver can download and refer. Its not processing the uplaoded files anywhere in the application.
It is unclear what the expectations on the application are in this case and thus your question can not be answered.
For example if the recipient expects data from untrusted sources anyway, it can be expected that the PDF is checked at the recipient. If the recipient instead expects fully trustable data, then more checks need to be done on the way to the recipient. These might be done in the web application during upload but also might be done at a later stage during delivery.
In other words: you cannot have a look at the application in isolation but must see the bigger picture. From that you can derive the requirements on the application. Only with the actual requirements you can decide if this is a vulnerability in the application or not. And only with the bigger picture it can be determined if and how such a vulnerability could actually be exploited.
As Steffen Ullrich said, it's not sufficient to consider a single application alone, but I'll write my answer assuming the user/application receiving the file trusts these uploads (which is bad if the file comes directly from an untrusted user).
On Windows, (I don't use Windows right now) the identification of file type, and thus what program(s) to read/run it with, is mostly done with file extensions. This means a .pdf will be opened by a PDF reader by default, and should fail if the file is actually a .exe file with a .pdf extension. However, since the user, assuming they're not very computer-literate, may change some things to get it to work, such as renaming the file or changing what program to run the file with, all without understanding what problems that would cause. Additionally, there's the option to hide file extensions on Windows, meaning a file not_a_virus.pdf.exe would be seen by the user as not_a_virus.pdf while have a .exe extension. Some programs may also check the file's MIME type like a Linux system would, which I'll discuss next.
On Linux systems (a lot of web servers are Linux), the file type is determined not (mainly) by file extensions, but by MIME types, using magic bytes. Binary files should have some magic bytes at the beginning of the file to identify the MIME type, as listed on Wikipedia. If a file has the bytes 47 49 46 38 37 61 at the beginning, running file my_file should return my_file: GIF image data, version 87a,, since those are the magic bytes of GIF files. Also, if a script (text) file has the line #!/bin/bash on the first line, it will be identified as a "bash script", and running ./my_script will use the bash binary to read/run the file.
As per best practice, you should validate both the file extensions & the file contents. Some programs use only the file extensions, some use only the contents (magic bytes), and some use both to identify the file type. You should simply assume the worst and not trust user inputs.
The existing answers do a great job in explaining the best practices from a defensive side. I would like to answer this from testing perspective.
A file upload issue can be exploited if the following two conditions are met:
If both these conditions are met then the chances of exploitation are highest.
Based on the information provided in the question, I would recommends trying techniques to bypass the extension validations: see OWASP cheatsheet also see What does it mean to have a "file name with NULL bytes in serialized instances"?
You should also test if there are any checks restricting the size of the file being uploaded, if there are no limits, a malicious user can exhaust the storage available on the server causing a DOS.
Do refer to OWASP documentation to see if any other tests are applicable in your scenario.
You should not trust any extension on files in general, special if your services offers functionality for upload files. If you service offers a service for upload files my recommendation is to use a anti-virus to scan the file. This make cost you money in terms of having the AV license, or you can go to open source solutions. Also you can build your own stuff with Yara signatures but this will require human effort for maintain the signatures.
