This sounds like a driver related problem. Most laptop keyboards come with special software that enable the alternate function key. It's possible that there is a "function lock" that inverts the FN key to always on.
Since it only breaks once you launch into windows, there is probably a program that is causing this to break. Go to the Run line and type in MSCONFIG. Go to the startup tab and uncheck any Dell related executables. Restart the computer and see if that makes a difference. If you disable everything and it still breaks, then it is a service that is causing the keyboard to break.
Once you identified the program or service that is causing the problem, you can uninstall it and look for an updated version that doesn't have this bug. Unfortunately, if Dell doesn't have a patched version, then there is not much you can do the "fix" the issue and still retain FN functionality.