How secure is the new Windows 8 anti virus known as Windows Defender?
Does it have a protection against malware which uses UAC bypass/process injection/rootkits/process persistence/running the binary directly in the memory? How much can I trust Windows Defender? Is it better than regular AVs like Kaspersky/AntiVir?
Let us analyze each one of the techniques you want the AV to protect against:
UAC Bypass: Any process in the Windows environment running with the trusted root certificate can turn off the UAC bit of its own process, as well as any process spawned by it. This means that if your malicious code can inject itself into a process running with the trusted cert, it will have all the privileges of the injected process. Then, if you create another process, you can easily turn its UAC bit off, because this is a built-in feature of Microsoft Windows. This is the technique employed by the Metasploit framework for UAC Bypass.
Process Injection: Microsoft provides an API called LoadLibrary through which you can load any arbitrary DLL from the disk into a running process. The only thing that malicious code does is load the arbitrary DLL from within memory, and not from the disk. This is achieved through a technique called Reflective DLL Injection, which Meterpreter makes use of as well.
Root Kit Detection: Rootkits operate at ring zero (kernel level), while antivirus products run in userspace. Most of the time, the AV only hooks certain APIs in kernel land. Any process running below the user space cannot be analyzed by the AV. Before Vista, AV products used to load drivers in the kernel for monitoring. However, after the introduction of PatchGuard, that technique can no longer be used by antivirus software.
Running the Process Directly Within Memory: This is an area where AVs have made some progress. Nowadays, even if you are directly interacting with a running process, the AV examines the traffic received by the process from the network, and checks it for malicious signatures. However, there are two shortcomings to this approach: first, it is signature based checking, so it is inherently weak. Secondly, it is done only for common Windows processes such as SMB.
As you can see, the things you most want to protect against, are the kinds of things against which no AV product can effectively defend. Most of the items you have mentioned are not malicious by nature. Rather, these are considered "features." In Windows 8, Windows Defender is the combination of Microsoft Security Essentials and Microsoft Defender software. On the plus side, it is free, and has low performance impact. However, if you really want to protect against the techniques you have mentioned, Windows Defender, or any other AV product, won't be able to provide an effective solution. For these kind of attacks, Microsoft has another product called the Enhanced Mitigation Experience Toolkit (EMET).
I should start this by saying that no modern A-V is bullet proof and all of them can be bypassed by a determined attacker.
So the decision about whether to trust Windows 8's built-in A-V comes down to your priorities and the level of security that you're looking to achieve.
There have been some comparative analyses of A-V solutions which reckon that defender is poorer against targeted attacks than other solutions but then I'd suggest that all A-V is to some extent poor against that sort of thing.
The advantages of defender that I can see are that it is integrated with the OS so less likely to disrupt the operation of the system and also in my experience has a low performance hit compared to other A-V systems that I've used.
So as I say it's a trade-off between the level of security desired against the convenience of the solution.
If you're looking for really high levels of security I'd be more inclined to look at solutions like bit9 which use a white-listing approach rather than the traditional signature based approach that A-V uses.
