Is WebGL a potential security problem due to the low level access it provides?
For example, a web page can attempt to compile and run any shader source it wants.
It seems that security would especially be a problem with open source web browsers, as an attacker could more easily find vulnerabilities in the implementation.
Yes, WebGL is indeed a potential security risk, though the magnitude of the risk is hard to assess and open to debate. There are some tricky issues here. The browsers have put in place some defenses against the security risks, but there seems to be some debate about whether those defenses will prove adequate in the long run.
One major risk is that WebGL involves running code directly on the video card, and exposing APIs that provide direct access to video card APIs. The browser does attempt to sandbox this code (to a certain extent), and browsers do enforce a number of security restrictions designed to prevent malicious behavior. However, many of these APIs and their implementations were not originally designed to be provided to untrusted entities (they were only usable by native applications, which are fully trusted), so there are concerns about whether exposing them to arbitrary web sites might enable web sites to attack your system.
There was one high-visibility white paper (see also the sequel) which looked at the security of the WebGL implementation in browsers at the time, and found a number of vulnerabilities. They found some memory safety issues in several WebGL APIs, and also found some attacks that would allow one web site to read pixel data of other web sites (which could enable a breach of confidentiality). See also this third study, which demonstrated the existence of these vulnerabilities on a number of browsers and web cards (at the time).
Browsers have responded to this with a variety of defenses: they have blacklisted video cards with known security problems; they have tried to fix the known memory safety problems; and they have restricted use of WebGL per the same-origin policy, to prevent a malicious web site from using WebGL to spy on users' use of other web sites.
There is some ongoing debate over whether these defenses will prove adequate in the long term. Microsoft has taken the position that WebGL is too great a security risk and the existing defenses are not robust enough. On the other hand, Mozilla takes the position that the defenses they have put in place will be adequate, and that WebGL provides important value to the web. Ars Technica has an excellent round-up of the issue; and here is another press report.
P.S. I completely disagree with your statement about it being particularly a problem for open source web browsers. That's a myth. See Open Source vs Closed Source Systems, which already covers these arguments. (See also Chrome vs Explorer - how to explain in plain words that open-source is better? for additional thoughtful discussion on this topic.)
Some key points:
It seems that security would especially be a problem with open source web browsers.
You sir are very wrong. It has proven many times that opensource programs are often a lot more secure than closed source ones, because there are a lot more eyes that check the code.
Also all of the browsers run these things in a sandbox. Breaking out of the sandbox will be difficult, but it will be as much of a problem in closed source as in open-source browsers.
Let's compare WebGL with javascript.
The critical difference between (shader code run via) WebGL and javascript is that the shader code is run on the GPU card, while javascript is run on the CPU.
Whether the code is interpreted or compiled is of little consequence; the resulting potential for vulnerabilities is still there.
So javascript has more capacity for abuse, since once a rogue script breaks out of the browser jail it basically has access to your PC. A rogue WebGL shader script could get, uh, access to your GPU.
There are factors that complicate this simple view; javascript has been around for a while, so has received more scrutiny and had more holes closed. javascript is much more complicated. Javascript as a language is much more complicated. etc etc.
WebGL enables access to the GL pipeline to your GPU cores. Some manufacturers integrate GPUs directly on the CPU chip. This allows the GPU to share internal CPU memory, rather than having its own memory, as is the case with external graphics chipsets. This is a potential information security risk.
Power hungry parallel processors like GPUs are also ideal for cryptography applications.
There are many examples of webgl-based bitminers in botnets. While not an information security threat (conversely, bitminers are the basis for blockchain information security) these botnets steal a staggering amount of power from the vast networks of unwitting devices they run on.
This means Bitcoin has a growing but largely hidden carbon footprint, which is a potential environmental security threat.
If you ever notice that some websites slow down your display without increasing CPU load, it is very likely they are running webgl code on your GPU. Also a likely cause of mysterious mobile device power drains.
Mobile applications (like web browsers) can run GPU code perpetually in the background, and can require a device reboot to halt them.
