Why does XSS affect so many websites?

Question

According to an article I read 65% of all websites globally suffer from XSS. Why can't developers find and fix it?

Please help me understand. I'm not from a security or tech background.

Answer 1

XSS is a form of code injection, i.e. the attacker manages to inject its own malicious code (usually JavaScript) into trusted code (HTML, CSS, JavaScript) provided by the site. It is similar to SQLi in that it is caused by dynamically constructed code, i.e. SQL statements, HTML pages etc.

But while there are established techniques to solve SQLi (i.e. use parameter binding) it is even harder with XSS. To defend against XSS every user controlled input must be properly escaped, which is much harder than it sounds:

• Escaping rules depend on the context, i.e. HTML, HTML attribute, XHTML, URL, script, CSS contexts all have different escaping rules.
• These contexts can be nested, i.e. something like <a href=javascript:XXX> is a JavaScript context inside a URL context inside a HTML attribute.
• On top of that you have encoding rules for characters (UTF-8, UTF-7, latin1, ...) which again can be context specific.
• The context and encoding need to be known when constructing the output: while most template engines support proper HTML escaping they are often not aware of the surrounding context and thus cannot apply the context specific escaping rules.
• On top of that browsers can behave slightly different regarding escaping or encoding.
• Apart from that you'll often find in grown code with no proper separation of logic and presentation that HTML fragments gets stored in the database and it is not always clear to the developer if a specific value is HTML clean already or need to be escaped.

And while there are some simple ways to protect against XSS by design (especially Content-Security-Policy) these only work with the newest browsers, have to explicitly enabled by the developer and are often only effective after large (and expensive) changes to the code (like removing inline script).

But still, there is a good chance to do it right if you have developers with the proper knowledge and are able to start from scratch using modern toolkits which already take care of XSS. Unfortunately in many cases there is legacy code which just needs to be kept working on. And development is usually done by developers which are not aware of XSS at all or don't know all the pitfalls.

And as long as web development is done in environments very sensitive to cost and time the chances are low that this will change. The aim in this environment is to make the code working at all in a short time and with small costs. Companies usually compete by features and time to market and not by who as the most secure product. And XSS protection currently just means additional costs with no obvious benefit for many managers.

Answer 2

It seems easy, but is hard

First of all, the attack surface is huge. You need to deal with XSS if you want to display user input in HTML anywhere. This is something almost all sites do, unless they are built purely in static HTML.

Combine this with that fact that while XSS might seem easy to deal with, it is not. The OWASP XSS prevention cheat sheet is 4 000 words long. You need quite a large sheet to fit all that text in.

The complexity arise because XSS works differently in different contexts. You need to do one thing if you are inserting untrusted data in JavaScript, one thing if you insert into attribute values, another thing if you insert between tags, etc. OWASP lists six different contexts that all need different rules, plus a number of contexts where you should never insert data.

Meanwhile developers are always searching for panacheas and silver bullets. They want to deletage all the hard work to a single sanitize() function that they can always call on untrusted data, call it a day, and go on with the real programming. But while many such functions have been coded, none of them work.

Let me repeat: XSS might seem easy to deal with, but it is not. The main danger lies in the first part of that sentence, not the second. The percieved simplicity encourage developers to make their own home brewed solutions - their own personalized sanitize() functions, filled with generalisations, incomplete black lists, and faulty assumptions. You filtered out javascript: in URL's, but did you think about vbscript:? Or javaSCripT :? You encoded quotes, but did you remember to actualy enclose the attribute value in quotes? And what about character encodings?

The situation is quite similar to SQLi. Everybody kept looking for the one sanitation function to rule them all, be it named addslashes, mysql_escape_string or mysql_real_escape_string. It's cargo cult security, where people think it is enough to ritually call the right function to appease the Gods, instead of actually embracing the complexity of the problem.

So the danger is not that developers are blissfully unaware of XSS. It is that they think that they got the situation under control, because hey, I programmed a function for it.

But why is it hard?

The web suffers from the problem that structure (HTML), presentation (CSS) and dynamics (JavaScript) are all mixed into long strings of text we put in .html files. That means that are many boundaries you can cross to move from one context to another.

Again, this is the same as with SQLi. There you mix instructions and parameter values in one string. To stop SQLi, we figured out that we have to separate the two completely and never mix them - i.e. use parametrised queries.

The solution for XSS is similar:

1. Separate the scripts from the HTML.
2. Turn off inline scripts in the HTML.

We created an easy way to do the second part - Content Security Policy. All you need to do is to set a HTTP header. But the first part is hard. For an old web app a complete rewrite might be easier than to surgically disentangle the scripting from the HTML. So then it is easier to just rely on that magic sanitize() and hope for the best.

But the good news are that newer apps often are developed after stricter principles the use of CSP possible, and almost all browsers support it now. I'm not saying we are going to get the percentage of vulnerable sites down to 0%, but I am hopeful that it will fall a lot from 60% in the coming years.

---

TL;DR: This turned into a rather long rant. Not sure if there is much useful information in here - just read Steffen Ullrichs great answear if you want a clear analysis.

Answer 3

My set of opinion on security and XSS:

1. Rule of programming: You can't know everything. Sooner or later you are going to make a mistake.
2. Rule of programmer: A programmer works 12h a day: 3 is discussing with other programmers random things, 3 is thinking at other things, 3 is discussing on what it should code, 3 it's programming .... projects are made for 12h a day of almost non stop programing.
3. XSS is simple. And there are a lot of inputs waiting for XSS.
4. Security still comes last and most of the time is seen as a deficient.

Answer 4

As mentioned in the answer to a similar post of yours (SQL injection is 17 years old. Why is it still around?):

There is no general fix for SQLi because there is no fix for human stupidity

Developers sometimes get lazy or careless and that causes them to not check the application they are developing.

Another popular reason is that the developers aren't aware that there is a security issue. They never went through any security training and so they don't know what is a potential security threat and whats not. And for company's that don't understand that a penetration test is important this lack of security knowledge is a potential threat by itself.

Simple security issues such as SQLI and XSS will only be fixed when a company decided to spend time on code review and testing. Until then you have 65% of all websites globally suffering from the XSS vulnerability.

Answer 5

The root problem

The web was simply not designed to allow secure multi-authorship or rich interaction. Nobody talked about separating content from presentation until the late 1990s. By that point, like the QWERTY keyboard, we were basically stuck for no good reason with an existing system. Nobody wanted to "break the web", so mistakes were copied and ported to browsers beyond the "prime vector".

The aggravating factor

Before ~2005, virtually everything you saw on a given website was authored or approved by the people who control the site. As the "Web2.0" movement progressed, and people demanded interaction (at least add comments, come on!), the pressure to add these features to existing sites was great. Web folks did what they could to satisfy boss's demands to accomplish a task they knew nothing about: secure interaction. And for a while, probably too long, they got away with it. How many folks buy burgler alarms before they've even been broken into?

Modern Times

Using a CSP will greatly reduce the harm potential of dropping another ball. That doesn't mean you should forgo validation and sanitation by any means, because of Save As.. copies, cached w/o headers, etc, but it does mean that you can be a LOT safer with a couple hours of effort. Yes, the header is long-winded and in a confusing format at first, but invest the hour or two it takes to grok, and the few more it will take to test your site. CSP is really easy to test since you can do a silent warn-only run (look it up).

If you roll a CSP and take traditional measures, those old browsers will die when the hard drive stop spinning, and we'll all live safe and happily ever after...

Answer 6

Others have touched on the classic issues surrounding systems designed by humans for other humans: The reality is laziness and—sometimes—stupidity coupled with “Why would this happen to me?” arrogance.

Oh, how many hours of my life have been spent patching systems and—more importantly—fighting with management to get the time/resources allocated to patch systems so they can be “bullet proof.”

And I’m lucky in that respect: While I am frustrated fighting with management at an organization to get a system patched, many places hire developers to develop a system but then don’t consider basic (and boring)maintenance to be something worth investing in. Why pay a tech a monthly retainer to maintain a system when it is often cheaper to let the system stand until it collapses and then chase a fix in the middle of a breach?

An ounce of prevention being better than a pound of cure often takes a backseat to folks being pennywise, pound-foolish.

That said, the best thing anyone administering a system can do is to have a solid disaster (or non-disaster) recovery plan in place. Have the code version controlled, have the server deployment setup automated in a provisioning script and have backups of databases. That way if/when a system goes down, cleanup is quick instead of being a tedious disaster.

Answer 7

The accurate detection of dangerous security vulnerabilities such as SQLI and XSS during code implementation phase of SDLC is still limited to the type of programming language.

• Static analysis for such vulnerabilities is most widely done in PHP.
• Some dynamic methods employ fuzzy techniques during testing or deployment phase.

Unfortunately, both methods are still rather limited in their rate of accuracies. Furthermore, XSS and SQLI vulnerabilities target computer applications and can appear in so many disguised forms that are unknown to software developers. As such, I don't think it is a question of stupidity. I can assure some companies have already trained their developers for secure programming but cannot totally block these vulnerabilities. I would agree that there is much more that needs to be done to prevent such critical attacks.