That's a lot of questions. We'll stick just to the core problem and mitigation technique:
There's the chance that my connection may be proxied or otherwise examined by a caching proxy. And there's a decent chance that I can fool the proxy if I can get the right data into my request and the desired data into the response. The basic idea was demonstrated in a paper called Talking to Yourself for Fun and Profit.
As an attacking client, what I want to do is push content up to the server that contains a seemingly valid request string:
... something ...
GET /jquery-latest.min.js HTTP/1.1
Host: code.jquery.com
...
... something that triggers the desired response from the server
The only requirement is that you figure out a way to get the server to send you a desired string, which typically isn't too hard. Then the server predictably parrots back the desired response:
HTTP/1.1 200 OK
content-type: text/javascript
... my trojaned javascript file
As far as the server and client are concerned, we're just passing strings back and forth. No real reason for concern, no harm done, no possible vector for exploit.
But a proxy watching the traffic would see what looks a lot like an HTTP request, and then what looks a lot like an HTTP response. And if the proxy is naive enough (and many are), it will update its cache accordingly, so that the next time someone requests http://code.jquery.com/jquery-latest.min.js through the proxy, they'll get my trojaned version instead of the real CDN-delivered code.
To guard against this, the traffic in one direction (in this case client-to-server) is mildly scrambled. It doesn't have to be cryptographically secure as you don't care if someone can decode it. You just need to make sure that the client can't predict what string he'll be transmitting over the network. And since he can't predict his network traffic, he can't use this mechanism to trigger a proxy server to update its cache.
The proxy will only see a cacheable event if BOTH the request AND response contain what looks like HTTP traffic. So only one direction need to be jumbled.
This does not apply to XHR, since XHR requests are "normal" HTTP requests, and are well-understood by proxies.