Why do we still not have high performance multi-monitor remote desktop applications?

Question

It is 2017. We can stream 4k livestreams using a fraction of the available bandwidth and compute resources.

Why do we still not have remote desktop applications that provide decent performance? What is the difference that makes it so much harder to stream a mostly static picture while transmitting mouse and keyboard actions that should not be more than a few bytes?

Answer 1

First off, we do have remote desktop applications that provide decent performance - but there is no single universal solution to do it well.

In the typical / catch-all case, a computer needs to transmit exactly whats on the screen to another location. The only thing it may be able to do to speed things up is compress the output - this is typically the case with VNC for example. Depending on the speed of the connection and complexity of the screen this might work well, or it may be slow. If lossy compression is used, then stuff may not be readable.

On the other extreme is something like chromecast - which is providing instructions to the remote computer to do stuff, so it can take maximum use of compression (ie it handles the original compressed stream from the source, rather then needing to decompress it and ship the decompressed or decompressed and recompressed data). Of-course, this requires apps which are compatible with the technology - and most desktop apps arn't.

In-between there are solutions like Spice, X forwarding rdp, which try use API's to reduce the amount of data transmitted by reproducing what should be displayed on the original screen.

Another issue is latency - if you are in New Zealand (for example), and sharing a connection in Europe, you are looking at > 200ms latencies thanks to the speed of light in a vacuum - nothing you can do to the connection can make it lower latency - and this makes it feel slow.

Similarly, when dealing with moving video, save for Chromecast type solutions, its often not possible to know what is compressed (and how it was compressed) - and tight, lossy compression is time consuming making this difficult. (I would be surprised if 4k videos can be compressed in realtime without very heavy load on the system - ie 4k videos may stream and play in realtime, but are not generally encoded in realtime while systems are active doing other real tasks)