Virtualized guest vs host CPU usage

Question

I am using qemu with kvm as my hypervisor. I am running a gentoo guest on an ubuntu host. The guest is being given a single processor.

When I run a compute intensive job (say empty-loop) I get almost 100% CPU usage on guest as well as host. However, when I play a high resolution video ( 1080p) I see that the guest is showing upto 100% CPU usage whereas the host is showing only about 30-40% average CPU usage. I wonder what could be causing this.

If the process was Disk bound, then I believe a second run of the video would have improved the host-to-guest CPU usage ratio, as the video would have been fetched to the main memory ( the video is approx 1 minute, 40 MB and the guest ram is 4 gigs), but that does not happen. Then what could cause this difference in CPU usage ?

I dont fully understand how vlc handles H264 videos. It reports that it is dropping frames possibly due to slow computer. My only guess is that perhaps a graphics card is involved that the qemu waits for and the guest is not aware of this waiting.

My host machine has 12 cores with 24 gigs of RAM.

EDIT :

The problem had nothing to do with running a virtual machine. It was because I was accessing my test machine through SSH. X11 forwarding was causing a major performance slowdown to my qemu process. Perhaps it is similar to how a normal program slows down if we have too many printf statements. The network was becoming a bottleneck in my case.