4

I have installed debian 7 on a physical machine. This is the configuration of the machine:

  • 3 hard drives using RAID 5
  • Strip element size: 1M
  • Read policy: Adaptive read ahead
  • Write policy: Write Through
  • /boot 200 MB ext2
  • / 15 GB ext3
  • SWAP 10GB
  • LVM rest (~500GB)

I installed postgresql, created a big database (over 1GB). I have an SQL request that takes a lot of time to run (a SELECT statement, so it only reads data from the database). This request takes approximately 5.5 seconds to run. Then, I installed XEN, created a domU, with another debian distro. On this OS, I also installed postgresql, with the same database. The same SQL request takes only 2.5 seconds to run.

I checked the kernel on both dom0 and domU. uname-a returns "Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2+deb7u2 x86_64 GNU/Linux" on both systems. I checked the kernel parameters, which are approximately the same. For those that are relevant, I changed their values to make them match on both systems using sysctl. I saw no changes (the requests still take the same amount of time).

After this, I checked the file systems. I used ext3 on domU. Still no changes.

I installed hdparm, and ran hdparm -Tt on both systems, on all my partitions on both systems, and I get similar results.

Now, I am stuck, I don't know what is different, and what could be the cause of such a big difference.

Additional Info:

  • Debian runs on a Dell server PowerEdge 2950
  • postgresql: 9.1.9 (both dom0 and domU)
  • xen-linux-system: 3.2.0
  • xen-hypervisor: 4.1

Thanks

EDIT:

As Krzysztof Księżyk suggested, it might be due to some file caching system. I ran the dd command to test both the read and write speed.

Here is domU:

root@test1:~# dd if=/dev/zero of=/root/dd count=5MB bs=1MB
^C2020+0 records in
2020+0 records out
2020000000 bytes (2.0 GB) copied, 18.8289 s, 107 MB/s

root@test1:~# dd if=/root/dd of=/dev/null count=5MB bs=1MB
2020+0 records in
2020+0 records out
2020000000 bytes (2.0 GB) copied, 15.0549 s, 134 MB/s

And here is dom0:

root@debian:~# dd if=/dev/zero of=/root/dd count=5MB bs=1MB
^C1693+0 records in
1693+0 records out
1693000000 bytes (1.7 GB) copied, 8.87281 s, 191 MB/s

root@debian:~# dd if=/root/dd of=/dev/null count=5MB bs=1MB
1693+0 records in
1693+0 records out
1693000000 bytes (1.7 GB) copied, 0.501509 s, 3.4 GB/s

What can be the cause of this caching system? And how can we "fix" it? Can we apply it to dom0?

EDIT 2:

I switched my virtual disk type. To do so I followed this article.

I did a dd if=/dev/vg0/test1-disk of=/mnt/test1-disk.img bs=16M Then in /etc/xen/test1.cfg, I changed the disk parameter to use file: instead of phy: it should have removed the file caching, but I still get the same numbers (domU being much faster for Postgres)

longneck
  • 22,793
  • 4
  • 50
  • 84
Paco
  • 91
  • 6
  • 1
    If you insist on using dd for benchmarking, try using the dsync output flag, as mentioned [here](http://romanrm.ru/en/dd-benchmark) to bypass the filesystem cache. – dlu Jul 03 '13 at 17:54
  • What are the Dom0/DomU CPU and RAM-layouts? Do you PIN or hide some CPUs? Does the Dom0 have less RAM than the DomU? Are you running the DomU in PV or HVM mode? – Nils Jul 08 '13 at 08:56

2 Answers2

2

It's probably because of file caching. Even if baremetal system and system in domU have the same configurations regarding caching etc. some file operations in domU will execute faster than on baremetal because hypervisor transparently caches IO requests from domU system. You can test it using dd command.

Krzysztof Księżyk
  • 502
  • 4
  • 9
  • You're right, dom0 appears to be much faster on writes and reads. I'm going to edit my post with the extra information. Thanks – Paco Jun 11 '13 at 12:26
  • 1
    also note that this level of cache speedup is mostly useless for anything beyond benchmarks. – Javier Jun 11 '13 at 13:38
0

I don't know but this is very strange behavior. I'm curious to understand Why the dd test dom0 is faster than domU and Postgres happens to the contrary.

About Postgres I think the same as Krzysztof Księżyk. This happens when there is cache overlapping in the filesystem or something with RAM memory. In your tests, are you disabling linux cache?

What's the memory allocated to dom0 and domU?

Fernando Ike
  • 54
  • 4