0

I am trying to load two shared libraries while running Qemu using libvirt. To preload the shared libraries I have added an environment variable in the $domain.xml file with the following command:

virt-xml generic --edit --confirm --qemu-commandline 'env=LD_PRELOAD=/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so:/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so'

However, while running the domain, I get the following message of failure to preload the shared files.

2019-07-17 17:35:22.939+0000: starting up libvirt version: 1.3.1, package: 1ubuntu10.27 (Marc Deslauriers <marc.deslauriers@ubuntu.com> Tue, 02 Jul 2019 09:22:37 -0400), qemu version: 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.39), hostname: probir-XPS-13-9343
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=spice LD_PRELOAD=/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so:/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so /usr/bin/kvm-spice -name generic -S -machine pc-i440fx-xenial,accel=kvm,usb=off -cpu Broadwell-noTSX-IBRS -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 8f755c9c-adf8-4baf-8d6a-947ebdf61ff8 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-generic/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/var/lib/libvirt/images/generic.img,format=qcow2,if=none,id=drive-ide0-0-0 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-0-1,readonly=on -device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev tap,fd=26,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:82:62:ba,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on
Domain id=9 is tainted: custom-argv
ERROR: ld.so: object '/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
char device redirected to /dev/pts/21 (label charserial0)

I further went ahead to verify the shared library with ldd and ls -l. Following results came out:

probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ls -l /home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so
-rwxrwxr-x 1 probir probir 834240 Jun 13 17:53 /home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so
probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ldd /home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so
    linux-vdso.so.1 =>  (0x00007ffdd97d5000)
    libxed.so => /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/xed/kits/xed-install-base/lib/libxed.so (0x00007f706442e000)
    libmonitor.so.0 => /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so.0 (0x00007f706420a000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7063fed000)
    libbfd-2.26.1-system.so => /usr/lib/x86_64-linux-gnu/libbfd-2.26.1-system.so (0x00007f7063ca8000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f7063926000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f7063710000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7063346000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f7064f48000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f7063142000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f7062f28000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7062c1f000)

and 

probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ls -l /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so 
lrwxrwxrwx 1 probir probir 19 Mar 28 13:45 /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so -> libmonitor.so.0.0.0
probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ldd /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so
    linux-vdso.so.1 =>  (0x00007ffc5d12b000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff8ae23e000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff8ade74000)
    /lib64/ld-linux-x86-64.so.2 (0x00007ff8ae666000)

I cannot figure out what is causing the failure of preload. Previously, I manually preloaded the same shared libraries for other application.

Proy
  • 103
  • 4
  • Check the permissions. – Michael Hampton Jul 17 '19 at 18:45
  • Which one? I have the "ls -l" results in the post. Which one look suspicious? – Proy Jul 17 '19 at 18:47
  • You also have to check the permissions of every parent directory, because these must also allow traversal (o+x). Try `namei -l ` to check these quickly. – Michael Hampton Jul 17 '19 at 18:49
  • I have re-installed the shared library in "/usr/local/lib/libmonitor.so". Running the virt-manager in root mode (sudo -s). Still having this issue. – Proy Jul 17 '19 at 19:13
  • You still need to check the permissions as described. – Michael Hampton Jul 17 '19 at 19:16
  • parent directory tmp (drwxrwxrwt) and libmonitor.so(lrwxrwxrwx) permission look like this. Still getting error: ERROR: ld.so: object '/tmp/libmonitor.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. – Proy Jul 17 '19 at 19:21
  • drwxr-xr-x root root / drwxr-xr-x root root home drwxr-xr-x probir probir probir drwxr-xr-x probir probir Downloads drwxrwxr-x probir probir kpv_temp drwxrwxr-x probir probir temp drwxrwxr-x probir probir kvmTest drwxrwxr-x probir probir dependencies drwxrwxr-x probir probir libmonitor drwxrwxr-x probir probir installDir drwxrwxr-x probir probir lib lrwxrwxrwx probir probir libmonitor.so -> libmonitor.so.0.0.0 -rwxr-xr-x probir probir libmonitor.so.0.0.0 – Proy Jul 17 '19 at 19:52
  • I can also access the directories is libvirt-qemu user: ``` libvirt-qemu@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib$ ls -l libmonitor.so lrwxrwxrwx 1 probir probir 19 Jul 17 13:25 libmonitor.so -> libmonitor.so.0.0.0 ``` – Proy Jul 17 '19 at 20:06

1 Answers1

0

May be apparmor blocking ld_preload libraries. Check: is libvirt controlled by apparmor:

apparmor_status | grep libvirt

If so, you can stop apparmor:

systemctl stop apparmor
systemctl restart libvirtd

Or create permission for your libs in apparmor configuration file:

/etc/apparmor.d/abstractions/libvirt-qemu
AlexBG
  • 101