68

I'm trying to list services on my CentOS image running in Docker using

systemctl list-units  

but I get this error message:

Failed to get D-Bus connection: Operation not permitted

Any suggestions what the problem might be?

ivanleoncz
  • 1,433
  • 4
  • 18
  • 32
Snowcrash
  • 1,087
  • 2
  • 16
  • 19

4 Answers4

49

My guess is that you're running a non-privileged container. systemd requires CAP_SYS_ADMIN capability but Docker drops that capability in the non privileged containers, in order to add more security.

systemd also requires RO access to the cgroup file system within a container. You can add it with –v /sys/fs/cgroup:/sys/fs/cgroup:ro

So, here a few steps on how to run CentOS with systemd inside a Docker container:

  1. Pull centos image
  2. Set up a docker file like the one below:
FROM centos
MAINTAINER "Yourname" <youremail@address.com>
ENV container docker
RUN yum -y update; yum clean all
RUN yum -y install systemd; yum clean all; \
(cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]
  1. Build it - docker build -t centos7-systemd - < mydockerfile

  2. Run a container with docker run --rm --privileged -ti -e container=docker -v /sys/fs/cgroup:/sys/fs/cgroup centos7-systemd /usr/sbin/init

  3. You should have systemd in your container

burtsevyg
  • 158
  • 2
  • 7
13dimitar
  • 2,360
  • 1
  • 12
  • 15
  • Pretty neat! However, at least I'm getting more info back now. Here's what I get logged: `[ INFO ] Update UTMP about System Boot/Shutdown is not active. [DEPEND] Dependency failed for Update UTMP about System Runlevel Changes. Job systemd-update-utmp-runlevel.service/start failed with result 'dependency'. [ OK ] Started Journal Service. [ OK ] Reached target System Initialization. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket.` – Snowcrash Jan 10 '17 at 20:43
  • 2
    In case I wasn't clear! I'm still getting the error `Failed to get D-Bus connection: Operation not permitted` – Snowcrash Jan 12 '17 at 12:17
  • You've build your own image from the Dockerfile copied in my answer, you run a container from that image, and you still get an error? – 13dimitar Jan 12 '17 at 12:21
  • 8
    Bingo! I was running the container with `/bin/bash` in order to get a shell. However, this gave me the previously mentioned error. When I ran it with `/usr/sbin/init` as suggested then attached with a shell all went well. Clearly I'm missing a nuance about `/usr/sbin/init`. This answer deserves some substantial upvoting. – Snowcrash Jan 12 '17 at 22:29
  • 1
    I have been on this for 2 days, and I still can't figure out what `/sys/fs/cgroup:/sys/fs/cgroup` is or where it is coming from... I know how to mount guest folder to hist like: `/src/:/var/www` but where is your file coming from? It is causing me lots of errors because I pasted the code, I'm thinking I should create those somewhere – samayo Sep 12 '18 at 01:57
  • 1
    docker "invalid reference format" , when I just copy-pasted it , when I retyped it though it worked – FantomX1 Oct 19 '20 at 12:59
  • If I'm using macbook, how to solve this problem. There's no `/sys/fs/cgroup` in macos. – Tony Chou Feb 03 '21 at 15:02
  • using /usr/sbin/init instead of sh fixed it for me. – franchyze923 Jun 17 '21 at 13:09
  • `docker build --rm` there is no --rm options. – burtsevyg Nov 11 '21 at 22:49
  • It should be in `docker run` – burtsevyg Nov 11 '21 at 22:50
  • Failed to mount cgroup at /sys/fs/cgroup/systemd: Operation not permitted systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN) Detected virtualization docker. Detected architecture x86-64. Welcome to CentOS Linux 7 (Core)! Set hostname to <86d781941db3>. Cannot determine cgroup we are running in: No such file or directory Failed to allocate manager object: No such file or directory [!!!] Failed to allocate manager object, freezing. And then get stuck here – Pereira May 17 '22 at 16:01
14

This isn't a direct answer to your question, but it may actually be more important, and I came across this realization as I was reading the other answers here.

I've had some experience migrating some complicated systems to Docker, and one of the significant realizations I've had is that you should ideally have one Docker container per application/service or "per daemon".

One very significant reason for this is that Docker will not cleanly shutdown services that you start with systemctl and in fact you may end up with the same sort of database corruptions that come from an unexpected power outage.

To dive into this a little deeper: when Docker issues a "stop" command to a container, it sends the SIGTERM signal only the one single process that was started with the CMD/ENTRYPOINT, not to all the services and daemons. So that one service has the warning to shutdown cleanly and all the others get unceremoniously terminated.

If you absolutely have to package two services in the same container (i.e. your application and a PostgreSQL database or something like that) then you need to have your CMD/ENTRYPOINT be a script that catches SIGTERM and then rebroadcasts it to those known services. It can be done, but if you have the opportunity, rethink your solution and try to break it into multiple containers.

An addendum

There's an interesting note/page on the Docker site about using supervisord if you absolutely need to have multiple services running in the same container.

9

I've managed to fix this issue in a CentOS:7 Docker container. I've followed mainly the Guide on CentOS Docker image project.

FROM centos:7

ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;

# Install anything. The service you want to start must be a SystemD service.

CMD ["/usr/sbin/init"]

Now, build the image, and run it using at least the following arguments to docker run command: -v /run -v /sys/fs/cgroup:/sys/fs/cgroup:ro

Then main point is that /usr/sbin/init must be the first process inside the Docker container.

So if you want to use a custom script that executes some commands before running /usr/sbin/init, launch it at the end of your script using exec /usr/sbin/init (in a bash script).

Here is an example:

ADD cmd.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/cmd.sh

CMD ["/usr/local/bin/cmd.sh"]

And here is the content of cmd.sh:

#!/bin/bash

# Do some stuffs

exec /usr/sbin/init # To correctly start D-Bus thanks to https://forums.docker.com/t/any-simple-and-safe-way-to-start-services-on-centos7-systemd/5695/8

You could have System is booting up. See pam_nologin(8) if your using the PAM system, in that case, delete /usr/lib/tmpfiles.d/systemd-nologin.conf in your Dockerfile because it creates the file /var/run/nologin which generates this specific error.

Anthony O.
  • 674
  • 1
  • 5
  • 13
  • `systemd-nologin.conf` / `nologin` for the win because CentOS/RHEL 7 claims `UsePAM no` is unsupported and will complain in logs as such. Not sure if RH openssh portable patched / broke it somehow or they're trying to lower their support surface from novice customers. –  Apr 18 '17 at 14:48
  • Excellent, this worked for me. Key was adding -v /run. – MiloDC Dec 04 '20 at 00:56
2

I didn't want to have to launch systemd as init/PID 1. After doing the cleanup steps mentioned by others, I launch systemd from within a startup script as /usr/lib/systemd/systemd --system &.

This allowed systemd to start and launch the registered services, but systemctl was failing with the D-Bus error.

For me, the missing link was the absence of the /run/systemd/system directory, discovered this by straceing systemctl.

Creating this directory manually before running systemctl allows systemctl to work for me.

inquisitive
  • 103
  • 4
Chaim Geretz
  • 131
  • 3