I know that containers use the host's kernel and from what I understand , that's the reason we do not need an OS. My questions are (and I couldn't find any good explanation online) :
1) If that's the case , hoe do we get a shell prompt and how do we have stuff like systemctl , services , etc on the container
2) How do we install a CentOS container for example on an Ubuntu host ? in this case does the container have an OS installed in the container image?
It largely depends on what your view of an O.S. is.
Containers fundamentally are a method to isolate both a running process from all other processes on a machine, and the binary and all its dependencies from the host machine filesystem. This then makes them trivial to move between and run on any number of machines. They are in no way virtualisation, therefore as you mention in your question, any process running in a container is running on the host machines kernel. From some points of view an operating system, fundamentally is the kernel.
Balanced against that, is that most software is not built to be monolithic, but is instead linked at compile time, against any number of shared libraries. So when you install an operating system distribution, as well as a kernel, you get a whole series of libraries and utilities, and any programs you install from the O.S. repos will be built against the libraries that are shipped with that OS distribution.
Now when building a container, all that is required is the binary that you want to run, plus anything that that binary depends on. There is no requirement to package a complete O.S. distro filesystem. If your binary is not linked against any other library, you can almost get away with the single binary and nothing else. Check out this project https://github.com/davidcarboni-archive/ddd written by a chap who works with the same client I currently do, which demonstrates how little is required to build a functional container.
So in an ideal world, given best practice largely is to build a container that runs a single process, each container would consist of nothing more than the relevant binary, plus any libraries, config files and working directories it needs. It is however, quite an involved and time consuming process to get that right in a lot of cases. Therefore it has now become a very common pattern, to simply package up the entire filesystem from a donor OS inside a container, as that way you know that any dependencies the process to be run has, will be present. It also adds the convenience of meaning that package management tools will be present, which makes building the container easier. It also means that utilities such as shells are also present, which makes life easier to debug a container (tho in some peoples minds, other than perhaps when you are developing a new container image, if you need to access a shell inside the container, you are doing it wrong).
Whilst I understand why this pattern has become so prevalent, I do think it has disadvantages. First demonstrated by your question - it has made containers look a lot like virtualistation and thus sown a lot of confusion. More over, as someone who has just finished applying CIS server hardening to an estate of machines, packaging everything including the kitchen sink in every container, doesn't feel like great security practice, and I suspect at some point, that may come back to bite us.
To expand on your two specific questions:
1) I have addressed the subjects of shells already. As for things like systemd, they essentially have no place in a container. The Dockefile defines the process to run when the container is started. If you are trying to run a service manager and multiple services inside a container, you are likely doing it wrong.
2) This is back to the question of what is an O.S.? The only sense that you install a given distro in a container, is that you get the filesystem and therefore the distros copies of binaries and shared libraries. It is not the same as virtualisation, where you have a copy of say, Centos running in a virtual machine on a host running say, Debian.
Yes, they do. Every container is based on an OS image, e.g. Alpine, CentOS or Ubuntu.
They just share the host kernel, but run every user-space process in a separate name space specific for that container.
Look into a dockerfile example (adapted by me) to understand this:
FROM ubuntu
MAINTAINER Kimbro Staken version: 0.1
RUN apt-get update && apt-get install -y apache2 && apt-get clean && rm -rf /var/lib/apt/lists/*
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
EXPOSE 80
CMD ["/usr/sbin/apache2", "-D", "FOREGROUND"]
This tells Docker to create a container from an Ubuntu base image (latest version if no version is specified), install and run Apache in it and expose port 80 to the host OS.
What is an OS anyway?
The Linux kernel by itself meets most of the key requirements to be an operating system. It time slices the CPU, it manages console IO, disk IO and networking, it can provide a hardware-agnosic framebuffer for graphics output and so-on. You can write a program that runs on the Linux kernel with no libraries loaded and takes advantage of these features.
However when speaking about modern general purpose operating systems most people take a much broader view of what the OS is. In addition to the kernel they include a bunch of other stuff. The shell, the init system, the X server, the common libraries, the system for loading driver modules so you don't have to build all your drivers into the kernel, the tools for bringing network interfaces up, the tools for mounting additional file-systems, the package manager and so-on. Collectively we call these parts that are used in addition to the kernel to build a modern OS the "userland".
Most of the identity of a Linux distro comes from the userland. If you have a Centos userland running on a Ubuntu kernel it will feel much more like Centos than Ubuntu.
Containers are a feature that allows a single kernel to pretend to be multiple seperate kernels. Each with it's own PIDs, it's own filesystem heiracy, it's own network interfaces and it's own user accounts.
As with a real system running Linux kernel, you can write a program that runs in a container with no support. Indeed it's easier in a container than on a real system because you don't have to worry about drivers and your container-manager may be able to pre-initialise the container so that the correct filesystems are already mounted and network interfaces are already up.
But most of the time what people want out of containers is "virtualisation but cheaper". They want to split up the logical functions of a server so each one can be backed-up, migrated, updated etc seperately, but they want to keep using the software stack they are using which takes advantage of some or all of the aforementioned non-kernel parts of a modern OS and they don't want to pay the cost of VMs.
So containers often end up containing the entire userland of a regular Linux distro. Complete with an init system, a shell, a ssh server. Does that count as an OS? well that is ultimately a question of how you define OS!
They do, but these OS are emulations of OS and not real OS.
To understand the difference, we have to recall the architecture of an OS, e.g. any flavor of Unix (whether it is AIX, Linux, SVR4, Solaris, SunOS, etc.), or Windows. The core of every OS is the so called real-time executive, which manages all available resources such as memory, CPU, file system, network resources, stream drivers etc. and allocates them optimally to running programs (processes, tasks). The real-time executive is also called kernel. For example, when you are creating new flavors of Linux distributions, you are essentially performing "kernel building" i.e. you are customizing the kernel for that particular distribution).
Therefore, OS has kernel space and user space and interface between them. Application programs run in user space as processes or tasks and can also communicate with the kernel space directly via system calls (e.g. sysctl() in Unix). The interface between the user space and kernel space is implemented differently in containers than in VMs. In VMs, this interface is implemented the same way as in any other platform with that OS. In containers, the implementation of this interface leverages the underlying OS kernel space i.e. the underlying OS kernel can be different than the OS that is emulated in the user space. Container daemon runs in the user space of the host OS and translates all system calls from containers' OS to system calls of the host OS and vice versa. In short, in containers, the kernel space of their OS is emulated while in VMs it is not.
yes, every docker image is built on top of base image. the first statement in the dockerfile is defines a base image to be used to build the new docker image. say, you want to build on centOS OS. then use below
It instructs Docker to create an image usign centos as base image FROM centos:latest
1) If that's the case , hoe do we get a shell prompt and how do we have stuff like systemctl , services , etc on the container
You can use a Full OS image container, some examples:
all previously image are Official image that mean is maintained by the same group maintain the OS. By running those image you are run an entire OS and they run on ANY Linux host.
To access the prompt you have two ways:
For the second method you have to expose the port 22 to the external word.
2) How do we install a CEntOS container for example on an Ubuntu host ? in this case does the container have an OS installed in the container image?
For run a CEntOS container in a docker environment is really easy, just install docker and run:
docker run --rm -p 2020:22 -v vol_data:/data centos:7
Previous command will:
--rm-p 2020:22 (this to avoid conflict on the already in use host port.vol_data and mount it inside the container at /datacentos:7To run systemd inside the container the docker hub page of the image will describe you how to do it.
Another way to reach your goal you could think about to use machinectl (should part of systemd, not sure on this), here some reference: https://www.freedesktop.org/software/systemd/man/machinectl.html
Off course you can use also chroot environment, but you have to do it all by and and is not really friendly to implement.
To clarify one point regarding the O.S., an operating system is an environment ready to be use (GUI or not) so the kernel is one of the many part of an operating system, in short, yes inside a container image you will have an OS which just not access directly the physical hardware, could say without an hardware at all. The container image studied to run only a service it is a minimal installation of components necessary to run that particular service (could anyway be expanded).
To understood better what is an OS and all its part (Linux OS off course) I suggesting you to take a look LFS Project.
