Docker for Linux part II – intermediate commands

In the previous part of our series, we learned how to install the community version of Docker Engine. We also learned how to download, start, and delete a container and its image. Today we will be using an EuroLinux 7 based image along with the FBI repository. For more on the publicly available repository, which we make freely available without any restrictions, see the article on FBI in EuroLinux.

Docker versioning basics – what are tags and layers?

The basic building blocks of modern OCI (Open Container Initiative) compliant containers are metadata and layers. Technically, a layer is a tar archive containing changes between successive versions of a container. For a simple image build, each directive is a new layer. However, newer tools allow multiple build directives to be collected into a single layer. However, the topic of alternative build tools will not be discussed until 2 articles later. A new layer is also built when saving state (commit – also literally translated as creating a snapshot. However, it is often used in case of so-called snapshots, which may cause even more confusion in the concepts) of the container by the user.

In addition, containers use a copy-on-write (CoW) mechanism. This allows a single image to be used by multiple containers. CoW mechanism is widely used in IT, because it allows to save memory (both RAM and disk), but also is often many times faster.

The storage driver is responsible for writing and storing individual layers. Older Docker implementations used AUFS. In newer implementations it is OverlayFS version 2. The main idea of these solutions is to keep the changes in files, relative to the original image, while maintaining the best possible performance. Since the discussion and technical details are far beyond the scope of this article, I will refer you to the Docker documentation:

Docker Doc: Use the OverlayFS storage driver

Docker Doc: Select a storage driver.

Now knowing that layers are overlays of container changes and knowing the basic issues associated with them, we can, to paraphrase a friendly green Ogre, conclude:

„Layers. Onions have layers. Containers have layers… You get it? We both have layers.”

After a brief dose of theory, let’s now create a EuroLinux image that will have a web server (httpd) installed. We will create another layer for it, which we will save and tag.

First, let’s download the image from the Docker Hub platform.

[Alex@Normandy Docker]$  docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
[Alex@Normandy Docker]$ docker pull eurolinux/eurolinux-7
Using default tag: latest
latest: Pulling from eurolinux/eurolinux-7
e3693a234614: Pull complete 
Digest: sha256:5447be149a8c283a67fb49977c4dacc84d6a23e01bae03057fb3d077ffbf798b
Status: Downloaded newer image for eurolinux/eurolinux-7:latest
docker.io/eurolinux/eurolinux-7:latest

Next, let’s start the container and install the Apache (httpd) server in it. To shorten the stdout output of the yum program, it has been redirected to /dev/null. Let’s also note the naming of the container as my_httpd.

[Alex@Normandy Docker]$ docker run -it --name my_httpd eurolinux/eurolinux-7
eurolinux/eurolinux-7         eurolinux/eurolinux-7:latest  
[Alex@Normandy Docker]$ docker run -it --name my_httpd eurolinux/eurolinux-7
[root@1c81d8d92282 /]# yum install -y httpd >/dev/null
warning: /var/cache/yum/x86_64/7/fbi/packages/apr-util-1.5.2-6.el7.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 18cd4a9e: NOKEY
Importing GPG key 0x18CD4A9E:
 Userid     : "EuroLinux (7) <****@euro-linux.com>"
 Fingerprint: 2332 b393 7b50 49e5 6415 c200 75c3 33f4 18cd 4a9e
 Package    : el-release-7.7-1.el7.x86_64 (@el-base)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-eurolinux7

All we need to do now is exit the container and perform a commit on it. Note that after exiting, the container automatically stops running. This is because the entry point (the default command) to the container is a Bash shell. The entire procedure for saving a new container image including tagging is shown in the following example:

[Alex@Normandy Docker]$ docker ps # Won't show us our non-running container
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[Alex@Normandy Docker]$ docker ps -a
CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS                          PORTS               NAMES
1c81d8d92282        eurolinux/eurolinux-7   "/bin/bash"         10 minutes ago      Exited (0) About a minute ago                       my_httpd
[Alex@Normandy Docker]$ docker commit my_httpd # Saving our container's state
sha256:b03b612c306d42edb56dec5ea7cbe890d53c45eb3c071aaafeb2e1eada7fcad2
[Alex@Normandy Docker]$ docker images # As you can see, the image has neither a tag nor a repository
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
                                b03b612c306d        9 seconds ago       208MB
eurolinux/eurolinux-7   latest              3eed865a31cf        4 weeks ago         168MB
[Alex@Normandy Docker]$ docker tag b03b612c306d my_httpd # By default the tag 'latest' will be created
[Alex@Normandy Docker]$ docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
my_httpd                latest              b03b612c306d        21 seconds ago      208MB
eurolinux/eurolinux-7   latest              3eed865a31cf        4 weeks ago         168MB
[Alex@Normandy Docker]$ docker tag my_httpd:latest my_httpd:v1 # Addint the tag v1
[Alex@Normandy Docker]$ docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
my_httpd                latest              b03b612c306d        11 minutes ago      208MB
my_httpd                v1                  b03b612c306d        11 minutes ago      208MB
eurolinux/eurolinux-7   latest              3eed865a31cf        4 weeks ago         168MB

The next example shows that removing the tag ‘latest’ does not mean removing the tag v1:

[Alex@Normandy Docker]$ docker rmi my_httpd:latest
Untagged: my_httpd:latest
[Alex@Normandy Docker]$ docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
my_httpd                v1                  b03b612c306d        12 minutes ago      208MB
eurolinux/eurolinux-7   latest              3eed865a31cf        4 weeks ago         168MB

Although the image built this way is fully functional, it has one serious drawback – it does not build “automagically”. Furthermore, the bash shell is still the default entry point (the program running by default). We will tell you more about efficient and functional building of container images in the next part of our tutorial.

Exposing a service’s port

Having already prepared container with the web server, we can let ourselves test it. For this purpose, I suggest checking its internal IP address first, and then the response of the server.

Run the container in the background (daemon switch -d) with the name httpdv1 and run the /sbin/httpd command with the -DFOREGROUND option.

[Alex@Normandy Docker]$ docker run --name httpdv1 -d my_httpd:v1 '/sbin/httpd' '-DFOREGROUND'
57ea7edcfe33d97120268cf732803be098f8c892dea86dc125f94dd545da7336
[Alex@Normandy Docker]$  docker

To find an IP, sometimes we can execute one of the publicly available commands, such as ip, ifconfig, hostname -I. Unfortunately, these will not always be available. This is the case with EuroLinux containers, among others, which only have the necessary software. In this case, the docker inspect command, which is used to display low-level data about a given object (it doesn’t have to be a container), comes in handy. Below is a sample attempt to find the IP address of a container:

[Alex@Normandy docker-II]$ docker exec -i -t httpdv1 /bin/bash
[root@53168e4ee170 /]# ip a
bash: ip: command not found
[root@53168e4ee170 /]# ifconfig
bash: ifconfig: command not found
[root@53168e4ee170 /]# hostname -I
bash: hostname: command not found
[root@53168e4ee170 /]# exit
[Alex@Normandy docker-II]$ docker inspect httpdv1 | grep -i ip | grep -i addr
            "LinkLocalIPv6Address": "",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "GlobalIPv6Address": "",
            "IPAddress": "172.17.0.2",
                    "IPAddress": "172.17.0.2",
                    "GlobalIPv6Address": "",
[Alex@Normandy docker-II]$

After finding the IP address, we can move on to the next step, which is to test if the web server packed in the container is indeed listening on the right port. To do this, you can use a browser or a simple cURL.

[Alex@Normandy dockerII]$ curl  s 172.17.0.2:80 | head 5
<!DOCTYPE html PUBLIC "//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml1
1/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
    <head>
        <title>Test Page for the Apache HTTP Server on EuroLinux</title>

With a container running, there is no simple way to assign a container port to a host port. This can of course be done by adding proper packet forwarding using the Netfilter framework (the high-level interface to Netfilter is both firewalld and iptables), but such a solution is very difficult to maintain. Because of this fact, it is best to stop and delete our existing container.

[Alex@Normandy docker-II]$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
53168e4ee170        my_httpd:v1         "/sbin/httpd -DFOREG…"   33 minutes ago      Up 33 minutes                           httpdv1
[Alex@Normandy docker-II]$ docker stop httpdv1 ; docker rm httpdv1
httpdv1
httpdv1
[Alex@Normandy docker-II]$

The docker run command has a --publish switch (shortened to -p) responsible for assigning the container port to the host port. An example starts a container named httpdv2, assigning port 80 of the container to 8080 of the host, from the my_httpd image and starts the /sbin/httpd command with the -DFOREGROUND argument. A page download test occurs next.

[Alex@Normandy docker-II]$ docker run --name httpdv2 -d -p 8080:80 my_httpd:v1 '/sbin/httpd' '-DFOREGROUND'
3baf52e6f35b8d1e43348f001f04dc1214f277c5d895392cad75afaca5b7a7c2
[Alex@Normandy docker-II]$ curl -s localhost:8080 | head -1

Mounting a directory

One of the most important features of containers is their temporality. So the easiest services to containerize are those that are stateless. As it is not difficult to guess, this is only a small part. Most services need to keep their state somewhere, for example, with the help of a database. In this case, the container generally mounts the appropriate data volume (usually a directory from the container) to the appropriate directory on the host (this directory can be, for example, a network file system).

Docker has two options for mounting disk resources. The first is -v (--volume) and the second is --mount. These used to be different because of Docker’s mode of operation (mount was used for docker swarm, or container orchestration). Today, they can be used interchangeably in most cases. The important difference is that -v will automatically create a directory to which the container resources should be mounted. If no suitable directory is available, the docker run command with the mount option will exit with an error and inform the user about the lack of a suitable folder.

Below is the creation of the third version of our container, which will pull the default page (index.html) from the local directory:

mkdir pages
echo 'Hello World' > pages/index.html
[Alex@Normandy docker-II]$ docker run --name httpdv3 -d -p 8080:80 -v $(pwd)/pages:/var/www/html my_httpd:v1 '/sbin/httpd' '-DFOREGROUND'
16478c3cd9e4094804ff2f4f2dda5709c80ac603b4fb302226ae4a8a5da7eb93
[Alex@Normandy docker-II]$ curl localhost:8080
Hello World!

Note that the directory you want to mount must be an absolute path. Otherwise, Docker will return an error.

[Alex@Normandy docker-II]$ docker run --name httpdv3 -d -p 8080:80 -v ./pages:/var/www/html my_httpd:v1 '/sbin/httpd' '-DFOREGROUND'
docker: Error response from daemon: create ./pages: "./pages" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.
[Alex@Normandy docker-II]$

It is worth checking if the connection to the container works both ways. In order to perform a simple test, we will use the docker exec command with touch.

[Alex@Normandy docker-II]$ ll pages/
total 4
-rw-r--r--. 1 Alex Alex 13 Nov  2 18:33 index.html
[Alex@Normandy docker-II]$ docker exec httpdv3 'touch' '/var//www/html/test'
[Alex@Normandy docker-II]$ ll pages/
total 4
-rw-r--r--. 1 Alex Alex 13 Nov  2 18:33 index.html
-rw-r--r--. 1 root root  0 Nov  2 18:42 test

Summary and Announcement

In this article:

we created a new container image, saving its state
tagged the new image
run the image with the selected service
we exposed the port from the container to the local address
we used a volume to mount the web server data, which, unlike the container and its data, is persistent.

Despite our efforts, however, our container has a number of drawbacks:

it runs the service as the root user. In many environments, by default containers are not allowed to run their processes as root
it cannot be controlled by environment variables
the default run command is still /bin/bash
building is not automatic.

In the next part of our docker adventures we will look at building a custom image. We’ll also learn how to run programs inside a container as a normal (unprivileged) user. We will then discuss managing the container using variables passed to the container. In doing so, we will create several images that we will make available under the EuroLinux banner on the Docker Hub platform.

Bonus: deleting all containers and their images.

To remove all containers and images, we’ll use the list commands with the switches -a, the shortcut option --all, and -q, the shortcut option --quiet. Using the -q flag for the docker ps and docker images commands ensures that only IDs are displayed. Sample call to docker images without the -q flag and with the -q flag:

[root@localhost ~]# docker images 
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
eurolinux/centos-8      latest              5cf35e13f8d3        3 weeks ago         182MB
eurolinux/eurolinux-7   latest              3eed865a31cf        3 weeks ago         168MB
eurolinux/eurolinux-7   eurolinux-7-7.7.1   65cb490a7493        6 weeks ago         168MB
[root@localhost ~]# docker images -q -qa
5cf35e13f8d3
3eed865a31cf
65cb490a7493

Outputs with ID alone are ideal for use in other commands. An alternative is to create pipelines using grep and/or awk.

In order to remove all running containers and their images one can use:

docker stop $(docker ps -q) # stop execution of running containers
docker rm $(docker ps -a -q) # delete all containers
docker rmi $(docker images -a -q) # delete container images