Docker Guide and QRC
In this section we will have a quick run through of the core docker commands.
Docker has some official docs on this too.
What is Docker
Docker is a containerisation technology. It allows us to create sandboxed processes, (with their own filesystems) on our computers. The processes run on-top of our native filesystem, and make use of its core functionality but give the illusion of being a separate system
These sandboxed processes contain everything that is needed to run the application.
We misuse docker to create lightweight VMs with examples for you to play with. It means that I can distribute tasks in an easy way, as everyone should have exactly the same setup. Additionally, as the images make use of existing OS functionality, they are pretty lightweight. Linux distros, on the other hand are getting fatter (you are looking at around 2GB minimal installs now)
This means you only need to grab a few hundred MB of docker image, rather than a whole new VM each time.
!!! note
While docker is pretty amazing, there are a few things it cant do.
As we share the same OS, things like kernel-level exploits are hard to do.
Additionally, networking functionality (so stuff like firewalls),
is also shared with the host, so becomes difficult.
However, for about 90% of what we need, it's going to be more efficient.
Terminology
Image : A Docker image is the sandbox that contains our filesystem and programs. I like to think of them as the equivalent of an ISO file
Container : A container is a running instance of an image. We can have multiple containers, each based on the same image, which can let us scale applications (although this isn't really applicable here).
Stack : A group of containers running together to serve a purpose. This is a really nifty thing as it can let us plumb together groups of containers and make them appear like a single VM. For example, we can combine images for a Webserver, and database into one seamless system.
Docker-Compose
While we can use low level docker commands (see below) to start and stop containers, or manage the running docker processes. I intend to make use of docker-compose.
Docker-Compose is a system that allows us to specify a "recipe" for running docker containers. For example, specifying which images to start, what port they should listen on and other information. It also allows us to group images together into a stack.
Docker compose makes use of compose files docker-compose.yml that contain
the instructions for building the stack.
!!! tip "Multiple Compose Files"
If we need to deal with multiple compose files, you have two options
- Call them different names and tell compose what one to run using the ```-f``` flag
- Put them in different directories
I prefer the second approach. Compose will also build its images based on the dir
you are working in, so it simplifies image management.
Starting and stopping compose-based services
To start a compose based service docker-compose up
dang@DESKTOP-KJDVQ2J:~/6005/6005-CW-Koala$ docker-compose up
Starting 6005-cw-koala_pyramid_1 ... done
Attaching to 6005-cw-koala_pyramid_1
!!! important "It's not hung"
Docker will continue to run in the background, so it will look like the system
has hung. (In the case of some images it may display logging information)
You will need a new window to interact with the system.
To Stop a compose based service you need to do two things.
- If docker is running in the foreground, stop it with ctrl-c
docker-compose down
# Stop the Container Running
^CGracefully stopping... (press Ctrl+C again to force)
Stopping webtrainer_database_1 ... done
Stopping webtrainer_web_1 ... done
# Remove from Container List
dang@DESKTOP-KJDVQ2J:~/6005/webtrainer$ docker-compose down
Removing webtrainer_database_1 ... done
Removing webtrainer_web_1 ... done
Removing network webtrainer_main_network
Low-Level Docker access.
If we are not using compose we also have the option to deal with docker directly.
Starting and Stopping containers
Container Management
You may also want to examine the currently running containers
To list containers that are currently running you can use docker ps
dang@DESKTOP-KJDVQ2J:~$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
To list all containers (including those that are stopped) docker ps -a
dang@DESKTOP-KJDVQ2J:~$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e4a8e2d38bdb 6005-cw-koala_pyramid "pserve production.i…" 2 weeks ago Exited (137) 2 weeks ago 6005-cw-koala_pyramid_1
8cefe7adba95 7024cem/webdb "docker-entrypoint.s…" 4 weeks ago Exited (255) 2 weeks ago 0.0.0.0:3306->3306/tcp, 33060/tcp webtrainer_database_1
e7d7e6005693 7024cem/webtrainer "docker-php-entrypoi…" 4 weeks ago Exited (255) 2 weeks ago 0.0.0.0:80->80/tcp webtrainer_web_1
6f3bbd7104c7 7024cem/webdb "docker-entrypoint.s…" 5 weeks ago Exited (255) 4 weeks ago 0.0.0.0:3306->3306/tcp, 33060/tcp 6005_database_1
5fecfd61782f 7024cem/webtrainer "docker-php-entrypoi…" 5 weeks ago Exited (255) 4 weeks ago 0.0.0.0:80->80/tcp 6005_web_1
To remove a stopped containers use docker rm <name>
dang@DESKTOP-KJDVQ2J:~$ docker rm 6005-cw-koala_pyramid_1
6005-cw-koala_pyramid_1
Image Management
You can also manage the images currently installed on your system
To list the current set of images use docker image list
dang@DESKTOP-KJDVQ2J:~$ docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
6005-cw-koala_pyramid latest afc71342b28f 4 weeks ago 946MB
python 3 5336a27a9b1f 5 weeks ago 886MB
cueh/ftpanon latest e68af1bf98d7 2 months ago 148MB
cueh/scanningweb latest 4d4adf600a0d 2 months ago 413MB
cueh/typewriter_postfix latest 7c2fe5b9677a 3 months ago 161MB
7024cem/webtrainer latest ee9ee758bfbe 18 months ago 382MB
7024cem/webdb latest 9b235fdedb19 18 months ago 376MB
cueh/debian_ssh latest 80f6116013fe 19 months ago 131MB
To remove an image: docker rmi <image name>
dang@DESKTOP-KJDVQ2J:~$ docker rmi 6005-cw-koala_pyramid:latest
Untagged: 6005-cw-koala_pyramid:latest
Deleted: sha256:afc71342b28feb67e1e7a41a07a5450b432926f46daa6147b8eae98cb5771876
Deleted: sha256:0311c5037503db900b04073069146141c186166a24b4861a26ec88bc750750e3
Deleted: sha256:d538d349d16647f86d475cb1f52cf930d804f87e642c49a1bb846335af73ca84
There is also a command that will prune image layers that are not used by anything else
docker image prune
FAQ
You are asking me to install a lot of images, what about my HDD space.
One of the really cool things about docker is the "layering" system.
This means that images can reuse elements of other images to reduce the
overall size. Unfortunately, the docker image list command reports the total size of
the image, rather than taking account of any layers its reusing1
I have tried to design the images we use to make best use of this layering approach. For example, most of the web-based challenges use the same base image, and the only thing unique to each one is the vulnerable website itself.
This means that you may have 10 web challenges each reporting as being ~200MB each, when in reality it' one "base" image of ~190MB, with a separate 10MB layer for each challenge. So instead of 2GB it's only ~300 MB of space that is actually used.
I keep getting Permission Denied Errors
Docker needs to run as the root user. Try repeating the command with sudo.
$sudo docker-compose up
I get asked if the Daemon is running
If you get a message like
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
In this case it's likely that docker is not started. You can confirm this using whatever init tools you have (for example systemctl / init.d)
#Get Status of docker
$systemctl status docker
#Start Docker
$systemctl start docker
I get a message about network in use
This can sometimes happen if we don't close docker-compose down correctly.
If we just stop the containers they are kept as inactive along with any
configuration that was in place. You can avoid this by using
docker-compose down when you have finished a task
Another method is to use the docker-network prune command.
As this will remove old network configurations from the list.
I cant access the system on WSL
Usually, the docker container is running 'on top' of the host system. This means that any services are exposed as they would be on localhost.
However, with WSL, we are running our Linux base inside a virtualised environment, so can't access at localhost.
Instead, you will need to use the address of the WSL machine that is running docker.
Footnotes
-
My laptop reports that the docker images take up more than twice the space that I actually have available.
↩