It's hard to overemphasize how useful Docker has become for building CI/CD pipelines. If you're maintaining a pipeline that supports multiple operating systems, more than one platform, or even just conflicting versions of the same application, those containers are a blessing: you have a powerful tool for isolating those conflicts. You can download images for nearly every one of your build and deploy tasks, build custom containers to address the conflicts, and focus on keeping your build streams moving. But those containers proliferate. Sometimes you need to clean up or even delete all Docker images to release or refresh resources.
In this post, we'll cover how to manage your Docker images. Let's get right to it!
Delete All Images for the Impatient
This posts' title promises to tell you how to delete all Docker images, so let's start with that.
The command is simple:
docker rmi $(docker images -a -q)
Here it is in action.
First, this window lists images and then deletes all of them:
After a great deal more output, you can see that this one line command deleted all of the images:
So that's it; we're done, right?
No, let's unpack this command and then look at more of the options available for maintaining Docker images.
Delete All Images in Detail
We started out by listing the images on our system with docker image ls:
This command lists your images with some details, such as the image tag, ID, size, and age.
Then came the command to delete the images. It starts with docker rmi. Rmi deletes Docker images by ID.
The remove command is followed by this:
$(docker images -a -q)
Let's run that alone, without the dollar sign and parentheses:
Running docker images with -a and-q yields a list of all image IDs!
Let's put it back in the dollar sign and parentheses:
We see the IDs without the carriage returns.
So the original command simply passes a list of images IDs to Docker's image remove command:
docker rmi $(docker images -a -q)
Cleaning Unused Images
Removing every image on your Docker host is the extreme case. It's more common to need to clean up unused and dangling images. Let's see how you can perform these maintenance tasks.
Docker builds its images in layers. If two images on a system share a common ancestor, Docker loads the ancestor just once. This is one of the ways that Docker saves space. Let's look at an example.
Here's a simple Dockerfile:
from alpine
RUN touch /foo
It starts with the alpine image and adds a single file to it.
This Dockerfile is in a directory named test. Working from there, let's list the images on the system, build the image namec foo, and then list the images on the system.
egoebelbecker@zaku:~/test$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE jenkins/ssh-agent latest c82a6a075443 44 hours ago 237MB redis latest f1b6973564e9 3 days ago 113MB nginx latest c316d5a335a5 3 days ago 142MB jenkins/jenkins latest 47f4a9dd6716 4 days ago 442MB docker/getting-started latest 26d80cd96d69 8 weeks ago 28.5MB egoebelbecker@zaku:~/test$ docker build -t foo . Sending build context to Docker daemon 2.048kB Step 1/2 : from alpine latest: Pulling from library/alpine 59bf1c3509f3: Pull complete Digest: sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300 Status: Downloaded newer image for alpine:latest ---> c059bfaa849c Step 2/2 : RUN touch /foo ---> Running in 50f6ba9d390c Removing intermediate container 50f6ba9d390c ---> 2a23ebc1af77 Successfully built 2a23ebc1af77 Successfully tagged foo:latest egoebelbecker@zaku:~/test$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE foo latest 2a23ebc1af77 3 seconds ago 5.59MB jenkins/ssh-agent latest c82a6a075443 44 hours ago 237MB redis latest f1b6973564e9 3 days ago 113MB nginx latest c316d5a335a5 3 days ago 142MB jenkins/jenkins latest 47f4a9dd6716 4 days ago 442MB docker/getting-started latest 26d80cd96d69 8 weeks ago 28.5MB alpine latest c059bfaa849c 2 months ago 5.59MB egoebelbecker@zaku:~/test$
Docker downloaded alpine and created foo.
Let's add another step to the Dockerfile. Create a new file named foobar and modify the Dockerfile to read like this. Be sure to copy it exactly. The error is there for a reason.
from alpine
COPY foobar /
RUN touch /foo
COPY foobar /tmp
RUN toch /bar
Try to build this image.
egoebelbecker@zaku:~/test$ docker build -t bar . Sending build context to Docker daemon 2.56kB Step 1/5 : from alpine ---> c059bfaa849c Step 2/5 : COPY foobar / ---> e3fecf6357b6 Step 3/5 : RUN touch /foo ---> Running in 3cbb9281e9c8 Removing intermediate container 3cbb9281e9c8 ---> 0ba7f3647b9e Step 4/5 : COPY foobar /tmp ---> 467026d0c6d8 Step 5/5 : RUN toch /bar ---> Running in b1dbe7e986c2 /bin/sh: toch: not found The command '/bin/sh -c toch /bar' returned a non-zero code: 127
It fails because toch is a typo.
List your image again.
egoebelbecker@zaku:~/test$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> 467026d0c6d8 4 seconds ago 5.59MB foo latest 2a23ebc1af77 11 minutes ago 5.59MB jenkins/ssh-agent latest c82a6a075443 45 hours ago 237MB redis latest f1b6973564e9 3 days ago 113MB nginx latest c316d5a335a5 3 days ago 142MB jenkins/jenkins latest 47f4a9dd6716 4 days ago 442MB docker/getting-started latest 26d80cd96d69 8 weeks ago 28.5MB alpine latest c059bfaa849c 2 months ago 5.59MB
The failed build created a dangling image. It added two copies of foobar to a copy of the alpine image, then failed on the broken touch command. So we have an image that appears to be taking up 5 MB of disk space.
Docker image prune cleans up dangling images.
egoebelbecker@zaku:~/test$ docker image prune
WARNING! This will remove all dangling images.
Are you sure you want to continue? [y/N] y
Total reclaimed space: 0B
It didn't work? That means the dangling image isn't really dangling yet.
egoebelbecker@zaku:~/test$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS NAMES b1dbe7e986c2 467026d0c6d8 "/bin/sh -c 'toch /b…" 7 minutes ago Exited (127) 7 minutes ago cool_gates
The broken build left a container loaded. Let's remove it and then try to prune again.
egoebelbecker@zaku:~/test$ docker rm cool_gates cool_gates egoebelbecker@zaku:~/test$ docker image prune WARNING! This will remove all dangling images. Are you sure you want to continue? [y/N] y Deleted Images: deleted: sha256:467026d0c6d81337bb10fffb6244c028f78749e0cb2a0251c11871d51c378fa4 deleted: sha256:c990e6ab97f197632bd86c3ec11601d8ba74edac0c0056ffea4df82e25283b6d deleted: sha256:0ba7f3647b9e0789b73f8e2682b8247315f12367f228d5f8f0a03276e027a952 deleted: sha256:073afc57bac110e8ce0a1a21991f7e13cfb564ec985f443260656d0311991b43 deleted: sha256:e3fecf6357b64219a7e99473eeb4c5b1e44d905dac04d0b8a56657dd4bc3b7b8 deleted: sha256:53c3b0757c47aa3be85ad83f6fcf39fe107d0028b7503a93f754a45a487092a6 Total reclaimed space: 0B egoebelbecker@zaku:~/test$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE foo latest 2a23ebc1af77 21 minutes ago 5.59MB jenkins/ssh-agent latest c82a6a075443 45 hours ago 237MB redis latest f1b6973564e9 3 days ago 113MB nginx latest c316d5a335a5 3 days ago 142MB jenkins/jenkins latest 47f4a9dd6716 4 days ago 442MB docker/getting-started latest 26d80cd96d69 8 weeks ago 28.5MB alpine latest c059bfaa849c 2 months ago 5.59MB egoebelbecker@zaku:~/test$
That worked! Prune removed the dangling image this time. But it turns out it wasn't taking up any space at all! That's because Docker was only storing the differences between alpine and the incomplete image, which was only a few references to an empty file.
Dangling images aren't only created by build errors. Docker also creates them when it pulls a new version of an image while the older version is still running in a container. Running a prune from time to time can recover a lot of drive space on your Docker hosts.
Cleaning Up Containers
There isn't much you can do to prevent the mess a broken image build creates. You have to remember to check for stopped containers and remove them when a build fails, even if that means stopping and removing all containers at once. But you can make sure that your containers clean up after themselves when you run them.
Let's run the docker/getting-started image, then stop it right away. (We'll give it a name when we run it.)
egoebelbecker@zaku:~$ docker run -d --name example docker/getting-started 21c5c2e8c5f6158e78c766c5dde2e6fd8b8fd2f9053727f2f5769a26ac7373a4 egoebelbecker@zaku:~$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 21c5c2e8c5f6 docker/getting-started "/docker-entrypoint.…" 4 seconds ago Up 3 seconds 80/tcp example egoebelbecker@zaku:~$ docker stop example example egoebelbecker@zaku:~$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 21c5c2e8c5f6 docker/getting-started "/docker-entrypoint.…" 11 seconds ago Exited (0) 1 second ago example egoebelbecker@zaku:~$
The container stayed loaded in the stopped state. If we tried to clean up the associated image, we would fail.
Fortunately, Docker has a solution for this. The --rm flag tells Docker to clean up containers for us.
Run getting-started again with that flag, then stop it.
egoebelbecker@zaku:~$ docker run -d --rm --name example docker/getting-started 9ca576489926a399ae7d83b1f8d9938b63f8c26eddd13c79de0eca1ae8442ca6 egoebelbecker@zaku:~$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9ca576489926 docker/getting-started "/docker-entrypoint.…" 4 seconds ago Up 3 seconds 80/tcp example egoebelbecker@zaku:~$ docker stop example example egoebelbecker@zaku:~$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
It's gone! The advantage to the --rm option is that docker run cleans up after itself. The downside is that Docker will remove the logs, too. If you need to view your applications logs after a container is topped, be sure to save them somewhere else.
Cleaning up Docker Images
In this article, you learned how to clean up your Docker images. We saw how to remove all of the images on your host, how to clean up unused and dangling images, and how to get Docker to clean up after itself when it runs a container.
Now you're familiar with some key Docker tasks and when you need to perform them. Get to work using Docker in your CI/CD pipelines today!
This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).