# Introduction

This documentation describes how docker/OCI image pulls on a local linux box can be configured to connect to mirrors or pull through cache proxies.

The audience is developers who want to have faster or more reliable pulls, and want to avoid rate limits from external registries.

## Overview

This documantation has three parts:

1. backgound, (docker) image basics - this file
1. Installation of mirrors and caches
1. some hacks, e.g. a local mirror

## It's all about Processes!

We talk about 'docker images' and the way we 'pull' them from 'registries'.

So let's first go one step back and think about the context we use these terms and why.

The reason is that we want to run containers as processes and that they somehow need to come into life, and images are very low level artifacts on top of running conatiners.

> This is our developer's intention! We want processes running our application code!

![alt text](./img/1-linux-processes.png)

### Spawn processes by different stacks

The funny thing is that however you spawn a process you see it at the end as a normal linux process:

#### Run by a shell

```bash
bash -c 'exec -a my-job-bash sleep infinity' &
```

#### Run by Docker

```bash
docker run --restart=always -d ubuntu bash -c 'exec -a my-job-docker sleep infinity'
```

#### Run by Kubernetes

```bash
kubectl run k8sjob --image=ubuntu -- bash -c 'exec -a my-job-k8s sleep infinity'
```

#### Outcome

Your process list should look sth. like this ! :-)

```bash
~ $ ps ax | grep myjob
  22529 pts/1    S      0:00 my-job-bash infinity
  23154 ?        Ss+    0:00 my-job-docker infinity
  24163 ?        Ss     0:00 my-job-k8s infinity
```

### Extra

Try to kill the jobs and look what happens!

## Container Images

In the case of the Docker and Kubernetes 'Orchestrating overhead' we need 'images' as a base for the runtime.

> Hint: You can analyze a local image with https://github.com/containers/skopeo

* https://earthly.dev/blog/docker-image-storage-on-host/
* https://www.freecodecamp.org/news/where-are-docker-images-stored-docker-container-paths-explained/

```bash
~ $ docker image ls 
REPOSITORY                                                TAG              IMAGE ID       CREATED         SIZE
ubuntu                                                    latest           a04dc4851cbc   3 weeks ago     78.1MB
kindest/node                                              latest           2d9b4b74084a   2 months ago    1.05GB
ghcr.io/catthehacker/ubuntu                               act-latest       0fbcdbe238bf   2 months ago    1.47GB
registry                                                  2                282bd1664cf1   16 months ago   25.4MB
ubuntu                                                    18.04            f9a80a55f492   21 months ago   63.2MB
act-actions-action1-dockeraction                          latest           f9a80a55f492   21 months ago   63.2MB
rpardini/docker-registry-proxy                            0.6.2            6bbe4e47a504   4 years ago     12.3MB
registry.k8s.io/pause                                     latest           350b164e7ae1   10 years ago    240kB

~ $ docker exec -it cluster-with-registry-mirror-control-plane crictl image ls
IMAGE                                           TAG                  IMAGE ID            SIZE
docker.io/kindest/kindnetd                      v20241212-9f82dd49   d300845f67aeb       39MB
docker.io/kindest/local-path-helper             v20241212-8ac705d0   baa0d31514ee5       3.08MB
docker.io/kindest/local-path-provisioner        v20241212-8ac705d0   04b7d0b91e7e5       22.5MB
docker.io/library/ubuntu                        latest               a04dc4851cbcb       29.8MB
registry.k8s.io/coredns/coredns                 v1.11.3              c69fa2e9cbf5f       18.6MB
registry.k8s.io/etcd                            3.5.16-0             a9e7e6b294baf       57.7MB
registry.k8s.io/kube-apiserver-amd64            v1.32.0              73afaf82c9cc3       98MB
registry.k8s.io/kube-apiserver                  v1.32.0              73afaf82c9cc3       98MB
registry.k8s.io/kube-controller-manager-amd64   v1.32.0              f3548c6ff8a1e       90.8MB
registry.k8s.io/kube-controller-manager         v1.32.0              f3548c6ff8a1e       90.8MB
registry.k8s.io/kube-proxy-amd64                v1.32.0              aa194712e698a       95.3MB
registry.k8s.io/kube-proxy                      v1.32.0              aa194712e698a       95.3MB
registry.k8s.io/kube-scheduler-amd64            v1.32.0              faaacead470c4       70.6MB
registry.k8s.io/kube-scheduler                  v1.32.0              faaacead470c4       70.6MB
registry.k8s.io/pause                           3.10                 873ed75102791       320kB
```

### Distributable Images are stored in Registries

* Images which not have been built in the local host image store come from registries.
* They are complex compounds where the parts are stored in a `image repository`.
* For now it's not important how they are composed - we are just interested to understand how we pull them.
* **The parts we are pulling is our subject to cache!**

![alt text](img/1-images.png)

### Image Name Syntax

The naming conventions are a bit fuzzy. The best definition is this one:

*`registry/namespace/repo:tag`*

![alt text](./img/1-image-naming-convention.png)

## Engine

Next let's find out which components do the pull, so that we know what needs to be configured to use our mirror and caching components.

Images are prepared and processed by so called 'container engines', so that we have a running container at the end. These engines comply both to the OCI specification (for the underlying runtime), and secondly to the CRI spectification (for Kubernetes, to abstract away the container engine stuff).

### Different possibilities for the engine

<!--https://docs.google.com/presentation/d/1S-JqLQ4jatHwEBRUQRiA5WOuCwpTUnxl2d1qRUoTz5g p.37 -->
![alt text](./img/2-differnt-engines.png)

## OCI/CRI stack

As we want to run kind and docker we focus on engines which are both Docker and Kubernetes compliant.

<!-- https://www.kreyman.de/index.php/others/linux-kubernetes/232-unterschiede-zwischen-docker-containerd-cri-o-und-runc -->
![alt text](./img/docker-containerd-cri-o-und-runc.png)

This is possible by `containerd`, and then we have these two engine stacks with respect to Docker and Kubernetes:
> Originally containerd came from Docker, but it has a Plugin called 'cri-shim'

### Docker - container engine stack

* dockerd - container daemon
* containerd - high-level container runtime
* runc - low-level container runtime

### Kubernetes - container engine stack

* option 1 CRI-O-based:
  * CRI-O - high-level container runtime
  * runc - low-level container runtime
* option 2 containerd-based:
  * containerd - high-level container runtime
  * runc - low-level container runtime

The purposeful significance of containerd is also shown [in this even broader picture of tools and runtimes](https://sarusso.github.io/blog/container-engines-runtimes-orchestrators.html):
![alt text](./img/container-engines-runtimes-orchestrators.png)



### Kind uses containerd

The CRI part of the stack is implemented in our Kind and Docker case by containerd (and runc)

```bash
~ $ k get nodes -o wide
NAME                                         STATUS   ROLES           AGE    VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION     CONTAINER-RUNTIME
cluster-with-registry-mirror-control-plane   Ready    control-plane   7h4m   v1.32.0   172.18.0.4    <none>        Debian GNU/Linux 12 (bookworm)   6.8.0-53-generic   containerd://1.7.24
```

### How this relates to images

As a side note: How the engine operates on images to pass it to the dring component `runc` is nicely shown here:

<!--https://docs.google.com/presentation/d/1S-JqLQ4jatHwEBRUQRiA5WOuCwpTUnxl2d1qRUoTz5g p.36 -->

![alt text](./img/1-container-engine.png)

## Outcome

In a local

* Linux host setup

with

* Kind-Kubernetes and
* Docker

as container engines we can focus on dockerd and containerd when we want to handle image pulling.

We must also focus on dockerd (not only containerd, although it's also in der docker engine included) as dockerd overwrites the containerd mirror config.

## references

* https://collabnix.com/monitoring-containerd
* https://github.com/containerd/containerd/blob/main/docs/cri/registry.md#configure-registry-credentials-example---gcr-with-service-account-key-authentication
* https://medium.com/@charled.breteche/caching-docker-images-for-local-kind-clusters-252fac5434aa
* https://maelvls.dev/docker-proxy-registry-kind/