# Installation

This documentation describes how docker/OCI image pulls on a local linux box can be configured to connect to mirrors or pull through cache proxies.

The audience is developers who want to have faster or more reliable pulls, and want to avoid rate limits from external registries.

## Introduction

There are four different scenarios, which can be combined arbitrarily:

| Registry type | kind | docker |
| --- | --- | --- |
| mirror | scenario 1 | scenario 3 |
| cache | scenario 2 | scenario 4 |

The scenarios can be combined arbitrarily, but when use use sceario 1 you shouldn't forget to also use the mirror in kind:

| combination | s2 (kind + cache) | s3 (docker + mirror) | s4 (docker + cache) |
| --- | --- | --- | --- |
| s1 (kind + mirror) |  Mirror and Cache only for kind, you probably don't use docker too much | Both Docker and Kind are mirrored and dont have a cache |  |
| s2 (kind + cache)  |  | doesn't make sense without s1 | both kind and docker are cached and dont need a mirror, you probably are in free internet |
| s3 (docker + mirror) |  |  | doesn't make sense without s1  |

## Preliminaries

We will need two container images stored in our container host repo before we do changes to the pull configuration.

So be sure that you have them already successfully pulled, or do it right now:

```bash
# precondition: your current docker config is able to pull images from docker.io
docker pull registry:2
docker pull rpardini/docker-registry-proxy:0.6.2
```

## Scenario 1: Registry Mirror on Kind for MMS company network

### What you need

You need to know the registries you want to mirror, and the address of the mirror, which here will be the MMS artifactory mirror. (remark: see in the last chapter for a test registry as mirror on your box.)

> Hint: Typically only 'docker.io' needs to be mirrored.

### Install

The installation is done by setting the `containerd` configuartion with a registry-mirror entry during kind setup.

```bash
# in MMS: 
MIRROR_NAME=common-docker.artifacts.mms-at-work.de
KIND_CLUSTER_NAME=cluster-with-registry-mirror 
```

In the follwing kind config we only mirror `docker.io`. You can append [as much mirror-entries as you want[(https://github.com/containerd/containerd/blob/main/docs/cri/registry.md#configure-registry-credentials-example---gcr-with-service-account-key-authentication)].

```bash
cat <<EOF | kind create cluster --name $KIND_CLUSTER_NAME --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri"]
    [plugins."io.containerd.grpc.v1.cri".registry]
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
          endpoint = ["${MIRROR_NAME}"]
EOF
```

> Hint: To get logs in info or debug level see section 'hacks'
> Hint: You can also change to `containerd config in a running node. Then just restart the containerd with `systemctl restart containerd`

## Outcome

Now a typical company network blocker is removed - all containerd's (in every cluster node) knows how to bypass the Dockerhub rate limit by requetsing `docker.io` images first from the mirror:

![alt text](./img/2-scenario-1-kind-mirror.png))

The drawback is that you still need the bandwidth to the mirror. So if you are in homeoffice you still will pull each new image on each new kind cliuster creation over the network.

## Scenario 2: Registry Cache Proxy on Kind

Now it gets more tricky or let's say 'even more local': We will install a [cache proxy](`https://github.com/rpardini/docker-registry-proxy) as docker container process on our host.

### What you need

You need the [Caching proxy](`https://github.com/rpardini/docker-registry-proxy). We will install it on your node.

### Install

```bash
CACHE_PROXY_NAME=docker_registry_proxy
DOCKER_KIND_NETWORK=kind
```

#### Caching Proxy

Run the caching proxy `https://github.com/rpardini/docker-registry-proxy`:

```bash
# we start the caching proxy. It will also cache the mirror, if used
# tip: this container is very long running and stores the cached data in a subfolder from your pwd. 
# So it's recommended to start it in a higher level folder.
docker run -itd \
    --restart always \
    --name $CACHE_PROXY_NAME \
    --network $DOCKER_KIND_NETWORK \
    --hostname $CACHE_PROXY_NAME \
    -p 0.0.0.0:${HOST_PORT:-3128}:3128 \
    -e ENABLE_MANIFEST_CACHE=true \
    -v $(pwd)/docker_mirror_cache:/docker_mirror_cache \
    -v $(pwd)/docker_mirror_certs:/ca \
    -e REGISTRIES="$MIRROR_NAME k8s.gcr.io gcr.io quay.io docker.elastic.co" \
    rpardini/docker-registry-proxy:0.6.2
```

#### Kind cluster

Next start your kind cluster if not running yet. You also can use the cluster from above with the mirror set.

> When you run a test registry mirror (see section 'hacks'), then you need either to provide the mirror's certificate inside the proxy server or you setup the proxy server with tls_no_verify (`-e VERIFY_SSL=false`). In the former everything will work as suspected, in the latter the mirror will complain about a missing TLS connectivity to the proxy.  

#### Configure nodes of the Kind cluster 

Now each node's `containerd` needs to be configured to use the Cache proxy on the host:

```bash
#!/bin/sh

# https://github.com/rpardini/docker-registry-proxy#kind-cluster

SETUP_URL=http://$CACHE_PROXY_NAME:3128/setup/systemd
pids=""
for NODE in $(kind get nodes --name "$KIND_CLUSTER_NAME"); do
  docker exec "$NODE" sh -c "\
      curl $SETUP_URL \
      | sed s/docker\.service/containerd\.service/g \
      | sed '/Environment/ s/$/ \"NO_PROXY=127.0.0.0\/8,10.0.0.0\/8,172.16.0.0\/12,192.168.0.0\/16\"/' \
      | bash" & pids="$pids $!" # Configure every node in background
done
wait $pids # Wait for all configurations to end
```

## Outcome

As the proxy is configured all requests - either mirror or registry - go directly to the proxy.

The proxy checks whether first a mirror is to be connected if the image is not in the cache or is to be updated.

![alt text](./img/2-scenario-2-kind-cache.png))

## Scenario 3: Registry Mirror on Docker

> Be aware that Docker only enables a mirror for `docker.io`

### What you need

You need to know he address of the mirror. Here we reuse the ${MIRROR_NAME} from scenario 2.

Only `docker.io` will be mirrored, this is a restriction of docker.

### Run

```bash
# as root

cat << EOD > /etc/docker/daemon.json
{
  "metrics-addr" : "127.0.0.1:9323",
  "experimental" : true,
  "features": { "buildkit": true },
  "registry-mirrors": ["${MIRROR_NAME}"],
  "insecure-registries" : []
}
EOD
```

```bash
# as root
systemctl restart docker
```

## Outcome

![alt text](./img/2-scenario-3-docker-mirror.png))

## Scenario 4: Registry Cache Proxy on Docker

Last not least we connect our hosts docker engine to the proxy cache.

### What you need

You need the Caching proxy. We will reuse it from scenario 2. 

### Run

We run the same systemd-settings for containerd as we did for containerd in the kind nodes. 

```bash
# as root

mkdir -p /etc/systemd/system/docker.service.d
cat << EOD > /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://localhost:3128/"
Environment="HTTPS_PROXY=http://localhost:3128/"
Environment="NO_PROXY=localhost,127.0.0.1,gitea.poc.edp.localtest.me"
EOD

curl http://localhost:3128/ca.crt > /usr/share/ca-certificates/docker_registry_proxy.crt

if fgrep -q "docker_registry_proxy.crt" /etc/ca-certificates.conf ; then
   echo "certificate refreshed"
else
   echo "docker_registry_proxy.crt" >> /etc/ca-certificates.conf
fi

update-ca-certificates --fresh
```

Now reload and restart:

```bash
# as root

systemctl daemon-reload
systemctl restart docker
```

## Outcome

As the proxy is configured all requests - either mirror or registry - go directly to the proxy. 

The proxy checks whether first a mirror is to be connected if the image is not in the cache or is to be updated.

![alt text](./img/2-scenario-4-docker-cache.png))

In the `docker info` output you see proxy and mirror setting:

```bash
~ $ docker info
Client: Docker Engine - Community
 Version:    27.0.2

Server:
...

 HTTP Proxy: http://localhost:3128/
 HTTPS Proxy: http://localhost:3128/
 No Proxy: localhost,127.0.0.1,gitea.poc.edp.localtest.me
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  https://registry-1.docker.io.mirror.test/
 ```
