hello-dagger/README.md

## Hello Dagger

An example repo demonstrating [Dagger](https://dagger.io).

### What is Dagger?

Dagger is not a CI/CD Runner like GitHub Actions or Gitlab CI. It is an
abstraction that uses and composes containers to run commands. Think of it as
somewhere between `just`, `nix`, and writing custom programs for your build
plus containers all over the place.

It exists as a cli tool which either starts a Dagger Engine locally or connects
to an existing one (in the cloud). This engine runs as a container and runs all
tasks inside the engine as a container as well. The cli tool communicates via
GraphQL with the engine.

Besides the TUI, Dagger Cloud is a commercial offering that is basically a
fancy add-on. It provides web-visualizations of errors during a dagger
execution and of performance metrics (profiling), as well as some caching
features.

The TUI and engine are published under the Apache 2 License and thus can be
used free of charge.

Besides using containers, Dagger's main selling point is the ability to program
tasks in high level languages like Go or Typescript instead of configuring them
in json or yaml. Dagger provides a DSL-like API that reminds one of
`Dockerfile` commands + FP (even though it isn't). Combined with the ecosystem
of the base language nearly anything can be coded and containerized fairly
easy.

### Walkthrough

This repo uses `mise` / `asdf` for setup, see `.tool-versions`.

Initialize the existing repo with dagger:

```shell
dagger init --sdk=typescript --source=./dagger
```

Typescript is just one option to write the config in, Python and Go are also
currently available with more to follow.

> Note: For some reason it wants to create a `LICENSE` file...

> LSPs should work properly out of the box as the dagger config is located in
> the `./dagger` directory. When using TS this is a self-sufficient node/yarn
> project.

To get an overview of available functions to call:

```shell
dagger functions
```

Run a function:

```shell
dagger call build --source=.
```

This runs the `build` function defined in `dagger/src/index.ts` and passes `.`
as the `source` parameter of that function. The return type of this function is
a `Container` type which just prints some general information on the resulting
container when called this way.

```
_type: Container
defaultArgs: []
entrypoint:
    - helloworld
mounts: []
platform: linux/amd64
user: ""
workdir: ""
```

The `Container` has to be consumed by other functions to actually export it to
some registry or run it. The same is true for other types like `File` and
`Directory` which also only expose the actual artifacts through effectful
functions, thus the FP-like feeling. Only 'simple' strings can be exposed on
the shell. Consequently, pipelining on the shell seems not to be a common
thing.

Dagger separates the containerized build environment from the host system.
Unless specifically specified, like with `--source=.`, dagger does not expose
host resources.

> Camel Case function names in the TS setup are available in Kebab Case on the
> shell (same applies for Go and Python). This does not only apply to your own
> declared functions but to all functions available on the returned data types.
>
> ```typescript
> this.build(source).asService().up({ ports: "8080:9000" });
> ```
>
> becomes
>
> ```shell
> dagger call build --source=. as-service up --ports=8080:9000
> ```
>
> Dagger calls this function chaining.

> After running a dagger command the Dagger Engine stays alive as a local
> container waiting for the next call.

> The [Cookbook](https://docs.dagger.io/cookbook) does contain a lot of
> extremely useful patterns on how to handle builds and containers. More
> importantly, it showcases ways to debug everyday problems when working on
> container builds, see `temrinal()`, `File.contents()`

Run `golangci-lint`:

```shell
dagger call lint --source=.
```

Run trivy:

```shell
dagger call security-scan --source=.
```

> Trivy is setup as a dagger module. Modules are most of the time simple
> wrappers instructing container images with some convenience functions. There
> is actually not too much magic here.

> Modules are extension libraries in dagger that can be written in Go, Python,
> or Typescript. Either way, they are usable in any other language, e.g. a
> Python module can be used in a Typescript config. The proper API functions
> are exposed on the `dag` object.

> Most modules are just thin wrappers around specific images that contain for
> example some tool and provide convenience functions.

Push the container image to some registry:

```shell
dagger call build --source=. publish --address=mtr.devops.telekom.de/...
```

This call makes use of the aforementioned chaining capabilities. One could
create a custom function to chain calls within the config or just chain call in
a shell call.

### Deployment to Kubernetes

Dagger at its core does not provide tooling to deploy artifacts to kubernetes
or anywhere else. There are, however, modules that expose APIs for, for
example, [`kubectl`](https://daggerverse.dev/mod/github.com/matipan/daggerverse/kubectl@84cbdbe89185ad94690a9ada1cdfb79f1878ecd7)
and [`helm`](https://daggerverse.dev/mod/github.com/sagikazarmark/daggerverse/helm@126b5fbbdad70dbf2a8689600baec2eb78c05ef4).
Creating automatic deployments based on these should be straight forward. But
at this point one could argue to use better suited tooling for this task rather
than to manually code it again.

### CI

Dagger does not provide a Job Runner. Thus, it has to run within a system like
GitHub Actions, Gitlab CI, or Jenkins. Once it's running, any workload within
dagger should run seamlessly like on your local machine thanks to dagger's
container abstraction.

A critical problem however is the fact that the dagger engine requires running
in privileged mode, see
[FAQ](https://docs.dagger.io/faq#can-i-run-the-dagger-engine-as-a-rootless-container).
Depending on the platform this might be a dealbreaker due to security issues.
If the platform does not provide a somewhat isolated vm environment that is
capable of running containers itself (in this case the dagger cli would spawn
its own instance of the dagger engine), the suggested setup includes hosting
one or more shared dagger engines within the platform, see the Gitlab CI
kubernetes
[example](https://docs.dagger.io/integrations/gitlab#kubernetes-executor).
Security concerns of using a shared instance of the dagger engine should be
investigated if one wants to go down this road.

## Conclusion

Dagger is an extremely neat tool that puts DX front and center. Easy and fast
setup on proper dev machines is a big plus. Intuitive APIs, easy
programmability and containers make creating a build and test environment an
actually nice experience. The debugging capabilities of your build pipeline are
excellent.

However, dagger only covers the pipeline up until the point where you push your
image to a registry. After that you are more or less on your own. This is
actually fine as it is somewhat out of scope.

The requirement of privileged mode and container capabilities in general for
the dagger engine is probably a big headache not only in CI but also on weirdly
restricted dev machines. Furthermore, during testing, dagger seems to not
create actually reproducible builds by default but has to be at least
explicitly configured accordingly, but this needs further investigation. `Nix`
might be a worthwhile alternative.