diff --git a/caching.md b/caching.md index 9368dc0..29cfe4e 100644 --- a/caching.md +++ b/caching.md @@ -34,6 +34,9 @@ - Currently of out Scope - Auth: Cache provides data to anyone who can reach it +- Nice to have + - Repo Cache: Can store uploaded artifacts + ### Architectural Solutions #### File System-Based Caching @@ -98,15 +101,17 @@ - basically only maven support - does not suffice for our use case -- Community Edition has more features but is limited in sizing. Upgrade to Pro edition necessary in those limits are exceeded. +- Community Edition has more features but is limited in sizing. Upgrade to Pro edition necessary if those limits are exceeded. #### Artifactory - Open source / free version + - Limited feature set - Separate distributions per repo type java / container / etc - Inconvenient and insufficient for our use case +- Full feature set requires paid license License evaluation needed [EULA](https://jfrog.com/artifactory/eula/) @@ -116,6 +121,8 @@ License evaluation needed [GH](https://github.com/artipie/artipie) [Wiki](https://github.com/artipie/artipie/wiki) +- Self-hosted and upstream artifact caching + - MIT License - might be abandoned / low dev activity / needs new maintainer @@ -124,17 +131,32 @@ License evaluation needed - Mostly headless - Brings a limited web interface - Repo creation, artifact viewing +- Buggy default config + - config changes require restart, which seems to be a bug? +- Easy to setup, once bugs and buggy config are mitigated/worked around +- File system and object storage supported +- No databases required - Pro: Config in yaml file +- Due to its simplicity it might be a good candidate for a first upstream caching solution + #### Pulp [Website](https://pulpproject.org/) [GH](https://github.com/pulp/pulpcore) +- Self-hosted and upstream artifact caching + - GPL 2.0 License - Pull-Through Caches are only technical previews and might not work correctly + - Pull-through cache does not fit into the concept of how artifacts are stored an tracked + - Intended workflow is to sync dedicated artifacts with some upstream repo, not the entire repo +- Setup and config are quite complex + - Build for high availability +- File system and object storage supported +- Requires SQL Db (Postgres) and possibly Redis #### kube-image-keeper @@ -186,5 +208,17 @@ License evaluation needed - Reduces build times significantly if dependencies have to be downloaded from outside networks - Avoid using fs cache, i.e. forgejo runner cache, long term or at all - Unless you can handle proper cache invalidation - - Promote immutable infra and reproducible builds without side effects + - Promotes immutable infra and reproducible builds without side effects - Use as additional layer if there is no local cache repo + +- Repo caches + - Can replace file system cache if network and repo are fast enough + - Optimal solution would be a Nexus/Artifactory-like unified solution + - Foss solutions like Artipie and Pulp have severe problems + - Requires us to add features/fixes/maintenance + - Due to scarce landscape of proper foss solutions we might have to opt for multiple dedicated solutions + - If we opt for a dedicated container cache, we should re-evaluate Harbor or Quay + +- Try to use Artipie as a first, simple solution and use Forgejo Runner caches in conjunction for even better performance + - If Artipie does not work correctly or does not fit some reason we didn't waste too much time on it + - If Artipie is abandoned but the concept works for us, we should consider maintaining it and continuing its development