Compare commits

...
This repository has been archived on 2025-07-04. You can view files and clone it, but cannot push or open issues or pull requests.

101 commits

Author SHA1 Message Date
e240efb872 feat(stacks): change references to point to new PROD 2025-07-04 10:49:42 +02:00
65ecf59bbf feat(forgejo): reconfigure loadbalancer to allow ssh 2025-07-04 10:05:57 +02:00
5f116820d0 feat(forgejo): new name and slogan 2025-07-02 14:04:40 +00:00
249ef87844 chore(image): switched to forgejo-edp 2025-07-02 09:51:57 +00:00
4678343084 chore(deps): update to forgejo v11, which is helm chart version v12 2025-07-02 09:47:21 +00:00
4e91bab0c8 feat(observability-client): user and password for vector referenced from a secret 2025-06-26 15:05:00 +02:00
12e12a5d60 fix(grafana): changed host name 2025-06-26 11:22:34 +02:00
1ee8310b51 feat(grafana): added ingress 2025-06-25 16:13:28 +02:00
3c8eaf8fff fix(enc): of victoria pvc's 2025-06-25 15:52:52 +02:00
60e1d119c1 feat(observability): encrypt persistent data 2025-06-25 10:00:25 +02:00
63218c5847 fix(o12y): 2025-06-23 16:29:08 +02:00
dc1052182d fix(grafana): 2025-06-23 16:18:16 +02:00
8d1d968b7b feat(o12y): parametrized enpoints 2025-06-23 16:02:39 +02:00
e1e1efa1e7 feat(observability): exchanged hardcoded observability endpoint to a parametrized one 2025-06-23 15:28:19 +02:00
80a8fe661b fix(victoriaMetrics): added cluster_environment label to logs and metrics 2025-06-23 11:27:37 +02:00
51fe5cc5b5 feat(observability): added cluster_environment label
Refs: DevFW/infra-deploy#36
2025-06-20 15:16:01 +02:00
2ef4f03ce6 feat(observability): added cluster_environment label
Refs: DevFW/infra-deploy#36
2025-06-20 15:08:06 +02:00
9625a8cf84 feat(observability): added dashboards for ingress, victoria-logs, argocd. Added grafana-operator 2025-06-20 14:19:33 +02:00
13e4dec40c
fix: 🔧 Update VictoriaMetrics datasource configuration
Refines the VictoriaMetrics datasource settings in Grafana by updating the name and type to ensure compatibility with the latest version.

Additionally, enables the logs datasource plugin for improved observability.
2025-06-20 10:18:24 +02:00
ca953b5074 fix(forgejo): switched from rollingUpdate to recreate 2025-06-19 14:25:30 +02:00
6e8c5673e2 feat(database): use verify-ca instead of verify-full 2025-06-17 14:58:10 +02:00
4b50107eaa feat(database): enable ssl mode in database 2025-06-17 14:43:04 +02:00
40fa8d0c11 feat(database): test with bogus folder name 2025-06-17 14:35:50 +02:00
21d6c39b5c feat(database): rename postgres certificate in k8s secrets 2025-06-17 14:30:51 +02:00
c303b38966 feat(database): rename postgres certificate in k8s secrets 2025-06-17 14:19:46 +02:00
5500b58a9a feat(database): use certificate folder for elasticsearch 2025-06-17 14:05:41 +02:00
d14138996e feat(database): change folder for certificate 2025-06-17 13:55:36 +02:00
e3a93114c9 feat(database): use itemized list in extra volume for certificate 2025-06-17 13:50:11 +02:00
47de41ba4c fix(database): try creating a custom ssl certificate directory 2025-06-17 13:34:02 +02:00
19cfe4dec8 fix(database): reverted changes 2025-06-17 11:18:33 +02:00
aa30d027bf feat(database): use common directory for certificates 2025-06-17 11:03:56 +02:00
bfad711a9a feat(database): enable postgres tls verification 2025-06-17 09:32:45 +02:00
6841bdf94d feat(postgres): added volumeMount for postgres cert 2025-06-16 17:28:02 +02:00
acf5c7f284 fix(pipeline): fix argocd path routing due to domain 2025-06-16 15:07:16 +02:00
087b6d9c49 fix(application urls): fixed argocd url 2025-06-16 14:52:19 +02:00
ab8791d530 fix(application urls): renamed gitea domain 2025-06-16 14:42:11 +02:00
8cf22ec66f
feat(pvc): Increase persistence size and add annotations
Updates the persistence size from 5Gi to 200Gi to accommodate larger data storage needs.

Adds annotations for KMS key ID to enhance security and management of persistent volumes.
2025-06-16 14:01:57 +02:00
1a12aa3674
fix(mail): 🔧 Update connection string reference in values.yaml
Corrects the key reference in the configuration for Gitea's email user credentials from 'password' to 'connection-string' to ensure proper connection handling.
2025-06-16 13:23:40 +02:00
2e57ab463e
feat(mail): Update mailer configuration and add credentials
Enhances the mailer setup by updating SMTP details to use secure connection and new credentials.

Adds a reference to secret key for email password, improving security for email communication.
2025-06-16 13:09:23 +02:00
c87a920b74 Update template/stacks/observability-client/vm-client-stack.yaml 2025-06-13 07:34:09 +00:00
0fab276a36 feat(o12y): Added SIG Kubernetes Metrics Server, closes #33 2025-06-12 14:19:26 +02:00
c1c2d4f1ea
Merge branch 'o12yclient' 2025-06-10 11:20:08 +02:00
923d549290 fix(observability): Changed auth route target to new name 2025-06-10 09:16:42 +00:00
19c4694119 fix(observability): Removed auth lifetime config 2025-06-10 09:16:42 +00:00
eacdcf2eae feat(observability): Disabled grafana auth protection 2025-06-10 09:16:42 +00:00
050c774db0 fix(observability): Switched to ServerSideApply for o12y stack 2025-06-10 09:16:42 +00:00
b2ca785ff2 refactor(observability): Renamed argo app to o12y 2025-06-10 09:16:42 +00:00
bcfd471073 fix(vmetrics): fixed the vmetrics route 2025-06-10 09:16:42 +00:00
17b13041b4 feat(observability): Created observability-client stack
Moved vector from core stack to observability-client
Added victoriametrics-k8s-stack to observability-client for easy vmagent
and scraping config
2025-06-10 09:16:42 +00:00
9bd4871127 Update template/stacks/forgejo/forgejo-server.yaml 2025-06-06 09:50:20 +00:00
e5b633fbf4 Update template/stacks/forgejo/forgejo-server.yaml 2025-06-06 09:46:11 +00:00
fc860747fd feat(forgejo,argocd): Fixed the Forgejo ingress and moved argocd and forgejo ingresses into the argocd and forgejo application manifests folder 2025-06-06 11:34:30 +02:00
fc12862e12 feat(forgejo,argocd): Fixed the Forgejo ingress and moved argocd and forgejo ingresses into the argocd and forgejo application manifests folder 2025-06-06 11:29:46 +02:00
490e4fcfd9 fix(forgejo): renamed forgejo service to match forgejo-server- 2025-06-06 10:12:13 +02:00
358be3205b
fix(forgejo): Properly interpolate minio bucket name in forgejo config 2025-06-04 16:27:10 +02:00
b775019744
feat: 🎉 Add SSL certificate configuration for deployment
Adds configuration for SSL certificate in the deployment settings by introducing environment variables and volume mounts for the Elasticsearch certificate.

This enhancement improves security by ensuring that the application can properly utilize SSL certificates for secure communication.
2025-06-03 16:54:06 +02:00
4761fef87c feat(forgejo): Resolved duplicate forgejo argocd application name 2025-06-03 14:19:47 +02:00
104b811e7e Update template/registry/forgejo.yaml 2025-06-03 12:11:31 +00:00
02d9d207dd feat(forgejo): separate forgejo from core into its own stack 2025-06-03 10:17:24 +02:00
dd46f37e43
feat: Add Elasticsearch indexer configuration
Introduces the configuration for the issue indexer using Elasticsearch, enabling the ISSUE_INDEXER feature.

Sets the ISSUE_INDEXER_ENABLED flag to true and specifies the connection string sourced from a secret.

Prepares for future enhancements by including placeholders for repository indexing options.
2025-06-02 17:39:15 +02:00
e1bf3012e2 feat(forgejo): database reference refactoring 2025-06-02 15:05:51 +02:00
942cedd845
feat(observability): Switched to static endpoints due to bug in CRD selector
CRD selected the wrong port otherwise
2025-06-02 14:57:12 +02:00
fc34fb4ee6 enabled authorized access to vlogs and vmetrics 2025-06-02 14:21:31 +02:00
32bb201e82 feat(forgejo): rename forgejo database host secret key 2025-06-02 13:13:42 +02:00
Bot
15457a0f81 feat(forgejo): Added postgres password 2025-05-30 18:02:59 +02:00
Bot
7a05ca605b feat(forgejo): Added postgres to forgejo ini 2025-05-30 16:49:03 +02:00
fda834d703 feat(redis): removed duplicate entries in forgejo values.yaml 2025-05-30 09:25:14 +00:00
3752fbd341 feat(observability): Added rewrite rules for prometheus remote write to victoria metrics 2025-05-28 16:00:27 +02:00
d4ef3d4a44 feat(grafana): added basic persistence for grafana 2025-05-28 14:54:10 +02:00
00dd935a88 Update template/stacks/core/forgejo/values.yaml 2025-05-28 12:21:15 +00:00
774871c878
feat: 🎉 Add MinIO credentials for repository archiving
Adds MinIO access and secret keys for repository archiving functionality in the configuration.

This enhancement ensures that the necessary credentials are securely referenced, improving access to MinIO storage for archived repositories.

Relates to improved storage management.
2025-05-28 10:31:28 +02:00
528b44a1ba feat(pipeline): Created managed storage for forgejo 2025-05-27 16:33:20 +02:00
95ba18bb56 Removed cert-manager argocd application manifest releaseName entry to prevent out-of-sync state 2025-05-27 08:59:08 +00:00
1f38cc5755 Delete template/stacks/core/ingress-apps/openbao.yaml 2025-05-27 06:59:54 +00:00
0c2e94dc24 Delete template/stacks/core/ingress-apps/mailhog.yaml 2025-05-27 06:59:48 +00:00
ad72626d27 Delete template/stacks/core/ingress-apps/kube-prometheus-stack-grafana.yaml 2025-05-27 06:59:32 +00:00
7cdeed9aff Delete template/stacks/core/ingress-apps/keycloak-ingress-localhost.yaml 2025-05-27 06:59:23 +00:00
5ca95ca4ff Delete template/stacks/core/ingress-apps/backstage.yaml 2025-05-27 06:59:18 +00:00
96c514912d Delete template/stacks/core/ingress-apps/argo-workflows-ingress.yaml 2025-05-27 06:59:12 +00:00
5f91a08c42 Removed unneded code 2025-05-26 17:13:29 +00:00
d76579d814 Update template/stacks/observability/victoria-k8s-stack/values.yaml 2025-05-26 17:12:47 +00:00
29d6cc2660
fix(victoria-k8s-stack): Fixed TLS connection for observability stack 2025-05-26 17:07:28 +02:00
ff978767f6 feat(victoria-k8s-stack): added vmauth 2025-05-26 16:37:28 +02:00
d80ef86286 fix: test environment to ini 2025-05-26 16:21:30 +02:00
1fce183187 fix(vector): use correct deployment name for vector 2025-05-26 13:26:59 +00:00
654daa1743 feat(observability): added vector as logshipper in the core stack 2025-05-21 11:57:36 +02:00
1f7b8e962e fix(victoria): fixes helm value path 2025-05-20 17:26:15 +02:00
9dd41f8b6d feat(otc): Setting ArgoCD retry limit to -1 to core and otc stack 2025-05-20 16:21:55 +02:00
08c8ea6a39 feat(observability): added new stack bases on victoria-k8s-stack 2025-05-20 15:47:05 +02:00
b824738a34 feat(monitoring): remove monitoring as in forgejo-as-a-service we switch to central monitoring 2025-05-20 14:57:57 +02:00
1343794825
chore(cert): Switchted to prod let's encrypt 2025-05-14 14:48:42 +02:00
137cfca08c feat: deleted keycloak related argocd, forgejo manifests 2025-05-14 13:56:10 +02:00
5075deec67 feat(otc) changed nginx-ingress service annotation from custom-eip to eip 2025-05-09 13:51:51 +02:00
74e97f0dcd feat(otc) fixed argocd ingress-nginx settings 2025-05-08 15:48:39 +02:00
56be9fa0b2 feat(otc) setup cert-manager in the ingresses 2025-05-08 15:31:03 +02:00
ec862e92eb Added cert-manager to otc stack 2025-05-08 15:10:59 +02:00
cc107f4ff4 feat(otc): Added LB IP ID 2025-05-07 17:10:12 +02:00
03f113f339 feat(otc): Fixed typo 2025-05-07 16:46:07 +02:00
6908182367 feat(otc): Moved ingress-nginx to otc stack, removed KIND stuff and added OTC annotations 2025-05-07 16:44:39 +02:00
d2cce953a1
feat: 🏗️ Add otc stack 2025-05-06 16:35:20 +02:00
48b6067bf8
feat: 🗃️ Add storageclass for otc 2025-05-06 16:31:38 +02:00
142 changed files with 3187 additions and 18932 deletions

View file

@ -1,7 +1,3 @@
# edpbuilder stacks
This repository contains the building blocks to instanciate Internal Developer Platform's.
### Install edpbuilder
To get started, you need to install [edpbuilder](https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW/edpbuilder).
This repository contains the building blocks to instantiate Internal Developer Platforms.

View file

@ -1,7 +1,7 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: monitoring
name: forgejo
namespace: argocd
labels:
env: dev
@ -12,7 +12,7 @@ spec:
name: in-cluster
namespace: argocd
source:
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/monitoring"
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/forgejo"
repoURL: "https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}"
targetRevision: HEAD
project: default

View file

@ -1,22 +1,24 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: ingress-apps
name: observability-client
namespace: argocd
labels:
example: ref-implementation
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
destination:
server: "https://kubernetes.default.svc"
name: in-cluster
namespace: argocd
source:
repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/observability-client"
repoURL: "https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}"
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/core/ingress-apps"
project: default
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -1,7 +1,7 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: ref-implementation
name: observability
namespace: argocd
labels:
env: dev
@ -12,7 +12,7 @@ spec:
name: in-cluster
namespace: argocd
source:
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/ref-implementation"
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/observability"
repoURL: "https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}"
targetRevision: HEAD
project: default

View file

@ -1,7 +1,7 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: local-backup
name: otc
namespace: argocd
labels:
env: dev
@ -12,7 +12,7 @@ spec:
name: in-cluster
namespace: argocd
source:
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/local-backup"
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/otc"
repoURL: "https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}"
targetRevision: HEAD
project: default

View file

@ -12,11 +12,13 @@ spec:
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: -1
destination:
name: in-cluster
namespace: argocd
sources:
- repoURL: https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW-CICD/argocd-helm.git
- repoURL: https://edp.buildth.ing/DevFW-CICD/argocd-helm.git
path: charts/argo-cd
# TODO: RIRE Can be updated when https://github.com/argoproj/argo-cd/issues/20790 is fixed and merged
# As logout make problems, it is suggested to switch from path based routing to an own argocd domain,

View file

@ -1,29 +0,0 @@
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: forgejo-access-token
namespace: argocd
spec:
secretStoreRef:
name: gitea
kind: ClusterSecretStore
refreshInterval: "0"
target:
name: forgejo-access-token
template:
engineVersion: v2
data:
forgejo_username: "{{.FORGEJO_ACCESS_USERNAME}}"
forgejo_token: "{{.FORGEJO_ACCESS_TOKEN}}"
metadata:
labels:
app.kubernetes.io/part-of: argocd
data:
- secretKey: FORGEJO_ACCESS_USERNAME
remoteRef:
key: forgejo-access-token
property: username
- secretKey: FORGEJO_ACCESS_TOKEN
remoteRef:
key: forgejo-access-token
property: token

View file

@ -1,24 +0,0 @@
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: auth-generic-oauth-secret
namespace: argocd
spec:
secretStoreRef:
name: keycloak
kind: ClusterSecretStore
refreshInterval: "0"
target:
name: auth-generic-oauth-secret
template:
engineVersion: v2
data:
client_secret: "{{.ARGOCD_CLIENT_SECRET}}"
metadata:
labels:
app.kubernetes.io/part-of: argocd
data:
- secretKey: ARGOCD_CLIENT_SECRET
remoteRef:
key: keycloak-clients
property: ARGOCD_CLIENT_SECRET

View file

@ -4,9 +4,10 @@ metadata:
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTP
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
cert-manager.io/cluster-issuer: main
{{{ if eq .Env.CLUSTER_TYPE "osc" }}}
dns.gardener.cloud/class: garden
dns.gardener.cloud/dnsnames: {{{ .Env.DOMAIN }}}
dns.gardener.cloud/dnsnames: {{{ .Env.DOMAIN_ARGOCD }}}
dns.gardener.cloud/ttl: "600"
{{{ end }}}
name: argocd-server
@ -14,7 +15,7 @@ metadata:
spec:
ingressClassName: nginx
rules:
- host: {{{ .Env.DOMAIN }}}
- host: {{{ .Env.DOMAIN_ARGOCD }}}
http:
paths:
- backend:
@ -22,9 +23,9 @@ spec:
name: argocd-server
port:
number: 80
path: /argocd
path: /
pathType: Prefix
tls:
- hosts:
- {{{ .Env.DOMAIN }}}
- {{{ .Env.DOMAIN_ARGOCD }}}
secretName: argocd-net-tls

View file

@ -1,54 +0,0 @@
---
apiVersion: batch/v1
kind: Job
metadata:
name: argocd-config
namespace: argocd
spec:
template:
metadata:
generateName: argocd-config-
spec:
restartPolicy: OnFailure
containers:
- name: push
image: docker.io/library/ubuntu:22.04
env:
- name: FORGEJO_USER
valueFrom:
secretKeyRef:
name: forgejo-access-token
key: forgejo_username
- name: FORGEJO_TOKEN
valueFrom:
secretKeyRef:
name: forgejo-access-token
key: forgejo_token
command: ["/bin/bash", "-c"]
args:
- |
#! /bin/bash
apt -qq update
apt -qq install git wget -y
if [[ "$(uname -m)" == "x86_64" ]]; then
wget https://github.com/mikefarah/yq/releases/download/v4.44.3/yq_linux_amd64
install yq_linux_amd64 /usr/local/bin/yq
rm yq_linux_amd64
else
wget https://github.com/mikefarah/yq/releases/download/v4.44.3/yq_linux_arm64
install yq_linux_arm64 /usr/local/bin/yq
rm yq_linux_arm64
fi
git config --global user.email "bot@bots.de"
git config --global user.name "bot"
git clone https://${FORGEJO_USER}:${FORGEJO_TOKEN}@{{{ .Env.DOMAIN_GITEA }}}/giteaAdmin/edfbuilder.git
cd edfbuilder
yq eval '.configs.cm."oidc.config" = "name: Keycloak\nissuer: https://{{{ .Env.DOMAIN }}}/keycloak/realms/cnoe\nclientID: argocd\nclientSecret: $auth-generic-oauth-secret:client_secret\nrequestedScopes: [\"openid\", \"profile\", \"email\", \"groups\"]"' -i stacks/core/argocd/values.yaml
git add stacks/core/argocd/values.yaml
git commit -m "adds Forgejo SSO config"
git push
backoffLimit: 99

View file

@ -1,11 +1,9 @@
global:
domain: {{{ .Env.DOMAIN }}}
domain: {{{ .Env.DOMAIN_ARGOCD }}}
configs:
params:
server.insecure: true
server.basehref: /argocd
server.rootpath: /argocd
cm:
application.resourceTrackingMethod: annotation
timeout.reconciliation: 60s
@ -21,7 +19,7 @@ configs:
clusters:
- "*"
accounts.provider-argocd: apiKey
url: https://{{{ .Env.DOMAIN }}}/argocd
url: https://{{{ .Env.DOMAIN_ARGOCD }}}
rbac:
policy.csv: 'g, provider-argocd, role:admin'

View file

@ -1,26 +0,0 @@
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: forgejo-access-token
namespace: gitea
spec:
secretStoreRef:
name: gitea
kind: ClusterSecretStore
refreshInterval: "0"
target:
name: forgejo-access-token
template:
engineVersion: v2
data:
forgejo_username: "{{.FORGEJO_ACCESS_USERNAME}}"
forgejo_token: "{{.FORGEJO_ACCESS_TOKEN}}"
data:
- secretKey: FORGEJO_ACCESS_USERNAME
remoteRef:
key: forgejo-access-token
property: username
- secretKey: FORGEJO_ACCESS_TOKEN
remoteRef:
key: forgejo-access-token
property: token

View file

@ -1,26 +0,0 @@
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: auth-generic-oauth-secret
namespace: gitea
spec:
secretStoreRef:
name: keycloak
kind: ClusterSecretStore
refreshInterval: "0"
target:
name: auth-generic-oauth-secret
template:
engineVersion: v2
data:
key: "{{.FORGEJO_CLIENT_ID}}"
secret: "{{.FORGEJO_CLIENT_SECRET}}"
data:
- secretKey: FORGEJO_CLIENT_ID
remoteRef:
key: keycloak-clients
property: FORGEJO_CLIENT_ID
- secretKey: FORGEJO_CLIENT_SECRET
remoteRef:
key: keycloak-clients
property: FORGEJO_CLIENT_SECRET

View file

@ -1,76 +0,0 @@
---
apiVersion: batch/v1
kind: Job
metadata:
name: forgejo-config
namespace: gitea
spec:
template:
metadata:
generateName: forgejo-config-
spec:
restartPolicy: OnFailure
containers:
- name: push
image: docker.io/library/ubuntu:22.04
env:
- name: FORGEJO_USER
valueFrom:
secretKeyRef:
name: forgejo-access-token
key: forgejo_username
- name: FORGEJO_TOKEN
valueFrom:
secretKeyRef:
name: forgejo-access-token
key: forgejo_token
command: ["/bin/bash", "-c"]
args:
- |
#! /bin/bash
apt -qq update
apt -qq install git wget -y
if [[ "$(uname -m)" == "x86_64" ]]; then
wget https://github.com/mikefarah/yq/releases/download/v4.44.3/yq_linux_amd64
install yq_linux_amd64 /usr/local/bin/yq
rm yq_linux_amd64
else
wget https://github.com/mikefarah/yq/releases/download/v4.44.3/yq_linux_arm64
install yq_linux_arm64 /usr/local/bin/yq
rm yq_linux_arm64
fi
git config --global user.email "bot@bots.de"
git config --global user.name "giteaAdmin"
git clone https://${FORGEJO_USER}:${FORGEJO_TOKEN}@{{{ .Env.DOMAIN_GITEA }}}/giteaAdmin/edfbuilder.git
cd edfbuilder
yq eval ".gitea.oauth = [
{
\"name\": \"Keycloak\",
\"provider\": \"openidConnect\",
\"existingSecret\": \"auth-generic-oauth-secret\",
\"autoDiscoverUrl\": \"https://{{{ .Env.DOMAIN }}}/keycloak/realms/cnoe/.well-known/openid-configuration\"
}
] |
(.gitea.oauth[] | .name) |= (. style=\"single\")
|
(.gitea.oauth[] | .provider) |= (. style=\"single\")
|
(.gitea.oauth[] | .existingSecret) |= (. style=\"single\")
|
(.gitea.oauth[] | .autoDiscoverUrl) |= (. style=\"single\")
" -i stacks/core/forgejo/values.yaml
yq eval '.gitea.config.oauth2_client =
{
"ENABLE_AUTO_REGISTRATION" : true,
"ACCOUNT_LINKING" : "auto"
}
' -i stacks/core/forgejo/values.yaml
git add stacks/core/forgejo/values.yaml
git commit -m "adds Forgejo SSO config"
git push
backoffLimit: 99

View file

@ -1,36 +0,0 @@
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
metadata:
name: forgejo-admin-password-generator
namespace: gitea
spec:
length: 36
digits: 5
symbols: 5
symbolCharacters: "/-+"
noUpper: false
allowRepeat: true
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: forgejo-admin-password-generator
namespace: gitea
spec:
refreshInterval: "0"
target:
name: gitea-credential
template:
engineVersion: v2
data:
username: giteaAdmin
password: "{{.INITIAL_ADMIN_PASSWORD}}"
dataFrom:
- sourceRef:
generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
name: forgejo-admin-password-generator
rewrite:
- transform:
template: "INITIAL_ADMIN_PASSWORD"

View file

@ -1,67 +0,0 @@
redis-cluster:
enabled: true
postgresql:
enabled: false
postgresql-ha:
enabled: false
persistence:
enabled: true
size: 5Gi
test:
enabled: false
gitea:
admin:
existingSecret: gitea-credential
config:
service:
DISABLE_REGISTRATION: true
other:
SHOW_FOOTER_VERSION: false
SHOW_FOOTER_TEMPLATE_LOAD_TIME: false
database:
DB_TYPE: sqlite3
session:
PROVIDER: memory
cache:
ADAPTER: memory
queue:
TYPE: level
server:
DOMAIN: '{{{ .Env.DOMAIN_GITEA }}}'
ROOT_URL: 'https://{{{ .Env.DOMAIN_GITEA }}}:443'
mailer:
ENABLED: true
FROM: forgejo@{{{ .Env.DOMAIN_GITEA }}}
PROTOCOL: smtp
SMTP_ADDR: mailhog.mailhog.svc.cluster.local
SMTP_PORT: 1025
service:
ssh:
type: NodePort
nodePort: 32222
externalTrafficPolicy: Local
image:
pullPolicy: "IfNotPresent"
# Overrides the image tag whose default is the chart appVersion.
#tag: "8.0.3"
# Adds -rootless suffix to image name
rootless: true
forgejo:
runner:
enabled: true
image:
tag: latest
# replicas: 3
config:
runner:
labels:
- docker:docker://node:16-bullseye
- self-hosted:docker://ghcr.io/catthehacker/ubuntu:act-22.04
- ubuntu-22.04:docker://ghcr.io/catthehacker/ubuntu:act-22.04
- ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-22.04

View file

@ -1,31 +0,0 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
name: argo-workflows-ingress
namespace: argo
spec:
ingressClassName: nginx
rules:
- host: localhost
http:
paths:
- backend:
service:
name: argo-server
port:
name: web
path: /argo-workflows(/|$)(.*)
pathType: ImplementationSpecific
- host: {{{ .Env.DOMAIN }}}
http:
paths:
- backend:
service:
name: argo-server
port:
name: web
path: /argo-workflows(/|$)(.*)
pathType: ImplementationSpecific

View file

@ -1,28 +0,0 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: backstage
namespace: backstage
spec:
ingressClassName: nginx
rules:
- host: localhost
http:
paths:
- backend:
service:
name: backstage
port:
name: http
path: /
pathType: Prefix
- host: {{{ .Env.DOMAIN }}}
http:
paths:
- backend:
service:
name: backstage
port:
name: http
path: /
pathType: Prefix

View file

@ -1,28 +0,0 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: keycloak-ingress-localhost
namespace: keycloak
spec:
ingressClassName: nginx
rules:
- host: localhost
http:
paths:
- backend:
service:
name: keycloak
port:
name: http
path: /keycloak
pathType: ImplementationSpecific
- host: {{{ .Env.DOMAIN }}}
http:
paths:
- backend:
service:
name: keycloak
port:
name: http
path: /keycloak
pathType: ImplementationSpecific

View file

@ -1,18 +0,0 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: kube-prometheus-stack-grafana
namespace: monitoring
spec:
ingressClassName: nginx
rules:
- host: {{{ .Env.DOMAIN }}}
http:
paths:
- backend:
service:
name: kube-prometheus-stack-grafana
port:
number: 80
path: /grafana
pathType: Prefix

View file

@ -1,18 +0,0 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: mailhog
namespace: mailhog
spec:
ingressClassName: nginx
rules:
- host: {{{ .Env.DOMAIN }}}
http:
paths:
- backend:
service:
name: mailhog
port:
number: 8025
path: /mailhog
pathType: Prefix

View file

@ -1,24 +0,0 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: openbao
namespace: openbao
{{{ if eq .Env.CLUSTER_TYPE "osc" }}}
annotations:
dns.gardener.cloud/class: garden
dns.gardener.cloud/dnsnames: openbao.{{{ .Env.DOMAIN }}}
dns.gardener.cloud/ttl: "600"
{{{ end }}}
spec:
ingressClassName: nginx
rules:
- host: openbao.{{{ .Env.DOMAIN }}}
http:
paths:
- backend:
service:
name: openbao
port:
number: 8200
path: /
pathType: Prefix

View file

@ -1,49 +0,0 @@
controller:
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
ingressClassResource:
name: nginx
# added for idpbuilder
allowSnippetAnnotations: true
# added for idpbuilder
config:
proxy-buffer-size: 32k
use-forwarded-headers: "true"
# monitoring nginx
metrics:
enabled: true
serviceMonitor:
additionalLabels:
release: "ingress-nginx"
enabled: true
{{{ if eq .Env.CLUSTER_TYPE "kind" }}}
hostPort:
enabled: true
terminationGracePeriodSeconds: 0
service:
type: NodePort
nodeSelector:
ingress-ready: "true"
tolerations:
- key: "node-role.kubernetes.io/master"
operator: "Equal"
effect: "NoSchedule"
- key: "node-role.kubernetes.io/control-plane"
operator: "Equal"
effect: "NoSchedule"
publishService:
enabled: false
extraArgs:
publish-status-address: localhost
# added for idpbuilder
enable-ssl-passthrough: ""
{{{ end }}}

View file

@ -14,9 +14,11 @@ spec:
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: -1
destination:
server: "https://kubernetes.default.svc"
source:
repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/core/forgejo-runner"
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/forgejo/forgejo-runner"

View file

@ -39,7 +39,7 @@ spec:
--token ${RUNNER_SECRET} \
--name ${RUNNER_NAME} \
--instance ${FORGEJO_INSTANCE_URL} \
--labels docker:docker://node:20-bookworm,ubuntu-22.04:docker://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/devfw-cicd/catthehackerubuntu:act-22.04,ubuntu-latest:docker://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/devfw-cicd/catthehackerubuntu:act-22.04
--labels docker:docker://node:20-bookworm,ubuntu-22.04:docker://edp.buildth.ing/devfw-cicd/catthehackerubuntu:act-22.04,ubuntu-latest:docker://edp.buildth.ing/devfw-cicd/catthehackerubuntu:act-22.04
env:
- name: RUNNER_NAME
valueFrom:

View file

@ -0,0 +1,38 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: forgejo-server
namespace: argocd
labels:
env: dev
spec:
project: default
syncPolicy:
automated:
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: -1
destination:
name: in-cluster
namespace: gitea
sources:
- repoURL: https://edp.buildth.ing/DevFW-CICD/forgejo-helm.git
path: .
# first check out the desired version (example v9.0.0): https://code.forgejo.org/forgejo-helm/forgejo-helm/src/tag/v9.0.0/Chart.yaml
# (note that the chart version is not the same as the forgejo application version, which is specified in the above Chart.yaml file)
# then use the devops pipeline and select development, forgejo and the desired version (example v9.0.0):
# https://edp.buildth.ing/DevFW-CICD/devops-pipelines/actions?workflow=update-helm-depends.yaml&actor=0&status=0
# finally update the desired version here and include "-depends", it is created by the devops pipeline.
# why do we have an added "-depends" tag? it resolves rate limitings when downloading helm OCI dependencies
targetRevision: v12.0.0-depends
helm:
valueFiles:
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/forgejo/forgejo-server/values.yaml
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
ref: values
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/forgejo/forgejo-server/manifests"

View file

@ -4,12 +4,13 @@ metadata:
annotations:
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: 512m
cert-manager.io/cluster-issuer: main
{{{ if eq .Env.CLUSTER_TYPE "osc" }}}
dns.gardener.cloud/class: garden
dns.gardener.cloud/dnsnames: {{{ .Env.DOMAIN_GITEA }}}
dns.gardener.cloud/ttl: "600"
{{{ end }}}
name: forgejo
name: forgejo-server
namespace: gitea
spec:
ingressClassName: nginx
@ -19,7 +20,7 @@ spec:
paths:
- backend:
service:
name: forgejo-http
name: forgejo-server-http
port:
number: 3000
path: /

View file

@ -0,0 +1,180 @@
# We use recreate to make sure only one instance with one version is running, because Forgejo might break or data gets inconsistant.
strategy:
type: Recreate
redis-cluster:
enabled: false
redis:
enabled: false
postgresql:
enabled: false
postgresql-ha:
enabled: false
persistence:
enabled: true
size: 200Gi
annotations:
everest.io/crypt-key-id: {{{ .Env.PVC_KMS_KEY_ID }}}
test:
enabled: false
deployment:
env:
- name: SSL_CERT_DIR
value: /etc/ssl/forgejo
extraVolumeMounts:
- mountPath: /etc/ssl/forgejo
name: custom-database-certs-volume
readOnly: true
extraVolumes:
- name: custom-database-certs-volume
secret:
secretName: custom-database-certs
gitea:
additionalConfigFromEnvs:
- name: FORGEJO__storage__MINIO_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: forgejo-cloud-credentials
key: access-key
- name: FORGEJO__storage__MINIO_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: forgejo-cloud-credentials
key: secret-key
- name: FORGEJO__queue__CONN_STR
valueFrom:
secretKeyRef:
name: redis-forgejo-cloud-credentials
key: connection-string
- name: FORGEJO__session__PROVIDER_CONFIG
valueFrom:
secretKeyRef:
name: redis-forgejo-cloud-credentials
key: connection-string
- name: FORGEJO__cache__HOST
valueFrom:
secretKeyRef:
name: redis-forgejo-cloud-credentials
key: connection-string
- name: FORGEJO__database__HOST
valueFrom:
secretKeyRef:
name: postgres-forgejo-cloud-credentials
key: host_port
- name: FORGEJO__database__NAME
valueFrom:
secretKeyRef:
name: postgres-forgejo-cloud-credentials
key: database
- name: FORGEJO__database__USER
valueFrom:
secretKeyRef:
name: postgres-forgejo-cloud-credentials
key: username
- name: FORGEJO__database__PASSWD
valueFrom:
secretKeyRef:
name: postgres-forgejo-cloud-credentials
key: password
- name: FORGEJO__indexer__ISSUE_INDEXER_CONN_STR
valueFrom:
secretKeyRef:
name: elasticsearch-cloud-credentials
key: connection-string
- name: FORGEJO__mailer__PASSWD
valueFrom:
secretKeyRef:
name: email-user-credentials
key: connection-string
admin:
existingSecret: gitea-credential
config:
APP_NAME: 'EDP'
APP_SLOGAN: 'Build your thing in minutes'
indexer:
ISSUE_INDEXER_ENABLED: true
ISSUE_INDEXER_TYPE: elasticsearch
# TODO next
REPO_INDEXER_ENABLED: false
# REPO_INDEXER_TYPE: meilisearch # not yet working
storage:
MINIO_ENDPOINT: obs.eu-de.otc.t-systems.com:443
STORAGE_TYPE: minio
MINIO_LOCATION: eu-de
MINIO_BUCKET: edp-forgejo-{{{ .Env.CLUSTER_ENVIRONMENT }}}
MINIO_USE_SSL: true
queue:
TYPE: redis
session:
PROVIDER: redis
cache:
ENABLED: true
ADAPTER: redis
service:
DISABLE_REGISTRATION: true
other:
SHOW_FOOTER_VERSION: false
SHOW_FOOTER_TEMPLATE_LOAD_TIME: false
database:
DB_TYPE: postgres
SSL_MODE: verify-ca
server:
DOMAIN: '{{{ .Env.DOMAIN_GITEA }}}'
ROOT_URL: 'https://{{{ .Env.DOMAIN_GITEA }}}:443'
mailer:
ENABLED: true
USER: ipcei-cis-devfw@mms-support.de
PROTOCOL: smtps
FROM: '"IPCEI CIS DevFW" <ipcei-cis-devfw@mms-support.de>'
SMTP_ADDR: mail.mms-support.de
SMTP_PORT: 465
service:
ssh:
type: LoadBalancer
nodePort: 32222
externalTrafficPolicy: Cluster
annotations:
kubernetes.io/elb.id: {{{ .Env.LOADBALANCER_ID }}}
image:
pullPolicy: "IfNotPresent"
# Overrides the image tag whose default is the chart appVersion.
#tag: "8.0.3"
# Adds -rootless suffix to image name
# rootless: true
fullOverride: forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/devfw-cicd/edp-forgejo:prerelease-v11-0-1-rootless
forgejo:
runner:
enabled: true
image:
tag: latest
# replicas: 3
config:
runner:
labels:
- docker:docker://node:16-bullseye
- self-hosted:docker://ghcr.io/catthehacker/ubuntu:act-22.04
- ubuntu-22.04:docker://ghcr.io/catthehacker/ubuntu:act-22.04
- ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-22.04

View file

@ -1,126 +0,0 @@
# Local Backup with Velero and Minio
This is example is adapted from the original icpbuilder stack.
The two significant changes from the original were made:
* disabled `hostPath` mount to persist backups within kind, since backups do not work sufficiently in this example due to PVC issues, see below.
* renamed `minio` namespace to `minio-backup` so it does not collide with other minio examples.
Within kind, it can only backup kubernetes objects. Data from PVC's is skipped, see below why.
[Velero](https://velero.io/) requires some compatible storage providers as its backup target. This local installation uses [MinIO](https://min.io/) as an example.
However, MinIO is not officially supported by Velero but works due to S3 compatibility.
The current setup does NOT persist backups but stores them in MinIO's PVCs. Proper backups should configure external storage, see [Supported Providers](https://velero.io/docs/main/supported-providers/).
## Installation
The stack is installed as part of the `./example.sh` run.
In order to persist a local backup you have to mount a local directory within `main.go`:
```yaml
nodes:
- role: control-plane
extraMounts:
- hostPath: /some/path/backup # replace with your own path
containerPath: /backup
```
Kind creates the directory on the host but you might have to adjust the permissions, otherwise the minio pod fails to start.
## Using it
After the installation velero and minio should be visible in ArgoCD.
During the installation credentials for minio are generated and shared with velero. You can access them manually:
```bash
kubectl -n minio-backup get secret root-creds -o go-template='{{ range $key, $value := .data }}{{ printf "%s: %s\n" $key ($value | base64decode) }}{{ end }}'
# example output
# rootPassword: aKKZzLnyry6OYZts17vMTf32H5ghFL4WYgu6bHujm
# rootUser: ge8019yksArb7BICt3MLY9
```
A bucket in minio was created and velero uses it for its backups by default, see helm `values.yaml` files.
### Backup and Restore
Backups and subsequent restores can be scheduled by either using the velero cli or by creating CRD objects.
Check the `./demo` directory for equivalent CRD manifests.
Create a backup of the backstage namespace, see `schedule` task for more permanent setups:
```shell
velero backup create backstage-backup --include-namespaces backstage
```
There are more options to create a fine granular backup and to set the backup storage.
See velero's docs for details.
Check the backup with:
```shell
velero backup get
```
To get more details on the backup you need to be able to connect to velero's backup storage, i.e. minio.
Using `kubefwd` here helps a lot (this is not necessary for restore).
```shell
kubefwd services -n minio-backup
```
More details with `describe` and `logs`:
```shell
velero backup describe backstage-backup --details
velero backup logs backstage-backup
```
Restore the backup into the original namespace, you might want to delete the existing namespace beforehand:
```shell
kubectl delete namespace backstage
velero restore create --from-backup backstage-backup
```
When restoring, velero does not replace existing objects in the backup target.
ArgoCD does pickup on the changes and also validates that the backup is in sync.
## Issues with Persistent Volumes
Velero has no issue to backup kubernetes objects like Deployments, ConfigMaps, etc. since they are just yaml/json definitions.
Volumes containing data are, however, more complex. The preferred type of backup are kubernetes' VolumeSnapshots as they consistently store the state
of a volume at a given point in time in an atomic action. Those snapshots live within the cluster and are subsequently downloaded into one of velero's
storage backends for safekeeping.
However, VolumeSnapshots are only possible on storage backends that support them via CSI drivers.
Backends like `nfs` or `hostPath` do NOT support them. Here, velero uses an alternative method
called [File System Backups](https://velero.io/docs/main/file-system-backup/).
In essence, this a simple copy operation based on the file system. Even though
this uses more sophisticated tooling under the hood, i.e. kopia, it is not
possible to create a backup in an atomic transaction. Thus, the resulting backup
might be inconsistent.
Furthermore, for file system backups to work velero installs a node-agent as a
DaemonSet on each Kubernetes node. The agent is aware of the node's internal
storage and accesses the directories on the host directly to copy the files.
This is not supported for hostPath volumes as they mount an arbitrary path
on the host. In theory, a backup is possible but due extra config and security
considerations intentionally skipped. Kind's local-path provider storage uses
a hostPath and is thus not supported for any kind of backup.
## TODOs
* The MinIO -backup installation is only intended as an example and must either
be configured properly or replaced.
* The current example does not automatically schedule backups.
* velero chart must be properly parameterized

View file

@ -1,9 +0,0 @@
# velero backup create backstage-backup --include-namespaces backstage
apiVersion: velero.io/v1
kind: Backup
metadata:
name: backstage-backup
namespace: velero
spec:
includedNamespaces:
- 'backstage'

View file

@ -1,10 +0,0 @@
# velero restore create --from-backup backstage-backup
apiVersion: velero.io/v1
kind: Restore
metadata:
name: backstage-backup
namespace: velero
spec:
backupName: backstage-backup
includedNamespaces:
- 'backstage'

View file

@ -1,17 +0,0 @@
replicas: 1
mode: standalone
resources:
requests:
memory: 128Mi
persistence:
enabled: true
storageClass: standard
size: 512Mi
# volumeName: backup # re-enable this to mount a local host path, see minio-pv.yaml
buckets:
- name: edfbuilder-backups
existingSecret: root-creds

View file

@ -1,13 +0,0 @@
# re-enable this config to mount a local host path, see `../helm/values.yaml`
# apiVersion: v1
# kind: PersistentVolume
# metadata:
# name: backup
# spec:
# storageClassName: standard
# accessModes:
# - ReadWriteOnce
# capacity:
# storage: 512Mi
# hostPath:
# path: /backup

View file

@ -1,154 +0,0 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: secret-sync
namespace: minio-backup
annotations:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/sync-wave: "-20"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: secret-sync
namespace: minio-backup
annotations:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/sync-wave: "-20"
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: secret-sync
namespace: minio-backup
annotations:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/sync-wave: "-20"
subjects:
- kind: ServiceAccount
name: secret-sync
namespace: minio-backup
roleRef:
kind: Role
name: secret-sync
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: secret-sync
namespace: velero
annotations:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/sync-wave: "-20"
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: secret-sync
namespace: velero
annotations:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/sync-wave: "-20"
subjects:
- kind: ServiceAccount
name: secret-sync
namespace: minio-backup
roleRef:
kind: Role
name: secret-sync
apiGroup: rbac.authorization.k8s.io
---
apiVersion: batch/v1
kind: Job
metadata:
name: secret-sync
namespace: minio-backup
annotations:
argocd.argoproj.io/hook: PostSync
spec:
template:
metadata:
generateName: secret-sync
spec:
serviceAccountName: secret-sync
restartPolicy: Never
containers:
- name: kubectl
image: docker.io/bitnami/kubectl
command: ["/bin/bash", "-c"]
args:
- |
set -e
kubectl get secrets -n minio-backup root-creds -o json > /tmp/secret
ACCESS=$(jq -r '.data.rootUser | @base64d' /tmp/secret)
SECRET=$(jq -r '.data.rootPassword | @base64d' /tmp/secret)
echo \
"apiVersion: v1
kind: Secret
metadata:
name: secret-key
namespace: velero
type: Opaque
stringData:
aws: |
[default]
aws_access_key_id=${ACCESS}
aws_secret_access_key=${SECRET}
" > /tmp/secret.yaml
kubectl apply -f /tmp/secret.yaml
---
apiVersion: batch/v1
kind: Job
metadata:
name: minio-root-creds
namespace: minio-backup
annotations:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/sync-wave: "-10"
spec:
template:
metadata:
generateName: minio-root-creds
spec:
serviceAccountName: secret-sync
restartPolicy: Never
containers:
- name: kubectl
image: docker.io/bitnami/kubectl
command: ["/bin/bash", "-c"]
args:
- |
kubectl get secrets -n minio-backup root-creds
if [ $? -eq 0 ]; then
exit 0
fi
set -e
NAME=$(openssl rand -base64 24)
PASS=$(openssl rand -base64 36)
echo \
"apiVersion: v1
kind: Secret
metadata:
name: root-creds
namespace: minio-backup
type: Opaque
stringData:
rootUser: "${NAME}"
rootPassword: "${PASS}"
" > /tmp/secret.yaml
kubectl apply -f /tmp/secret.yaml

View file

@ -1,25 +0,0 @@
resources:
requests:
memory: 128Mi
initContainers:
- name: velero-plugin-for-aws
image: velero/velero-plugin-for-aws:v1.11.0
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /target
name: plugins
# snapshotsEnabled: false # create snapshot crd?
# deployNodeAgent: true # install node agent as daemonset for file system backups?
configuration:
# defaultVolumesToFsBackup: true # backup pod volumes via fsb without explicit annotation?
backupStorageLocation:
- name: default
provider: aws
bucket: edfbuilder-backups
credential:
name: secret-key # this key is created within the minio-backup/secret-sync and injected into the velero namespace
key: aws
config:
region: minio
s3Url: http://minio.minio-backup.svc.cluster.local:9000 # internal resolution, external access for velero cli via fwd
s3ForcePathStyle: "true"

View file

@ -1,78 +0,0 @@
alloy:
create: false
name: alloy-config
key: config.alloy
uiPathPrefix: "/alloy"
configMap:
content: |-
logging {
level = "info"
format = "logfmt"
}
loki.write "local_loki" {
endpoint {
url = "http://loki-loki-distributed-gateway/loki/api/v1/push"
}
}
discovery.kubernetes "pod" {
role = "pod"
}
discovery.kubernetes "nodes" {
role = "node"
}
discovery.kubernetes "services" {
role = "service"
}
discovery.kubernetes "endpoints" {
role = "endpoints"
}
discovery.kubernetes "endpointslices" {
role = "endpointslice"
}
discovery.kubernetes "ingresses" {
role = "ingress"
}
discovery.relabel "pod_logs" {
targets = discovery.kubernetes.pod.targets
rule {
source_labels = ["__meta_kubernetes_namespace"]
action = "replace"
target_label = "namespace"
}
rule {
source_labels = ["__meta_kubernetes_pod_name"]
action = "replace"
target_label = "pod"
}
rule {
source_labels = ["__meta_kubernetes_pod_node_name"]
action = "replace"
target_label = "node"
}
rule {
source_labels = ["__meta_kubernetes_pod_container_name"]
action = "replace"
target_label = "container"
}
}
loki.source.kubernetes "all_pod_logs" {
targets = discovery.relabel.pod_logs.output
forward_to = [loki.write.local_loki.receiver]
}

View file

@ -1,25 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: grafana-dashboards
namespace: argocd
labels:
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/monitoring/kube-prometheus/dashboards"
destination:
server: "https://kubernetes.default.svc"
namespace: monitoring
syncPolicy:
syncOptions:
- CreateNamespace=true
automated:
selfHeal: true
retry:
limit: -1

View file

@ -1,39 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: kube-prometheus-stack
namespace: argocd
labels:
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
syncPolicy:
automated:
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true # do not copy metdata, since (because of its large size) it can lead to sync failure
retry:
limit: -1
backoff:
duration: 15s
factor: 1
maxDuration: 15s
destination:
name: in-cluster
namespace: monitoring
sources:
- repoURL: https://github.com/prometheus-community/helm-charts
path: charts/kube-prometheus-stack
targetRevision: HEAD
helm:
valueFiles:
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/monitoring/kube-prometheus/values.yaml
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
ref: values
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/monitoring/kube-prometheus/manifests"

View file

@ -1,268 +0,0 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-dashboard-1
labels:
grafana_dashboard: "1"
data:
k8s-dashboard-01.json: |
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 1,
"links": [
],
"panels": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 0
},
"id": 5,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"expr": "{app=\"crossplane\"}",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: App crossplane",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 8
},
"id": 4,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"expr": "{app=\"argo-server\"}",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: App argo-server",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 16
},
"id": 3,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"expr": "{app=\"forgejo\"}",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: App forgejo",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 24
},
"id": 2,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"expr": "{app=\"backstage\"}",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: App backstage",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 32
},
"id": 1,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"expr": "{app=\"shoot-control-plane\"}",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: App shoot-control-plane",
"type": "logs"
}
],
"preload": false,
"schemaVersion": 40,
"tags": [
],
"templating": {
"list": [
]
},
"time": {
"from": "now-6h",
"to": "now"
},
"timepicker": {
},
"timezone": "browser",
"title": "Loki Logs: Apps",
"uid": "ee4iuluru756of",
"version": 2,
"weekStart": ""
}

View file

@ -1,845 +0,0 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-dashboard-2
labels:
grafana_dashboard: "1"
data:
k8s-dashboard-02.json: |
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 30,
"links": [
],
"panels": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 0
},
"id": 19,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"server\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component server",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 8
},
"id": 17,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"repo-server\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component repo-server",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 16
},
"id": 16,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"redis\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component redis",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 24
},
"id": 15,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"query-frontend\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component query-frontend",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 32
},
"id": 14,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"querier\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component querier",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 40
},
"id": 13,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"prometheus-operator-webhook\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component prometheus-operator-webhook",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 48
},
"id": 12,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"prometheus-operator\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component prometheus-operator",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 56
},
"id": 11,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"metrics\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component metrics",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 64
},
"id": 10,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"kube-scheduler\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component kube-scheduler",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 72
},
"id": 9,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"kube-controller-manager\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component kube-controller-manager",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 80
},
"id": 8,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"kube-apiserver\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component kube-apiserver",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 88
},
"id": 7,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"ingester\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component ingester",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 96
},
"id": 6,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"gateway\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component gateway",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 104
},
"id": 5,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"etcd\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component etcd",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 112
},
"id": 4,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"distributor\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component distributor",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 120
},
"id": 3,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"controller\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component controller",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 128
},
"id": 2,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"cloud-infrastructure-controller\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component cloud-infrastructure-controller",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 136
},
"id": 1,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{component=\"applicationset-controller\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Component application-controller",
"type": "logs"
}
],
"preload": false,
"schemaVersion": 40,
"tags": [
],
"templating": {
"list": [
]
},
"time": {
"from": "now-6h",
"to": "now"
},
"timepicker": {
},
"timezone": "browser",
"title": "Loki Logs: Components",
"uid": "ae4zuyp1kui9sc",
"version": 2,
"weekStart": ""
}

View file

@ -1,537 +0,0 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-dashboard-3
labels:
grafana_dashboard: "1"
data:
k8s-dashboard-03.json: |
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 31,
"links": [
],
"panels": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 0
},
"id": 11,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"repo-server\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container repo-server",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 8
},
"id": 10,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"alloy\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container alloy",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 16
},
"id": 9,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"prometheus\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container prometheus",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 24
},
"id": 8,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"postgres\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container postgres",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 32
},
"id": 7,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"kube-prometheus-stack\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container kube-prometheus-stack",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 40
},
"id": 6,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"keycloak\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container keycloak",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 48
},
"id": 5,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"grafana\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container grafana",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 56
},
"id": 4,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"forgejo\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container forgejo",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 64
},
"id": 3,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"crossplane\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container crossplane",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 72
},
"id": 2,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"backstage\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container backstage",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"fieldConfig": {
"defaults": {
},
"overrides": [
]
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 80
},
"id": 1,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": false,
"showTime": false,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"pluginVersion": "11.3.1",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "P8E80F9AEF21F6940"
},
"editorMode": "builder",
"expr": "{container=\"argo-server\"} |= ``",
"queryType": "range",
"refId": "A"
}
],
"title": "Logs: Container argo-server",
"type": "logs"
}
],
"preload": false,
"schemaVersion": 40,
"tags": [
],
"templating": {
"list": [
]
},
"time": {
"from": "now-6h",
"to": "now"
},
"timepicker": {
},
"timezone": "browser",
"title": "Loki Logs: Container",
"uid": "ee50bcaehmv40e",
"version": 2,
"weekStart": ""
}

View file

@ -1,36 +0,0 @@
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
metadata:
name: grafana-admin-password-generator
namespace: monitoring
spec:
length: 36
digits: 5
symbols: 5
symbolCharacters: "/-+"
noUpper: false
allowRepeat: true
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: grafana-admin-password-generator
namespace: monitoring
spec:
refreshInterval: "0"
target:
name: kube-prometheus-stack-grafana-admin-password
template:
engineVersion: v2
data:
admin-user: admin
admin-password: "{{.INITIAL_ADMIN_PASSWORD}}"
dataFrom:
- sourceRef:
generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
name: grafana-admin-password-generator
rewrite:
- transform:
template: "INITIAL_ADMIN_PASSWORD"

View file

@ -1,21 +0,0 @@
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: auth-generic-oauth-secret
namespace: monitoring
spec:
secretStoreRef:
name: keycloak
kind: ClusterSecretStore
refreshInterval: "0"
target:
name: auth-generic-oauth-secret
template:
engineVersion: v2
data:
client_secret: "{{.GRAFANA_CLIENT_SECRET}}"
data:
- secretKey: GRAFANA_CLIENT_SECRET
remoteRef:
key: keycloak-clients
property: GRAFANA_CLIENT_SECRET

View file

@ -1,71 +0,0 @@
grafana:
namespaceOverride: "monitoring"
admin:
existingSecret: "kube-prometheus-stack-grafana-admin-password"
userKey: admin-user
passwordKey: admin-password
defaultDashboardsTimezone: Europe/Berlin
additionalDataSources:
- name: Loki
type: loki
url: http://loki-loki-distributed-gateway.monitoring:80
# syncPolicy:
# syncOptions:
# - ServerSideApply=true
sidecar:
dashboards:
enabled: true
label: grafana_dashboard
folder: /tmp/dashboards
updateIntervalSeconds: 10
folderAnnotation: grafana_folder
provider:
allowUiUpdates: true
foldersFromFilesStructure: true
grafana.ini:
server:
domain: {{{ .Env.DOMAIN }}}
root_url: "%(protocol)s://%(domain)s/grafana"
serve_from_sub_path: true
auth:
disable_login: true
disable_login_form: true
auth.generic_oauth:
enabled: true
name: Keycloak-OAuth
allow_sign_up: true
use_refresh_token: true
client_id: grafana
client_secret: $__file{/etc/secrets/auth_generic_oauth/client_secret}
scopes: openid email profile offline_access roles
email_attribute_path: email
login_attribute_path: username
name_attribute_path: full_name
auth_url: https://{{{ .Env.DOMAIN }}}/keycloak/realms/cnoe/protocol/openid-connect/auth
token_url: https://{{{ .Env.DOMAIN }}}/keycloak/realms/cnoe/protocol/openid-connect/token
api_url: https://{{{ .Env.DOMAIN }}}/keycloak/realms/cnoe/protocol/openid-connect/userinfo
redirect_uri: http://{{{ .Env.DOMAIN }}}/grafana/login/generic_oauth
role_attribute_path: "contains(groups[*], 'admin') && 'Admin' || contains(groups[*], 'editor') && 'Editor' || 'Viewer'"
extraSecretMounts:
- name: auth-generic-oauth-secret-mount
secretName: auth-generic-oauth-secret
defaultMode: 0440
mountPath: /etc/secrets/auth_generic_oauth
readOnly: true
serviceMonitor:
# If true, a ServiceMonitor CRD is created for a prometheus operator https://github.com/coreos/prometheus-operator
enabled: true
#monitoring nginx
prometheus:
prometheusSpec:
podMonitorSelectorNilUsesHelmValues: false
serviceMonitorSelectorNilUsesHelmValues: false

View file

@ -1,33 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: loki
namespace: argocd
labels:
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
syncPolicy:
automated:
selfHeal: true
syncOptions:
- CreateNamespace=true
destination:
name: in-cluster
namespace: monitoring
sources:
- repoURL: https://github.com/grafana/helm-charts
path: charts/loki-distributed
targetRevision: HEAD
helm:
valueFiles:
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/monitoring/loki/values.yaml
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
ref: values
## consider using the following version, if it works again
#- repoURL: https://github.com/grafana/loki
# path: production/helm/loki

View file

@ -1,7 +0,0 @@
loki:
commonConfig:
replication_factor: 1
auth_enabled: false
# storageConfig:
# filesystem: null

View file

@ -1,31 +1,29 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: velero
name: metrics-server
namespace: argocd
labels:
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
syncPolicy:
automated:
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: -1
destination:
name: in-cluster
namespace: observability
sources:
- repoURL: "https://vmware-tanzu.github.io/helm-charts"
targetRevision: 8.0.0
- chart: metrics-server
repoURL: https://kubernetes-sigs.github.io/metrics-server/
targetRevision: 3.12.2
helm:
releaseName: velero
valueFiles:
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/local-backup/velero/helm/values.yaml
chart: velero
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/observability-client/metrics-server/values.yaml
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
ref: values
destination:
server: "https://kubernetes.default.svc"
namespace: velero
syncPolicy:
syncOptions:
- CreateNamespace=true
automated:
prune: true
selfHeal: true

View file

@ -0,0 +1,4 @@
metrics:
enabled: true
serviceMonitor:
enabled: true

View file

@ -1,12 +1,10 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: alloy
name: vector
namespace: argocd
labels:
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
syncPolicy:
@ -14,16 +12,18 @@ spec:
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: -1
destination:
name: in-cluster
namespace: monitoring
namespace: observability
sources:
- repoURL: https://github.com/grafana/alloy.git
path: operations/helm/charts/alloy
targetRevision: HEAD
- chart: vector
repoURL: https://helm.vector.dev
targetRevision: 0.43.0
helm:
valueFiles:
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/monitoring/alloy/values.yaml
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/observability-client/vector/values.yaml
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
ref: values

View file

@ -0,0 +1,68 @@
# -- Enable deployment of vector
role: Agent
dataDir: /vector-data-dir
resources: {}
args:
- -w
- --config-dir
- /etc/vector/
env:
- name: VECTOR_USER
valueFrom:
secretKeyRef:
name: simple-user-secret
key: username
- name: VECTOR_PASSWORD
valueFrom:
secretKeyRef:
name: simple-user-secret
key: password
containerPorts:
- name: prom-exporter
containerPort: 9090
protocol: TCP
service:
enabled: false
customConfig:
data_dir: /vector-data-dir
api:
enabled: false
address: 0.0.0.0:8686
playground: true
sources:
k8s:
type: kubernetes_logs
internal_metrics:
type: internal_metrics
transforms:
parser:
type: remap
inputs: [k8s]
source: |
._msg = parse_json(.message) ?? .message
del(.message)
# Add the cluster environment to the log event
.cluster_environment = "{{{ .Env.CLUSTER_ENVIRONMENT }}}"
sinks:
vlogs:
type: elasticsearch
inputs: [parser]
endpoints:
- https://{{{ .Env.DOMAIN_O12Y }}}/insert/elasticsearch/
auth:
strategy: basic
user: ${VECTOR_USER}
password: ${VECTOR_PASSWORD}
mode: bulk
api_version: v8
compression: gzip
healthcheck:
enabled: false
request:
headers:
AccountID: "0"
ProjectID: "0"
query:
_msg_field: _msg
_time_field: _time
_stream_fields: cluster_environment,kubernetes.container_name,kubernetes.namespace

View file

@ -1,33 +1,30 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: minio
name: vm-client
namespace: argocd
labels:
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
syncPolicy:
automated:
selfHeal: true
syncOptions:
- CreateNamespace=true
destination:
name: in-cluster
namespace: observability
sources:
- repoURL: "https://charts.min.io"
targetRevision: 5.0.15
- chart: victoria-metrics-k8s-stack
repoURL: https://victoriametrics.github.io/helm-charts/
targetRevision: 0.48.1
helm:
releaseName: minio
valueFiles:
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/local-backup/minio/helm/values.yaml
chart: minio
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/observability-client/vm-client-stack/values.yaml
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
ref: values
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/local-backup/minio/manifests"
destination:
server: "https://kubernetes.default.svc"
namespace: minio-backup
syncPolicy:
syncOptions:
- CreateNamespace=true
automated:
selfHeal: true
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/observability-client/vm-client-stack/manifests"

View file

@ -0,0 +1,9 @@
apiVersion: v1
kind: Secret
metadata:
name: simple-user-secret
namespace: observability
type: Opaque
stringData:
username: simple-user
password: simple-password

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,25 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: grafana-operator
namespace: argocd
labels:
env: dev
spec:
project: default
syncPolicy:
automated:
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
destination:
name: in-cluster
namespace: observability
sources:
- chart: grafana-operator
repoURL: ghcr.io/grafana/helm-charts
targetRevision: v5.18.0
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/observability/grafana-operator/manifests"

View file

@ -0,0 +1,9 @@
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
name: argocd
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
url: "https://raw.githubusercontent.com/argoproj/argo-cd/refs/heads/master/examples/dashboard.json"

View file

@ -0,0 +1,36 @@
apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
name: grafana
labels:
dashboards: "grafana"
spec:
persistentVolumeClaim:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
ingress:
metadata:
annotations:
cert-manager.io/cluster-issuer: main
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
ingressClassName: nginx
rules:
- host: grafana.{{{ .Env.DOMAIN }}}
http:
paths:
- backend:
service:
name: grafana-service
port:
number: 3000
path: /
pathType: Prefix
tls:
- hosts:
- grafana.{{{ .Env.DOMAIN }}}
secretName: grafana-net-tls

View file

@ -0,0 +1,9 @@
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
name: ingress-nginx
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
url: "https://raw.githubusercontent.com/adinhodovic/ingress-nginx-mixin/refs/heads/main/dashboards_out/ingress-nginx-overview.json"

View file

@ -0,0 +1,9 @@
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
name: victoria-logs
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
url: "https://raw.githubusercontent.com/VictoriaMetrics/VictoriaMetrics/refs/heads/master/dashboards/vm/victorialogs.json"

View file

@ -1,37 +1,31 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: openbao
name: o12y
namespace: argocd
labels:
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
syncPolicy:
automated:
selfHeal: false
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
destination:
name: in-cluster
namespace: openbao
namespace: observability
sources:
- repoURL: https://github.com/openbao/openbao-helm.git
path: charts/openbao
targetRevision: HEAD
- chart: victoria-metrics-k8s-stack
repoURL: https://victoriametrics.github.io/helm-charts/
targetRevision: 0.48.1
helm:
valueFiles:
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/ref-implementation/openbao/values.yaml
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/observability/victoria-k8s-stack/values.yaml
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
ref: values
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/ref-implementation/openbao/manifests"
ignoreDifferences:
- group: admissionregistration.k8s.io
kind: MutatingWebhookConfiguration
jqPathExpressions:
- .webhooks[]?.clientConfig.caBundle
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/observability/victoria-k8s-stack/manifests"

View file

@ -0,0 +1,24 @@
apiVersion: operator.victoriametrics.com/v1beta1
kind: VLogs
metadata:
name: victorialogs
namespace: observability
spec:
retentionPeriod: "12"
removePvcAfterDelete: true
storageMetadata:
annotations:
everest.io/crypt-key-id: {{{ .Env.PVC_KMS_KEY_ID }}}
storage:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
resources:
requests:
memory: 500Mi
cpu: 500m
limits:
memory: 10Gi
cpu: 2

View file

@ -0,0 +1,15 @@
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMUser
metadata:
name: simple-user
namespace: observability
spec:
username: simple-user
password: simple-password
targetRefs:
- static:
url: http://vmsingle-o12y:8429
paths: ["/api/v1/write"]
- static:
url: http://vlogs-victorialogs:9428
paths: ["/insert/elasticsearch/.*"]

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,14 @@
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: main
spec:
acme:
email: admin@think-ahead.tech
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: cluster-issuer-account-key
solvers:
- http01:
ingress:
ingressClassName: nginx

View file

@ -0,0 +1,4 @@
crds:
enabled: true
replicaCount: 1

View file

@ -1,7 +1,7 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: forgejo
name: cert-manager
namespace: argocd
labels:
env: dev
@ -12,19 +12,21 @@ spec:
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: -1
destination:
name: in-cluster
namespace: gitea
namespace: cert-manager
sources:
- repoURL: https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW-CICD/forgejo-helm.git
path: .
targetRevision: v12.0.0-depends
- chart: cert-manager
repoURL: https://charts.jetstack.io
targetRevision: v1.17.2
helm:
valueFiles:
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/core/forgejo/values.yaml
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/otc/cert-manager/values.yaml
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
ref: values
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/core/forgejo/manifests"
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/otc/cert-manager/manifests"

View file

@ -12,16 +12,18 @@ spec:
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: -1
destination:
name: in-cluster
namespace: ingress-nginx
sources:
- repoURL: https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW-CICD/ingress-nginx-helm.git
- repoURL: https://edp.buildth.ing/DevFW-CICD/ingress-nginx-helm.git
path: charts/ingress-nginx
targetRevision: helm-chart-4.12.1-depends
helm:
valueFiles:
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/core/ingress-nginx/values.yaml
- $values/{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/otc/ingress-nginx/values.yaml
- repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
ref: values

View file

@ -0,0 +1,31 @@
controller:
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
service:
annotations:
kubernetes.io/elb.class: union
kubernetes.io/elb.port: '80'
kubernetes.io/elb.id: {{{ .Env.LOADBALANCER_ID }}}
kubernetes.io/elb.ip: {{{ .Env.LOADBALANCER_IP }}}
ingressClassResource:
name: nginx
# added for idpbuilder
allowSnippetAnnotations: true
# added for idpbuilder
config:
proxy-buffer-size: 32k
use-forwarded-headers: "true"
# monitoring nginx
metrics:
enabled: true
serviceMonitor:
additionalLabels:
release: "ingress-nginx"
enabled: true

View file

@ -1,25 +1,25 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: mailhog
name: storageclass
namespace: argocd
labels:
env: dev
example: otc
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
destination:
namespace: default
server: "https://kubernetes.default.svc"
source:
repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/ref-implementation/mailhog"
destination:
server: "https://kubernetes.default.svc"
namespace: mailhog
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/otc/storageclass"
project: default
syncPolicy:
syncOptions:
- CreateNamespace=true
automated:
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: -1

View file

@ -0,0 +1,18 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
storageclass.beta.kubernetes.io/is-default-class: "true"
labels:
kubernetes.io/cluster-service: "true"
name: default
parameters:
kubernetes.io/description: ""
kubernetes.io/hw:passthrough: "true"
kubernetes.io/storagetype: BS
kubernetes.io/volumetype: SATA
kubernetes.io/zone: eu-de-02
provisioner: flexvolume-huawei.com/fuxivol
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true

View file

@ -1,146 +0,0 @@
# Reference implementation
This example creates a local version of the CNOE reference implementation.
## Prerequisites
Ensure you have the following tools installed on your computer.
**Required**
- [idpbuilder](https://github.com/cnoe-io/idpbuilder/releases/latest): version `0.3.0` or later
- [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl): version `1.27` or later
- Your computer should have at least 6 GB RAM allocated to Docker. If you are on Docker Desktop, see [this guide](https://docs.docker.com/desktop/settings/mac/).
**Optional**
- AWS credentials: Access Key and secret Key. If you want to create AWS resources in one of examples below.
## Installation
**_NOTE:_**
- If you'd like to run this in your web browser through Codespaces, please follow [the instructions here](./codespaces.md) to install instead.
- _This example assumes that you run the reference implementation with the default port configguration of 8443 for the idpBuilder.
If you happen to configure a different host or port for the idpBuilder, the manifests in the reference example need to be updated
and be configured with the new host and port. you can use the [replace.sh](replace.sh) to change the port as desired prior to applying the manifest as instructed in the command above._
```bash
idpbuilder create --use-path-routing \
--package https://github.com/cnoe-io/stacks//ref-implementation
```
This will take ~6 minutes for everything to come up. To track the progress, you can go to the [ArgoCD UI](https://{{{ .Env.DOMAIN }}}:8443/argocd/applications).
### What was installed?
1. **Argo Workflows** to enable workflow orchestrations.
1. **Backstage** as the UI for software catalog and templating. Source is available [here](https://github.com/cnoe-io/backstage-app).
1. **External Secrets** to generate secrets and coordinate secrets between applications.
1. **Keycloak** as the identity provider for applications.
1. **Spark Operator** to demonstrate an example Spark workload through Backstage.
If you don't want to install a package above, you can remove the ArgoCD Application file corresponding to the package you want to remove.
For example, if you want to remove Spark Operator, you can delete [this file](./spark-operator.yaml).
The only package that cannot be removed this way is Keycloak because other packages rely on it.
#### Accessing UIs
- Argo CD: https://{{{ .Env.DOMAIN }}}:8443/argocd
- Argo Workflows: https://{{{ .Env.DOMAIN }}}:8443/argo-workflows
- Backstage: https://{{{ .Env.DOMAIN }}}:8443/
- Gitea: https://{{{ .Env.DOMAIN }}}:8443/gitea
- Keycloak: https://{{{ .Env.DOMAIN }}}:8443/keycloak/admin/master/console/
# Using it
For this example, we will walk through a few demonstrations. Once applications are ready, go to the [backstage URL](https://{{{ .Env.DOMAIN }}}:8443).
Click on the Sign-In button, you will be asked to log into the Keycloak instance. There are two users set up in this
configuration, and their password can be retrieved with the following command:
```bash
idpbuilder get secrets
```
Use the username **`user1`** and the password value given by `USER_PASSWORD` field to login to the backstage instance.
`user1` is an admin user who has access to everything in the cluster, while `user2` is a regular user with limited access.
Both users use the same password retrieved above.
If you want to create a new user or change existing users:
1. Go to the [Keycloak UI](https://{{{ .Env.DOMAIN }}}:8443/keycloak/admin/master/console/).
Login with the username `cnoe-admin`. Password is the `KEYCLOAK_ADMIN_PASSWORD` field from the command above.
2. Select `cnoe` from the realms drop down menu.
3. Select users tab.
## Basic Deployment
Let's start by deploying a simple application to the cluster through Backstage.
Click on the `Create...` button on the left, then select the `Create a Basic Deployment` template.
![img.png](images/backstage-templates.png)
In the next screen, type `demo` for the name field, then click Review, then Create.
Once steps run, click the Open In Catalog button to go to the entity page.
![img.png](images/basic-template-flow.png)
In the demo entity page, you will notice a ArgoCD overview card associated with this entity.
You can click on the ArgoCD Application name to see more details.
![img.png](images/demo-entity.png)
### What just happened?
1. Backstage created [a git repository](https://{{{ .Env.DOMAIN }}}:8443/gitea/giteaAdmin/demo), then pushed templated contents to it.
2. Backstage created [an ArgoCD Application](https://{{{ .Env.DOMAIN }}}:8443/argocd/applications/argocd/demo?) and pointed it to the git repository.
3. Backstage registered the application as [a component](https://{{{ .Env.DOMAIN }}}:8443/gitea/giteaAdmin/demo/src/branch/main/catalog-info.yaml) in Backstage.
4. ArgoCD deployed the manifests stored in the repo to the cluster.
5. Backstage retrieved application health from ArgoCD API, then displayed it.
![image.png](images/basic-deployment.png)
## Argo Workflows and Spark Operator
In this example, we will deploy a simple Apache Spark job through Argo Workflows.
Click on the `Create...` button on the left, then select the `Basic Argo Workflow with a Spark Job` template.
![img.png](images/backstage-templates-spark.png)
Type `demo2` for the name field, then click create. You will notice that the Backstage templating steps are very similar to the basic example above.
Click on the Open In Catalog button to go to the entity page.
![img.png](images/demo2-entity.png)
Deployment processes are the same as the first example. Instead of deploying a pod, we deployed a workflow to create a Spark job.
In the entity page, there is a card for Argo Workflows, and it should say running or succeeded.
You can click the name in the card to go to the Argo Workflows UI to view more details about this workflow run.
When prompted to log in, click the login button under single sign on. Argo Workflows is configured to use SSO with Keycloak allowing you to login with the same credentials as Backstage login.
Note that Argo Workflows are not usually deployed this way. This is just an example to show you how you can integrate workflows, backstage, and spark.
Back in the entity page, you can view more details about Spark jobs by navigating to the Spark tab.
## Application with cloud resources.
To deploy cloud resources, you can follow any of the instructions below:
- [Cloud resource deployments via Crossplane](../crossplane-integrations/)
- [Cloud resource deployments via Terraform](../terraform-integrations/)
## Notes
- In these examples, we have used the pattern of creating a new repository for every app, then having ArgoCD deploy it.
This is done for convenience and demonstration purposes only. There are alternative actions that you can use.
For example, you can create a PR to an existing repository, create a repository but not deploy them yet, etc.
- If Backstage's pipelining and templating mechanisms is too simple, you can use more advanced workflow engines like Tekton or Argo Workflows.
You can invoke them in Backstage templates, then track progress similar to how it was described above.

View file

@ -1,29 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: argo-workflows
namespace: argocd
labels:
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/ref-implementation/argo-workflows/manifests/dev"
destination:
server: "https://kubernetes.default.svc"
namespace: argo
syncPolicy:
syncOptions:
- CreateNamespace=true
automated:
selfHeal: true
retry:
limit: -1
backoff:
duration: 15s
factor: 1
maxDuration: 15s

View file

@ -1,2 +0,0 @@
resources:
- install.yaml

View file

@ -1,20 +0,0 @@
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: keycloak-oidc
namespace: argo
spec:
secretStoreRef:
name: keycloak
kind: ClusterSecretStore
target:
name: keycloak-oidc
data:
- secretKey: client-id
remoteRef:
key: keycloak-clients
property: ARGO_WORKFLOWS_CLIENT_ID
- secretKey: secret-key
remoteRef:
key: keycloak-clients
property: ARGO_WORKFLOWS_CLIENT_SECRET

View file

@ -1,7 +0,0 @@
resources:
- ../base
- external-secret.yaml
- sa-admin.yaml
patches:
- path: patches/cm-argo-workflows.yaml
- path: patches/deployment-argo-server.yaml

View file

@ -1,26 +0,0 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: workflow-controller-configmap
namespace: argo
data:
config: |
sso:
insecureSkipVerify: true
issuer: https://{{{ .Env.DOMAIN }}}/keycloak/realms/cnoe
clientId:
name: keycloak-oidc
key: client-id
clientSecret:
name: keycloak-oidc
key: secret-key
redirectUrl: https://{{{ .Env.DOMAIN }}}:443/argo-workflows/oauth2/callback
rbac:
enabled: true
scopes:
- openid
- profile
- email
- groups
nodeEvents:
enabled: false

View file

@ -1,30 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: argo-server
namespace: argo
annotations:
argocd.argoproj.io/sync-wave: "20"
spec:
template:
spec:
containers:
- name: argo-server
readinessProbe:
httpGet:
path: /
port: 2746
scheme: HTTP
env:
- name: BASE_HREF
value: "/argo-workflows/"
args:
- server
- --configmap=workflow-controller-configmap
- --auth-mode=client
- --auth-mode=sso
- "--secure=false"
- "--loglevel"
- "info"
- "--log-format"
- "text"

View file

@ -1,32 +0,0 @@
# Used by users in the admin group
# TODO Need to tighten up permissions.
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin
namespace: argo
annotations:
workflows.argoproj.io/rbac-rule: "'admin' in groups"
workflows.argoproj.io/rbac-rule-precedence: "10"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: argo-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin
namespace: argo
---
apiVersion: v1
kind: Secret
metadata:
name: admin.service-account-token
annotations:
kubernetes.io/service-account.name: admin
namespace: argo
type: kubernetes.io/service-account-token

View file

@ -1,27 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: backstage-templates
namespace: argocd
labels:
env: dev
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: https://{{{ .Env.CLIENT_REPO_DOMAIN }}}/{{{ .Env.CLIENT_REPO_ORG_NAME }}}
targetRevision: HEAD
path: "{{{ .Env.CLIENT_REPO_ID }}}/{{{ .Env.DOMAIN }}}/stacks/ref-implementation/backstage-templates/entities"
directory:
exclude: "catalog-info.yaml"
destination:
server: "https://kubernetes.default.svc"
namespace: backstage
syncPolicy:
syncOptions:
- CreateNamespace=true
automated:
selfHeal: true
retry:
limit: -1

View file

@ -1,48 +0,0 @@
apiVersion: backstage.io/v1alpha1
kind: Resource
metadata:
name: ${{values.name}}-bucket
description: Stores things
annotations:
argocd/app-name: ${{values.name | dump}}
spec:
type: s3-bucket
owner: guests
---
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: ${{values.name | dump}}
description: This is for testing purposes
annotations:
backstage.io/techdocs-ref: dir:.
backstage.io/kubernetes-label-selector: 'entity-id=${{values.name}}'
backstage.io/kubernetes-namespace: default
argocd/app-name: ${{values.name | dump}}
links:
- url: https://{{{ .Env.DOMAIN_GITEA }}}:443
title: Repo URL
icon: github
spec:
owner: guests
lifecycle: experimental
type: service
system: ${{values.name | dump}}
dependsOn:
- resource:default/${{values.name}}-bucket
---
apiVersion: backstage.io/v1alpha1
kind: System
metadata:
name: ${{values.name | dump}}
description: An example system for demonstration purposes
annotations:
backstage.io/techdocs-ref: dir:.
links:
- url: https://github.com/cnoe-io/stacks/tree/main/ref-implementation
title: CNOE Repo
icon: github
spec:
owner: guests
lifecycle: experimental
type: service

View file

@ -1,46 +0,0 @@
[![Codespell][codespell-badge]][codespell-link]
[![E2E][e2e-badge]][e2e-link]
[![Go Report Card][report-badge]][report-link]
[![Commit Activity][commit-activity-badge]][commit-activity-link]
# IDP Builder
Internal development platform binary launcher.
> **WORK IN PROGRESS**: This tool is in a pre-release stage and is under active development.
## About
Spin up a complete internal developer platform using industry standard technologies like Kubernetes, Argo, and backstage with only Docker required as a dependency.
This can be useful in several ways:
* Create a single binary which can demonstrate an IDP reference implementation.
* Use within CI to perform integration testing.
* Use as a local development environment for platform engineers.
## Getting Started
Checkout our [documentation website](https://cnoe.io/docs/reference-implementation/installations/idpbuilder) for getting started with idpbuilder.
## Community
- If you have questions or concerns about this tool, please feel free to reach out to us on the [CNCF Slack Channel](https://cloud-native.slack.com/archives/C05TN9WFN5S).
- You can also join our community meetings to meet the team and ask any questions. Checkout [this calendar](https://calendar.google.com/calendar/embed?src=064a2adfce866ccb02e61663a09f99147f22f06374e7a8994066bdc81e066986%40group.calendar.google.com&ctz=America%2FLos_Angeles) for more information.
## Contribution
Checkout the [contribution doc](./CONTRIBUTING.md) for contribution guidelines and more information on how to set up your local environment.
<!-- JUST BADGES & LINKS -->
[codespell-badge]: https://github.com/cnoe-io/idpbuilder/actions/workflows/codespell.yaml/badge.svg
[codespell-link]: https://github.com/cnoe-io/idpbuilder/actions/workflows/codespell.yaml
[e2e-badge]: https://github.com/cnoe-io/idpbuilder/actions/workflows/e2e.yaml/badge.svg
[e2e-link]: https://github.com/cnoe-io/idpbuilder/actions/workflows/e2e.yaml
[report-badge]: https://goreportcard.com/badge/github.com/cnoe-io/idpbuilder
[report-link]: https://goreportcard.com/report/github.com/cnoe-io/idpbuilder
[commit-activity-badge]: https://img.shields.io/github/commit-activity/m/cnoe-io/idpbuilder
[commit-activity-link]: https://github.com/cnoe-io/idpbuilder/pulse

View file

@ -1,16 +0,0 @@
![cnoe logo](./images/cnoe-logo.png)
# Example Basic Application
Thanks for trying out this demo! In this example, we deployed a simple application with a S3 bucket using Crossplane.
### idpbuilder
Checkout the idpbuilder website: https://cnoe.io/docs/reference-implementation/installations/idpbuilder
Checkout the idpbuilder repository: https://github.com/cnoe-io/idpbuilder
## Crossplane
Checkout the Crossplane website: https://www.crossplane.io/

View file

@ -1,3 +0,0 @@
module ${{ values.name }}
go 1.19

View file

@ -1,34 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
targetPort: 80
selector:
app: nginx

View file

@ -1,35 +0,0 @@
{%- if values.awsResources %}
resources:
{%- if 'Bucket' in values.awsResources %}
- ../base/
{%- endif %}
{%- if 'Table' in values.awsResources %}
- ../base/table.yaml
{%- endif %}
{%- endif %}
namespace: default
patches:
- target:
kind: Deployment
patch: |
apiVersion: apps/v1
kind: Deployment
metadata:
name: not-used
labels:
backstage.io/kubernetes-id: ${{values.name}}
spec:
template:
metadata:
labels:
backstage.io/kubernetes-id: ${{values.name}}
- target:
kind: Service
patch: |
apiVersion: apps/v1
kind: Service
metadata:
name: not-used
labels:
backstage.io/kubernetes-id: ${{values.name}}

View file

@ -1,6 +0,0 @@
site_name: 'Argo Spark Example'
nav:
- Home: index.md
- idpBuilder: idpbuilder.md
plugins:
- techdocs-core

View file

@ -1,126 +0,0 @@
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
description: Adds a Go application with AWS resources
name: app-with-aws-resources
title: Add a Go App with AWS resources
spec:
owner: guests
type: service
parameters:
- properties:
name:
title: Application Name
type: string
description: Unique name of the component
ui:autofocus: true
labels:
title: Labels
type: object
additionalProperties:
type: string
description: Labels to apply to the application
ui:autofocus: true
required:
- name
title: Choose your repository location
- description: Configure your bucket
properties:
apiVersion:
default: awsblueprints.io/v1alpha1
description: APIVersion for the resource
type: string
kind:
default: ObjectStorage
description: Kind for the resource
type: string
config:
description: ObjectStorageSpec defines the desired state of ObjectStorage
properties:
resourceConfig:
description: ResourceConfig defines general properties of this AWS resource.
properties:
deletionPolicy:
description: Defaults to Delete
enum:
- Delete
- Orphan
type: string
region:
type: string
providerConfigName:
type: string
default: default
tags:
items:
properties:
key:
type: string
value:
type: string
required:
- key
- value
type: object
type: array
required:
- region
type: object
required:
- resourceConfig
title: Bucket configuration options
type: object
steps:
- id: template
name: Generating component
action: fetch:template
input:
url: ./skeleton
values:
name: ${{parameters.name}}
- action: roadiehq:utils:serialize:yaml
id: serialize
input:
data:
apiVersion: awsblueprints.io/v1alpha1
kind: ${{ parameters.kind }}
metadata:
name: ${{ parameters.name }}
spec: ${{ parameters.config }}
name: serialize
- action: roadiehq:utils:fs:write
id: write
input:
content: ${{ steps['serialize'].output.serialized }}
path: kustomize/base/${{ parameters.name }}.yaml
name: write-to-file
- id: publish
name: Publishing to a gitea git repository
action: publish:gitea
input:
description: This is an example app
# Hard coded value for this demo purposes only.
repoUrl: {{{ .Env.DOMAIN_GITEA }}}:443/?repo=${{parameters.name}}
defaultBranch: main
- id: create-argocd-app
name: Create ArgoCD App
action: cnoe:create-argocd-app
input:
appName: ${{parameters.name}}
appNamespace: default
argoInstance: in-cluster
projectName: default
# necessary until we generate our own cert
repoUrl: https://{{{ .Env.DOMAIN_GITEA }}}:443/giteaAdmin/${{parameters.name}}
path: "kustomize/base"
- id: register
name: Register
action: catalog:register
input:
repoContentsUrl: ${{ steps['publish'].output.repoContentsUrl }}
catalogInfoPath: 'catalog-info.yaml'
output:
links:
- title: Open in catalog
icon: catalog
entityRef: ${{ steps['register'].output.entityRef }}

View file

@ -1,40 +0,0 @@
---
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: ${{values.name | dump}}
description: This is an example Backstage component representing the use of Argo Workflows and Spark Operator.
annotations:
backstage.io/techdocs-ref: dir:.
backstage.io/kubernetes-label-selector: 'entity-id=${{values.name}}'
backstage.io/kubernetes-namespace: argo
argocd/app-name: ${{values.name | dump}}
argo-workflows.cnoe.io/label-selector: env=dev,entity-id=${{values.name}}
argo-workflows.cnoe.io/cluster-name: local
apache-spark.cnoe.io/label-selector: env=dev,entity-id=${{values.name}}
apache-spark.cnoe.io/cluster-name: local
links:
- url: https://{{{ .Env.DOMAIN_GITEA }}}:443
title: Repo URL
icon: github
spec:
owner: guests
lifecycle: experimental
type: service
system: ${{values.name | dump}}
---
apiVersion: backstage.io/v1alpha1
kind: System
metadata:
name: ${{values.name | dump}}
description: An example system for demonstration purposes
annotations:
backstage.io/techdocs-ref: dir:.
links:
- url: https://github.com/cnoe-io/stacks/tree/main/ref-implementation
title: CNOE Repo
icon: github
spec:
owner: guests
lifecycle: experimental
type: service

View file

@ -1,160 +0,0 @@
<!-- markdownlint-disable-next-line MD041 -->
[![Security Status](https://github.com/argoproj/argo-workflows/actions/workflows/snyk.yml/badge.svg?branch=main)](https://github.com/argoproj/argo-workflows/actions/workflows/snyk.yml?query=branch%3Amain)
[![OpenSSF Best Practices](https://bestpractices.coreinfrastructure.org/projects/3830/badge)](https://bestpractices.coreinfrastructure.org/projects/3830)
[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/argoproj/argo-workflows/badge)](https://api.securityscorecards.dev/projects/github.com/argoproj/argo-workflows)
[![FOSSA License Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fargoproj%2Fargo-workflows.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2Fargoproj%2Fargo-workflows?ref=badge_shield)
[![Slack](https://img.shields.io/badge/slack-argoproj-brightgreen.svg?logo=slack)](https://argoproj.github.io/community/join-slack)
[![Twitter Follow](https://img.shields.io/twitter/follow/argoproj?style=social)](https://twitter.com/argoproj)
[![LinkedIn](https://img.shields.io/badge/LinkedIn-argoproj-blue.svg?logo=linkedin)](https://www.linkedin.com/company/argoproj/)
[![Release Version](https://img.shields.io/github/v/release/argoproj/argo-workflows?label=argo-workflows)](https://github.com/argoproj/argo-workflows/releases/latest)
[![Artifact HUB](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/argo-workflows)](https://artifacthub.io/packages/helm/argo/argo-workflows)
## What is Argo Workflows?
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes.
Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).
* Define workflows where each step is a container.
* Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic graph (DAG).
* Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes.
Argo is a [Cloud Native Computing Foundation (CNCF)](https://cncf.io/) graduated project.
## Use Cases
* [Machine Learning pipelines](use-cases/machine-learning.md)
* [Data and batch processing](use-cases/data-processing.md)
* [Infrastructure automation](use-cases/infrastructure-automation.md)
* [CI/CD](use-cases/ci-cd.md)
* [Other use cases](use-cases/other.md)
## Why Argo Workflows?
* Argo Workflows is the most popular workflow execution engine for Kubernetes.
* Light-weight, scalable, and easier to use.
* Designed from the ground up for containers without the overhead and limitations of legacy VM and server-based environments.
* Cloud agnostic and can run on any Kubernetes cluster.
[Read what people said in our latest survey](https://blog.argoproj.io/argo-workflows-events-2023-user-survey-results-82c53bc30543)
## Try Argo Workflows
You can try Argo Workflows via one of the following:
1. [Interactive Training Material](https://killercoda.com/argoproj/course/argo-workflows/)
1. [Access the demo environment](https://workflows.apps.argoproj.io/workflows/argo)
![Screenshot](assets/screenshot.png)
## Who uses Argo Workflows?
[About 200+ organizations are officially using Argo Workflows](https://github.com/argoproj/argo-workflows/blob/main/USERS.md)
## Ecosystem
Just some of the projects that use or rely on Argo Workflows (complete list [here](https://github.com/akuity/awesome-argo#ecosystem-projects)):
* [Argo Events](https://github.com/argoproj/argo-events)
* [Couler](https://github.com/couler-proj/couler)
* [Hera](https://github.com/argoproj-labs/hera-workflows)
* [Katib](https://github.com/kubeflow/katib)
* [Kedro](https://kedro.readthedocs.io/en/stable/)
* [Kubeflow Pipelines](https://github.com/kubeflow/pipelines)
* [Netflix Metaflow](https://metaflow.org)
* [Onepanel](https://github.com/onepanelio/onepanel)
* [Orchest](https://github.com/orchest/orchest/)
* [Piper](https://github.com/quickube/piper)
* [Ploomber](https://github.com/ploomber/ploomber)
* [Seldon](https://github.com/SeldonIO/seldon-core)
* [SQLFlow](https://github.com/sql-machine-learning/sqlflow)
## Client Libraries
Check out our [Java, Golang and Python clients](client-libraries.md).
## Quickstart
* [Get started here](quick-start.md)
* [Walk-through examples](walk-through/index.md)
## Documentation
You're here!
## Features
An incomplete list of features Argo Workflows provide:
* UI to visualize and manage Workflows
* Artifact support (S3, Artifactory, Alibaba Cloud OSS, Azure Blob Storage, HTTP, Git, GCS, raw)
* Workflow templating to store commonly used Workflows in the cluster
* Archiving Workflows after executing for later access
* Scheduled workflows using cron
* Server interface with REST API (HTTP and GRPC)
* DAG or Steps based declaration of workflows
* Step level input & outputs (artifacts/parameters)
* Loops
* Parameterization
* Conditionals
* Timeouts (step & workflow level)
* Retry (step & workflow level)
* Resubmit (memoized)
* Suspend & Resume
* Cancellation
* K8s resource orchestration
* Exit Hooks (notifications, cleanup)
* Garbage collection of completed workflow
* Scheduling (affinity/tolerations/node selectors)
* Volumes (ephemeral/existing)
* Parallelism limits
* Daemoned steps
* DinD (docker-in-docker)
* Script steps
* Event emission
* Prometheus metrics
* Multiple executors
* Multiple pod and workflow garbage collection strategies
* Automatically calculated resource usage per step
* Java/Golang/Python SDKs
* Pod Disruption Budget support
* Single-sign on (OAuth2/OIDC)
* Webhook triggering
* CLI
* Out-of-the box and custom Prometheus metrics
* Windows container support
* Embedded widgets
* Multiplex log viewer
## Community Meetings
We host monthly community meetings where we and the community showcase demos and discuss the current and future state of the project. Feel free to join us!
For Community Meeting information, minutes and recordings, please [see here](https://bit.ly/argo-wf-cmty-mtng).
Participation in Argo Workflows is governed by the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md)
## Community Blogs and Presentations
* [Awesome-Argo: A Curated List of Awesome Projects and Resources Related to Argo](https://github.com/terrytangyuan/awesome-argo)
* [Automation of Everything - How To Combine Argo Events, Workflows & Pipelines, CD, and Rollouts](https://youtu.be/XNXJtxkUKeY)
* [Argo Workflows and Pipelines - CI/CD, Machine Learning, and Other Kubernetes Workflows](https://youtu.be/UMaivwrAyTA)
* [Argo Ansible role: Provisioning Argo Workflows on OpenShift](https://medium.com/@marekermk/provisioning-argo-on-openshift-with-ansible-and-kustomize-340a1fda8b50)
* [Argo Workflows vs Apache Airflow](http://bit.ly/30YNIvT)
* [CI/CD with Argo on Kubernetes](https://medium.com/@bouwe.ceunen/ci-cd-with-argo-on-kubernetes-28c1a99616a9)
* [Define Your CI/CD Pipeline with Argo Workflows](https://haque-zubair.medium.com/define-your-ci-cd-pipeline-with-argo-workflows-25aefb02fa63)
* [Distributed Machine Learning Patterns from Manning Publication](https://github.com/terrytangyuan/distributed-ml-patterns)
* [Running Argo Workflows Across Multiple Kubernetes Clusters](https://admiralty.io/blog/running-argo-workflows-across-multiple-kubernetes-clusters/)
* [Open Source Model Management Roundup: Polyaxon, Argo, and Seldon](https://www.anaconda.com/blog/developer-blog/open-source-model-management-roundup-polyaxon-argo-and-seldon/)
* [Producing 200 OpenStreetMap extracts in 35 minutes using a scalable data workflow](https://www.interline.io/blog/scaling-openstreetmap-data-workflows/)
* [Argo integration review](http://dev.matt.hillsdon.net/2018/03/24/argo-integration-review.html)
* TGI Kubernetes with Joe Beda: [Argo workflow system](https://www.youtube.com/watch?v=M_rxPPLG8pU&start=859)
## Project Resources
* [Argo Project GitHub organization](https://github.com/argoproj)
* [Argo Website](https://argoproj.github.io/)
* [Argo Slack](https://argoproj.github.io/community/join-slack)
## Security
See [Security](security.md).

View file

@ -1,9 +0,0 @@
![cnoe logo](./images/cnoe-logo.png)
# Example Spark Application
Thanks for trying out this demo! In this example, we deployed a simple Apache Spark job through Argo Workflows.
To learn more about Spark Operators, check out [this link](https://github.com/kubeflow/spark-operator)
To learn more about Argo Workflows, see [this link](https://argoproj.github.io/workflows/)

View file

@ -1,86 +0,0 @@
# Kubeflow Spark Operator
[![Go Report Card](https://goreportcard.com/badge/github.com/kubeflow/spark-operator)](https://goreportcard.com/report/github.com/kubeflow/spark-operator)
## What is Spark Operator?
The Kubernetes Operator for Apache Spark aims to make specifying and running [Spark](https://github.com/apache/spark) applications as easy and idiomatic as running other workloads on Kubernetes. It uses
[Kubernetes custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) for specifying, running, and surfacing status of Spark applications.
## Overview
For a complete reference of the custom resource definitions, please refer to the [API Definition](docs/api-docs.md). For details on its design, please refer to the [Architecture](https://www.kubeflow.org/docs/components/spark-operator/overview/#architecture). It requires Spark 2.3 and above that supports Kubernetes as a native scheduler backend.
The Kubernetes Operator for Apache Spark currently supports the following list of features:
* Supports Spark 2.3 and up.
* Enables declarative application specification and management of applications through custom resources.
* Automatically runs `spark-submit` on behalf of users for each `SparkApplication` eligible for submission.
* Provides native [cron](https://en.wikipedia.org/wiki/Cron) support for running scheduled applications.
* Supports customization of Spark pods beyond what Spark natively is able to do through the mutating admission webhook, e.g., mounting ConfigMaps and volumes, and setting pod affinity/anti-affinity.
* Supports automatic application re-submission for updated `SparkApplication` objects with updated specification.
* Supports automatic application restart with a configurable restart policy.
* Supports automatic retries of failed submissions with optional linear back-off.
* Supports mounting local Hadoop configuration as a Kubernetes ConfigMap automatically via `sparkctl`.
* Supports automatically staging local application dependencies to Google Cloud Storage (GCS) via `sparkctl`.
* Supports collecting and exporting application-level metrics and driver/executor metrics to Prometheus.
## Project Status
**Project status:** *beta*
**Current API version:** *`v1beta2`*
**If you are currently using the `v1beta1` version of the APIs in your manifests, please update them to use the `v1beta2` version by changing `apiVersion: "sparkoperator.k8s.io/<version>"` to `apiVersion: "sparkoperator.k8s.io/v1beta2"`. You will also need to delete the `previous` version of the CustomResourceDefinitions named `sparkapplications.sparkoperator.k8s.io` and `scheduledsparkapplications.sparkoperator.k8s.io`, and replace them with the `v1beta2` version either by installing the latest version of the operator or by running `kubectl create -f config/crd/bases`.**
## Prerequisites
* Version >= 1.13 of Kubernetes to use the [`subresource` support for CustomResourceDefinitions](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#subresources), which became beta in 1.13 and is enabled by default in 1.13 and higher.
* Version >= 1.16 of Kubernetes to use the `MutatingWebhook` and `ValidatingWebhook` of `apiVersion: admissionregistration.k8s.io/v1`.
## Getting Started
For getting started with Spark operator, please refer to [Getting Started](https://www.kubeflow.org/docs/components/spark-operator/getting-started/).
## User Guide
For detailed user guide and API documentation, please refer to [User Guide](https://www.kubeflow.org/docs/components/spark-operator/user-guide/) and [API Specification](docs/api-docs.md).
If you are running Spark operator on Google Kubernetes Engine (GKE) and want to use Google Cloud Storage (GCS) and/or BigQuery for reading/writing data, also refer to the [GCP guide](https://www.kubeflow.org/docs/components/spark-operator/user-guide/gcp/).
## Version Matrix
The following table lists the most recent few versions of the operator.
| Operator Version | API Version | Kubernetes Version | Base Spark Version |
| ------------- | ------------- | ------------- | ------------- |
| `v1beta2-1.6.x-3.5.0` | `v1beta2` | 1.16+ | `3.5.0` |
| `v1beta2-1.5.x-3.5.0` | `v1beta2` | 1.16+ | `3.5.0` |
| `v1beta2-1.4.x-3.5.0` | `v1beta2` | 1.16+ | `3.5.0` |
| `v1beta2-1.3.x-3.1.1` | `v1beta2` | 1.16+ | `3.1.1` |
| `v1beta2-1.2.3-3.1.1` | `v1beta2` | 1.13+ | `3.1.1` |
| `v1beta2-1.2.2-3.0.0` | `v1beta2` | 1.13+ | `3.0.0` |
| `v1beta2-1.2.1-3.0.0` | `v1beta2` | 1.13+ | `3.0.0` |
| `v1beta2-1.2.0-3.0.0` | `v1beta2` | 1.13+ | `3.0.0` |
| `v1beta2-1.1.x-2.4.5` | `v1beta2` | 1.13+ | `2.4.5` |
| `v1beta2-1.0.x-2.4.4` | `v1beta2` | 1.13+ | `2.4.4` |
## Developer Guide
For developing with Spark Operator, please refer to [Developer Guide](https://www.kubeflow.org/docs/components/spark-operator/developer-guide/).
## Contributor Guide
For contributing to Spark Operator, please refer to [Contributor Guide](CONTRIBUTING.md).
## Community
* Join the [CNCF Slack Channel](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels) and then join `#kubeflow-spark-operator` Channel.
* Check out our blog post [Announcing the Kubeflow Spark Operator: Building a Stronger Spark on Kubernetes Community](https://blog.kubeflow.org/operators/2024/04/15/kubeflow-spark-operator.html).
* Join our monthly community meeting [Kubeflow Spark Operator Meeting Notes](https://bit.ly/3VGzP4n).
## Adopters
Check out [adopters of Spark Operator](ADOPTERS.md).

View file

@ -1,109 +0,0 @@
# apiVersion: argoproj.io/v1alpha1
# kind: Workflow
# metadata:
# name: ${{values.name}}
# namespace: argo
# labels:
# env: dev
# entity-id: ${{values.name}}
# spec:
# serviceAccountName: admin
# entrypoint: whalesay
# templates:
# - name: whalesay
# container:
# image: docker/whalesay:latest
# command: [cowsay]
# args: ["hello world"]
---
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
name: ${{values.name}}
namespace: argo
labels:
env: dev
entity-id: ${{values.name}}
spec:
serviceAccountName: admin
entrypoint: main
action: create
templates:
- name: main
steps:
- - name: spark-job
template: spark-job
- - name: wait
template: wait
arguments:
parameters:
- name: spark-job-name
value: '{{steps.spark-job.outputs.parameters.spark-job-name}}'
- name: wait
inputs:
parameters:
- name: spark-job-name
resource:
action: get
successCondition: status.applicationState.state == COMPLETED
failureCondition: status.applicationState.state == FAILED
manifest: |
apiVersion: "sparkoperator.k8s.io/v1beta2"
kind: SparkApplication
metadata:
name: {{inputs.parameters.spark-job-name}}
namespace: argo
- name: spark-job
outputs:
parameters:
- name: spark-job-name
valueFrom:
jsonPath: '{.metadata.name}'
resource:
action: create
setOwnerReference: true
manifest: |
apiVersion: "sparkoperator.k8s.io/v1beta2"
kind: SparkApplication
metadata:
name: spark-pi-${{values.name}}
namespace: argo
labels:
env: dev
entity-id: ${{values.name}}
spec:
type: Scala
mode: cluster
image: "docker.io/apache/spark:v3.1.3"
imagePullPolicy: IfNotPresent
mainClass: org.apache.spark.examples.SparkPi
mainApplicationFile: "local:///opt/spark/examples/jars/spark-examples_2.12-3.1.3.jar"
sparkVersion: "3.1.1"
restartPolicy:
type: Never
volumes:
- name: "test-volume"
hostPath:
path: "/tmp"
type: Directory
driver:
cores: 1
coreLimit: "1200m"
memory: "512m"
labels:
version: 3.1.1
serviceAccount: admin
volumeMounts:
- name: "test-volume"
mountPath: "/tmp"
executor:
cores: 1
instances: 1
memory: "512m"
labels:
version: 3.1.1
volumeMounts:
- name: "test-volume"
mountPath: "/tmp"

View file

@ -1,8 +0,0 @@
site_name: 'Argo Spark Example'
nav:
- Home: index.md
- Argo-Workflows: argo-workflows.md
- Apache Spark Operator: spark-operator.md
plugins:
- techdocs-core

View file

@ -1,62 +0,0 @@
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
description: Creates a Basic Kubernetes Deployment
name: argo-workflows-basic
title: Basic Argo Workflow with a Spark Job
spec:
owner: guests
type: service
parameters:
- title: Configuration Options
required:
- name
properties:
name:
type: string
description: name of this application
mainApplicationFile:
type: string
default: 'local:///opt/spark/examples/jars/spark-examples_2.12-3.1.3.jar'
description: Path to the main application file
steps:
- id: template
name: Generating component
action: fetch:template
input:
url: ./skeleton
values:
name: ${{parameters.name}}
- id: publish
name: Publishing to a gitea git repository
action: publish:gitea
input:
description: This is an example app
# Hard coded value for this demo purposes only.
repoUrl: {{{ .Env.DOMAIN_GITEA }}}:443/?repo=${{parameters.name}}
defaultBranch: main
- id: create-argocd-app
name: Create ArgoCD App
action: cnoe:create-argocd-app
input:
appName: ${{parameters.name}}
appNamespace: ${{parameters.name}}
argoInstance: in-cluster
projectName: default
# necessary until we generate our own cert
repoUrl: https://{{{ .Env.DOMAIN_GITEA }}}:443/giteaAdmin/${{parameters.name}}
path: "manifests"
- id: register
name: Register
action: catalog:register
input:
repoContentsUrl: ${{ steps['publish'].output.repoContentsUrl }}
catalogInfoPath: 'catalog-info.yaml'
output:
links:
- title: Open in catalog
icon: catalog
entityRef: ${{ steps['register'].output.entityRef }}

Some files were not shown because too many files have changed in this diff Show more