diff --git a/template/registry/local-backup.yaml b/template/registry/local-backup.yaml new file mode 100644 index 0000000..dc5cfbe --- /dev/null +++ b/template/registry/local-backup.yaml @@ -0,0 +1,24 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: local-backup + namespace: argocd + labels: + env: dev + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + destination: + name: in-cluster + namespace: argocd + source: + path: stacks/local-backup + repoURL: 'https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder' + targetRevision: HEAD + project: default + syncPolicy: + automated: + prune: true + selfHeal: true + syncOptions: + - CreateNamespace=true diff --git a/template/stacks/local-backup/README.md b/template/stacks/local-backup/README.md new file mode 100644 index 0000000..3ded63d --- /dev/null +++ b/template/stacks/local-backup/README.md @@ -0,0 +1,126 @@ +# Local Backup with Velero and Minio + +This is example is adapted from the original icpbuilder stack. + +The two significant changes from the original were made: + +* disabled `hostPath` mount to persist backups within kind, since backups do not work sufficiently in this example due to PVC issues, see below. +* renamed `minio` namespace to `minio-backup` so it does not collide with other minio examples. + +Within kind, it can only backup kubernetes objects. Data from PVC's is skipped, see below why. + +[Velero](https://velero.io/) requires some compatible storage providers as its backup target. This local installation uses [MinIO](https://min.io/) as an example. +However, MinIO is not officially supported by Velero but works due to S3 compatibility. + +The current setup does NOT persist backups but stores them in MinIO's PVCs. Proper backups should configure external storage, see [Supported Providers](https://velero.io/docs/main/supported-providers/). + +## Installation + +The stack is installed as part of the `./example.sh` run. + +In order to persist a local backup you have to mount a local directory within `main.go`: + +```yaml +nodes: + - role: control-plane + extraMounts: + - hostPath: /some/path/backup # replace with your own path + containerPath: /backup +``` + +Kind creates the directory on the host but you might have to adjust the permissions, otherwise the minio pod fails to start. + +## Using it + +After the installation velero and minio should be visible in ArgoCD. + +During the installation credentials for minio are generated and shared with velero. You can access them manually: + +```bash +kubectl -n minio-backup get secret root-creds -o go-template='{{ range $key, $value := .data }}{{ printf "%s: %s\n" $key ($value | base64decode) }}{{ end }}' +# example output +# rootPassword: aKKZzLnyry6OYZts17vMTf32H5ghFL4WYgu6bHujm +# rootUser: ge8019yksArb7BICt3MLY9 +``` + +A bucket in minio was created and velero uses it for its backups by default, see helm `values.yaml` files. + + +### Backup and Restore + +Backups and subsequent restores can be scheduled by either using the velero cli or by creating CRD objects. + +Check the `./demo` directory for equivalent CRD manifests. + +Create a backup of the backstage namespace, see `schedule` task for more permanent setups: + +```shell +velero backup create backstage-backup --include-namespaces backstage +``` + +There are more options to create a fine granular backup and to set the backup storage. +See velero's docs for details. + +Check the backup with: +```shell +velero backup get +``` + +To get more details on the backup you need to be able to connect to velero's backup storage, i.e. minio. +Using `kubefwd` here helps a lot (this is not necessary for restore). + +```shell +kubefwd services -n minio-backup +``` + +More details with `describe` and `logs`: + +```shell +velero backup describe backstage-backup --details +velero backup logs backstage-backup +``` + +Restore the backup into the original namespace, you might want to delete the existing namespace beforehand: + +```shell +kubectl delete namespace backstage +velero restore create --from-backup backstage-backup +``` + +When restoring, velero does not replace existing objects in the backup target. + +ArgoCD does pickup on the changes and also validates that the backup is in sync. + + +## Issues with Persistent Volumes + +Velero has no issue to backup kubernetes objects like Deployments, ConfigMaps, etc. since they are just yaml/json definitions. +Volumes containing data are, however, more complex. The preferred type of backup are kubernetes' VolumeSnapshots as they consistently store the state +of a volume at a given point in time in an atomic action. Those snapshots live within the cluster and are subsequently downloaded into one of velero's +storage backends for safekeeping. + +However, VolumeSnapshots are only possible on storage backends that support them via CSI drivers. +Backends like `nfs` or `hostPath` do NOT support them. Here, velero uses an alternative method +called [File System Backups](https://velero.io/docs/main/file-system-backup/). +In essence, this a simple copy operation based on the file system. Even though +this uses more sophisticated tooling under the hood, i.e. kopia, it is not +possible to create a backup in an atomic transaction. Thus, the resulting backup +might be inconsistent. + +Furthermore, for file system backups to work velero installs a node-agent as a +DaemonSet on each Kubernetes node. The agent is aware of the node's internal +storage and accesses the directories on the host directly to copy the files. +This is not supported for hostPath volumes as they mount an arbitrary path +on the host. In theory, a backup is possible but due extra config and security +considerations intentionally skipped. Kind's local-path provider storage uses +a hostPath and is thus not supported for any kind of backup. + +## TODOs + +* The MinIO -backup installation is only intended as an example and must either +be configured properly or replaced. + +* The current example does not automatically schedule backups. + +* velero chart must be properly parameterized + diff --git a/template/stacks/local-backup/demo/backup.yaml b/template/stacks/local-backup/demo/backup.yaml new file mode 100644 index 0000000..e335141 --- /dev/null +++ b/template/stacks/local-backup/demo/backup.yaml @@ -0,0 +1,9 @@ +# velero backup create backstage-backup --include-namespaces backstage +apiVersion: velero.io/v1 +kind: Backup +metadata: + name: backstage-backup + namespace: velero +spec: + includedNamespaces: + - 'backstage' diff --git a/template/stacks/local-backup/demo/restore.yaml b/template/stacks/local-backup/demo/restore.yaml new file mode 100644 index 0000000..b0fd5b5 --- /dev/null +++ b/template/stacks/local-backup/demo/restore.yaml @@ -0,0 +1,10 @@ +# velero restore create --from-backup backstage-backup +apiVersion: velero.io/v1 +kind: Restore +metadata: + name: backstage-backup + namespace: velero +spec: + backupName: backstage-backup + includedNamespaces: + - 'backstage' diff --git a/template/stacks/local-backup/minio.yaml b/template/stacks/local-backup/minio.yaml new file mode 100644 index 0000000..cd23156 --- /dev/null +++ b/template/stacks/local-backup/minio.yaml @@ -0,0 +1,33 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: minio + namespace: argocd + labels: + env: dev + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + project: default + sources: + - repoURL: 'https://charts.min.io' + targetRevision: 5.0.15 + helm: + releaseName: minio + valueFiles: + - $values/stacks/local-backup/minio/helm/values.yaml + chart: minio + - repoURL: https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder + targetRevision: HEAD + ref: values + - repoURL: https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder + targetRevision: HEAD + path: "stacks/local-backup/minio/manifests" + destination: + server: "https://kubernetes.default.svc" + namespace: minio-backup + syncPolicy: + syncOptions: + - CreateNamespace=true + automated: + selfHeal: true diff --git a/template/stacks/local-backup/minio/helm/values.yaml b/template/stacks/local-backup/minio/helm/values.yaml new file mode 100644 index 0000000..8a58d96 --- /dev/null +++ b/template/stacks/local-backup/minio/helm/values.yaml @@ -0,0 +1,23 @@ +replicas: 1 +mode: standalone + +resources: + requests: + memory: 128Mi + +persistence: + enabled: true + storageClass: standard + size: 512Mi + # volumeName: backup # re-enable this to mount a local host path, see minio-pv.yaml + +buckets: + - name: edfbuilder-backups + +consoleIngress: + enabled: true + ingressClassName: nginx + hosts: + - minio-backup.{{ .Values.edfbuilderTargetDomain }} + +existingSecret: root-creds diff --git a/template/stacks/local-backup/minio/manifests/minio-pv.yaml b/template/stacks/local-backup/minio/manifests/minio-pv.yaml new file mode 100644 index 0000000..8f2c88c --- /dev/null +++ b/template/stacks/local-backup/minio/manifests/minio-pv.yaml @@ -0,0 +1,13 @@ +# re-enable this config to mount a local host path, see `../helm/values.yaml` +# apiVersion: v1 +# kind: PersistentVolume +# metadata: +# name: backup +# spec: +# storageClassName: standard +# accessModes: +# - ReadWriteOnce +# capacity: +# storage: 512Mi +# hostPath: +# path: /backup diff --git a/template/stacks/local-backup/minio/manifests/secret-sync.yaml b/template/stacks/local-backup/minio/manifests/secret-sync.yaml new file mode 100644 index 0000000..c4c7032 --- /dev/null +++ b/template/stacks/local-backup/minio/manifests/secret-sync.yaml @@ -0,0 +1,154 @@ +apiVersion: v1 +kind: ServiceAccount +metadata: + name: secret-sync + namespace: minio-backup + annotations: + argocd.argoproj.io/hook: Sync + argocd.argoproj.io/sync-wave: "-20" +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: secret-sync + namespace: minio-backup + annotations: + argocd.argoproj.io/hook: Sync + argocd.argoproj.io/sync-wave: "-20" +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "create", "update", "patch"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: secret-sync + namespace: minio-backup + annotations: + argocd.argoproj.io/hook: Sync + argocd.argoproj.io/sync-wave: "-20" +subjects: + - kind: ServiceAccount + name: secret-sync + namespace: minio-backup +roleRef: + kind: Role + name: secret-sync + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: secret-sync + namespace: velero + annotations: + argocd.argoproj.io/hook: Sync + argocd.argoproj.io/sync-wave: "-20" +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "create", "update", "patch"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: secret-sync + namespace: velero + annotations: + argocd.argoproj.io/hook: Sync + argocd.argoproj.io/sync-wave: "-20" +subjects: + - kind: ServiceAccount + name: secret-sync + namespace: minio-backup +roleRef: + kind: Role + name: secret-sync + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: secret-sync + namespace: minio-backup + annotations: + argocd.argoproj.io/hook: PostSync +spec: + template: + metadata: + generateName: secret-sync + spec: + serviceAccountName: secret-sync + restartPolicy: Never + containers: + - name: kubectl + image: docker.io/bitnami/kubectl + command: ["/bin/bash", "-c"] + args: + - | + set -e + kubectl get secrets -n minio-backup root-creds -o json > /tmp/secret + ACCESS=$(jq -r '.data.rootUser | @base64d' /tmp/secret) + SECRET=$(jq -r '.data.rootPassword | @base64d' /tmp/secret) + + echo \ + "apiVersion: v1 + kind: Secret + metadata: + name: secret-key + namespace: velero + type: Opaque + stringData: + aws: | + [default] + aws_access_key_id=${ACCESS} + aws_secret_access_key=${SECRET} + " > /tmp/secret.yaml + + kubectl apply -f /tmp/secret.yaml +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: minio-root-creds + namespace: minio-backup + annotations: + argocd.argoproj.io/hook: Sync + argocd.argoproj.io/sync-wave: "-10" +spec: + template: + metadata: + generateName: minio-root-creds + spec: + serviceAccountName: secret-sync + restartPolicy: Never + containers: + - name: kubectl + image: docker.io/bitnami/kubectl + command: ["/bin/bash", "-c"] + args: + - | + kubectl get secrets -n minio-backup root-creds + if [ $? -eq 0 ]; then + exit 0 + fi + + set -e + + NAME=$(openssl rand -base64 24) + PASS=$(openssl rand -base64 36) + + echo \ + "apiVersion: v1 + kind: Secret + metadata: + name: root-creds + namespace: minio-backup + type: Opaque + stringData: + rootUser: "${NAME}" + rootPassword: "${PASS}" + " > /tmp/secret.yaml + + kubectl apply -f /tmp/secret.yaml diff --git a/template/stacks/local-backup/velero.yaml b/template/stacks/local-backup/velero.yaml new file mode 100644 index 0000000..4a818ab --- /dev/null +++ b/template/stacks/local-backup/velero.yaml @@ -0,0 +1,31 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: velero + namespace: argocd + labels: + env: dev + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + project: default + sources: + - repoURL: 'https://vmware-tanzu.github.io/helm-charts' + targetRevision: 8.0.0 + helm: + releaseName: velero + valueFiles: + - $values/stacks/local-backup/velero/helm/values.yaml + chart: velero + - repoURL: https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder + targetRevision: HEAD + ref: values + destination: + server: "https://kubernetes.default.svc" + namespace: velero + syncPolicy: + syncOptions: + - CreateNamespace=true + automated: + prune: true + selfHeal: true diff --git a/template/stacks/local-backup/velero/helm/values.yaml b/template/stacks/local-backup/velero/helm/values.yaml new file mode 100644 index 0000000..5b7d8d7 --- /dev/null +++ b/template/stacks/local-backup/velero/helm/values.yaml @@ -0,0 +1,25 @@ +resources: + requests: + memory: 128Mi +initContainers: + - name: velero-plugin-for-aws + image: velero/velero-plugin-for-aws:v1.11.0 + imagePullPolicy: IfNotPresent + volumeMounts: + - mountPath: /target + name: plugins +# snapshotsEnabled: false # create snapshot crd? +# deployNodeAgent: true # install node agent as daemonset for file system backups? +configuration: + # defaultVolumesToFsBackup: true # backup pod volumes via fsb without explicit annotation? + backupStorageLocation: + - name: default + provider: aws + bucket: edfbuilder-backups + credential: + name: secret-key # this key is created within the minio-backup/secret-sync and injected into the velero namespace + key: aws + config: + region: minio + s3Url: http://minio.minio-backup.svc.cluster.local:9000 # internal resolution, external access for velero cli via fwd + s3ForcePathStyle: "true" diff --git a/template/stacks/monitoring/grafana-dashboards.yaml b/template/stacks/monitoring/grafana-dashboards.yaml new file mode 100644 index 0000000..8a4b7be --- /dev/null +++ b/template/stacks/monitoring/grafana-dashboards.yaml @@ -0,0 +1,25 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: grafana-dashboards + namespace: argocd + labels: + env: dev + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + project: default + source: + repoURL: https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder + targetRevision: HEAD + path: "stacks/monitoring/kube-prometheus/dashboards" + destination: + server: "https://kubernetes.default.svc" + namespace: monitoring + syncPolicy: + syncOptions: + - CreateNamespace=true + automated: + selfHeal: true + retry: + limit: -1 \ No newline at end of file diff --git a/template/stacks/monitoring/kube-prometheus.yaml b/template/stacks/monitoring/kube-prometheus.yaml index c0d1347..2dac1f1 100644 --- a/template/stacks/monitoring/kube-prometheus.yaml +++ b/template/stacks/monitoring/kube-prometheus.yaml @@ -14,7 +14,7 @@ spec: selfHeal: true syncOptions: - CreateNamespace=true - - ServerSideApply=true + - ServerSideApply=true # TODO: RIRE What does this mean: do not copy metdata, since (because of its large size) it can lead to sync failure destination: name: in-cluster namespace: monitoring @@ -27,4 +27,4 @@ spec: - $values/stacks/monitoring/kube-prometheus/values.yaml - repoURL: https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder targetRevision: HEAD - ref: values \ No newline at end of file + ref: values diff --git a/template/stacks/monitoring/kube-prometheus/dashboards/dashboard_loki_apps.yaml b/template/stacks/monitoring/kube-prometheus/dashboards/dashboard_loki_apps.yaml new file mode 100644 index 0000000..9e84526 --- /dev/null +++ b/template/stacks/monitoring/kube-prometheus/dashboards/dashboard_loki_apps.yaml @@ -0,0 +1,268 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: grafana-dashboard-1 + labels: + grafana_dashboard: "1" +data: + k8s-dashboard-01.json: | + { + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 1, + "links": [ + ], + "panels": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 5, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "expr": "{app=\"crossplane\"}", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: App crossplane", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 8 + }, + "id": 4, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "expr": "{app=\"argo-server\"}", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: App argo-server", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 16 + }, + "id": 3, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "expr": "{app=\"forgejo\"}", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: App forgejo", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 24 + }, + "id": 2, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "expr": "{app=\"backstage\"}", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: App backstage", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 32 + }, + "id": 1, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "expr": "{app=\"shoot-control-plane\"}", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: App shoot-control-plane", + "type": "logs" + } + ], + "preload": false, + "schemaVersion": 40, + "tags": [ + ], + "templating": { + "list": [ + ] + }, + "time": { + "from": "now-6h", + "to": "now" + }, + "timepicker": { + }, + "timezone": "browser", + "title": "Loki Logs: Apps", + "uid": "ee4iuluru756of", + "version": 2, + "weekStart": "" + } \ No newline at end of file diff --git a/template/stacks/monitoring/kube-prometheus/dashboards/dashboard_loki_components.yaml b/template/stacks/monitoring/kube-prometheus/dashboards/dashboard_loki_components.yaml new file mode 100644 index 0000000..b35ed60 --- /dev/null +++ b/template/stacks/monitoring/kube-prometheus/dashboards/dashboard_loki_components.yaml @@ -0,0 +1,845 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: grafana-dashboard-2 + labels: + grafana_dashboard: "1" +data: + k8s-dashboard-02.json: | + { + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 30, + "links": [ + ], + "panels": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 19, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"server\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component server", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 8 + }, + "id": 17, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"repo-server\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component repo-server", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 16 + }, + "id": 16, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"redis\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component redis", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 24 + }, + "id": 15, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"query-frontend\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component query-frontend", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 32 + }, + "id": 14, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"querier\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component querier", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 40 + }, + "id": 13, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"prometheus-operator-webhook\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component prometheus-operator-webhook", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 48 + }, + "id": 12, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"prometheus-operator\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component prometheus-operator", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 56 + }, + "id": 11, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"metrics\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component metrics", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 64 + }, + "id": 10, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"kube-scheduler\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component kube-scheduler", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 72 + }, + "id": 9, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"kube-controller-manager\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component kube-controller-manager", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 80 + }, + "id": 8, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"kube-apiserver\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component kube-apiserver", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 88 + }, + "id": 7, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"ingester\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component ingester", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 96 + }, + "id": 6, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"gateway\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component gateway", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 104 + }, + "id": 5, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"etcd\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component etcd", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 112 + }, + "id": 4, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"distributor\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component distributor", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 120 + }, + "id": 3, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"controller\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component controller", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 128 + }, + "id": 2, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"cloud-infrastructure-controller\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component cloud-infrastructure-controller", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 136 + }, + "id": 1, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{component=\"applicationset-controller\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Component application-controller", + "type": "logs" + } + ], + "preload": false, + "schemaVersion": 40, + "tags": [ + ], + "templating": { + "list": [ + ] + }, + "time": { + "from": "now-6h", + "to": "now" + }, + "timepicker": { + }, + "timezone": "browser", + "title": "Loki Logs: Components", + "uid": "ae4zuyp1kui9sc", + "version": 2, + "weekStart": "" + } \ No newline at end of file diff --git a/template/stacks/monitoring/kube-prometheus/dashboards/dashboard_loki_container.yaml b/template/stacks/monitoring/kube-prometheus/dashboards/dashboard_loki_container.yaml new file mode 100644 index 0000000..267bd90 --- /dev/null +++ b/template/stacks/monitoring/kube-prometheus/dashboards/dashboard_loki_container.yaml @@ -0,0 +1,537 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: grafana-dashboard-3 + labels: + grafana_dashboard: "1" +data: + k8s-dashboard-03.json: | + { + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 31, + "links": [ + ], + "panels": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 11, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"repo-server\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container repo-server", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 8 + }, + "id": 10, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"promtail\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container promtail", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 16 + }, + "id": 9, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"prometheus\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container prometheus", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 24 + }, + "id": 8, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"postgres\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container postgres", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 32 + }, + "id": 7, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"kube-prometheus-stack\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container kube-prometheus-stack", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 40 + }, + "id": 6, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"keycloak\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container keycloak", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 48 + }, + "id": 5, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"grafana\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container grafana", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 56 + }, + "id": 4, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"forgejo\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container forgejo", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 64 + }, + "id": 3, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"crossplane\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container crossplane", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 72 + }, + "id": 2, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"backstage\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container backstage", + "type": "logs" + }, + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "fieldConfig": { + "defaults": { + }, + "overrides": [ + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 80 + }, + "id": 1, + "options": { + "dedupStrategy": "none", + "enableLogDetails": true, + "prettifyLogMessage": false, + "showCommonLabels": false, + "showLabels": false, + "showTime": false, + "sortOrder": "Descending", + "wrapLogMessage": false + }, + "pluginVersion": "11.3.1", + "targets": [ + { + "datasource": { + "type": "loki", + "uid": "P8E80F9AEF21F6940" + }, + "editorMode": "builder", + "expr": "{container=\"argo-server\"} |= ``", + "queryType": "range", + "refId": "A" + } + ], + "title": "Logs: Container argo-server", + "type": "logs" + } + ], + "preload": false, + "schemaVersion": 40, + "tags": [ + ], + "templating": { + "list": [ + ] + }, + "time": { + "from": "now-6h", + "to": "now" + }, + "timepicker": { + }, + "timezone": "browser", + "title": "Loki Logs: Container", + "uid": "ee50bcaehmv40e", + "version": 2, + "weekStart": "" + } \ No newline at end of file diff --git a/template/stacks/monitoring/kube-prometheus/values.yaml b/template/stacks/monitoring/kube-prometheus/values.yaml index d4f21b9..7b80872 100644 --- a/template/stacks/monitoring/kube-prometheus/values.yaml +++ b/template/stacks/monitoring/kube-prometheus/values.yaml @@ -6,15 +6,37 @@ grafana: userKey: admin-user passwordKey: admin-password - grafana.ini: - server: - domain: {{ .Values.edfbuilderTargetDomain }} - root_url: "%(protocol)s://%(domain)s/grafana" - serve_from_sub_path: true + defaultDashboardsTimezone: Europe/Berlin + + additionalDataSources: + - name: Loki + type: loki + url: http://loki-loki-distributed-gateway.monitoring:80 + + syncPolicy: + syncOptions: + - ServerSideApply=true ingress: enabled: true ingressClassName: nginx hosts: - {{ .Values.edfbuilderTargetDomain }} - path: /grafana \ No newline at end of file + path: /grafana + + sidecar: + dashboards: + enabled: true + label: grafana_dashboard + folder: /tmp/dashboards + updateIntervalSeconds: 10 + folderAnnotation: grafana_folder + provider: + allowUiUpdates: true + foldersFromFilesStructure: true + + grafana.ini: + server: + domain: {{ .Values.edfbuilderTargetDomain }} + root_url: "%(protocol)s://%(domain)s/grafana" + serve_from_sub_path: true \ No newline at end of file diff --git a/template/stacks/monitoring/kube-prometheus/values_test_osc.yaml b/template/stacks/monitoring/kube-prometheus/values_test_osc.yaml new file mode 100644 index 0000000..55c0451 --- /dev/null +++ b/template/stacks/monitoring/kube-prometheus/values_test_osc.yaml @@ -0,0 +1,15 @@ +grafana: + namespaceOverride: "monitoring" + + grafana.ini: + server: + domain: forgejo.edf-bootstrap.cx.fg1.ffm.osc.live + root_url: "%(protocol)s://%(domain)s/grafana" + serve_from_sub_path: true + + ingress: + enabled: true + ingressClassName: nginx + hosts: + - {{ .Values.edfbuilderTargetDomain }} + path: /grafana \ No newline at end of file diff --git a/template/stacks/monitoring/loki.yaml b/template/stacks/monitoring/loki.yaml new file mode 100644 index 0000000..10e254f --- /dev/null +++ b/template/stacks/monitoring/loki.yaml @@ -0,0 +1,34 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: loki + namespace: argocd + labels: + env: dev + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + project: default + syncPolicy: + automated: + selfHeal: true + syncOptions: + - CreateNamespace=true + destination: + name: in-cluster + namespace: monitoring + sources: + - repoURL: https://github.com/grafana/helm-charts + path: charts/loki-distributed + targetRevision: HEAD + helm: + valueFiles: + - $values/stacks/monitoring/loki/values.yaml + - repoURL: https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder + targetRevision: HEAD + ref: values + + ## consider using the following version, if it works again + #- repoURL: https://github.com/grafana/loki + # path: production/helm/loki + diff --git a/template/stacks/monitoring/loki/values.yaml b/template/stacks/monitoring/loki/values.yaml new file mode 100644 index 0000000..e682468 --- /dev/null +++ b/template/stacks/monitoring/loki/values.yaml @@ -0,0 +1,13 @@ +loki: + commonConfig: + replication_factor: 1 + auth_enabled: false + + #experimental + storageConfig: + # boltdb_shipper: + # shared_store: s3 + # aws: + # s3: s3://${cluster_region} + # bucketnames: ${bucket_name} + filesystem: null diff --git a/template/stacks/monitoring/promtail.yaml b/template/stacks/monitoring/promtail.yaml new file mode 100644 index 0000000..279ee88 --- /dev/null +++ b/template/stacks/monitoring/promtail.yaml @@ -0,0 +1,29 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: promtail + namespace: argocd + labels: + env: dev + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + project: default + syncPolicy: + automated: + selfHeal: true + syncOptions: + - CreateNamespace=true + destination: + name: in-cluster + namespace: monitoring + sources: + - repoURL: https://github.com/grafana/helm-charts + path: charts/promtail + targetRevision: HEAD + helm: + valueFiles: + - $values/stacks/monitoring/promtail/values.yaml + - repoURL: https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder + targetRevision: HEAD + ref: values \ No newline at end of file diff --git a/template/stacks/monitoring/promtail/values.yaml b/template/stacks/monitoring/promtail/values.yaml new file mode 100644 index 0000000..49faadc --- /dev/null +++ b/template/stacks/monitoring/promtail/values.yaml @@ -0,0 +1,45 @@ +# -- Overrides the chart's name +nameOverride: null + +# -- Overrides the chart's computed fullname +fullnameOverride: null + +global: + # -- Allow parent charts to override registry hostname + imageRegistry: "" + # -- Allow parent charts to override registry credentials + imagePullSecrets: [] + +daemonset: + # -- Deploys Promtail as a DaemonSet + enabled: true + autoscaling: + # -- Creates a VerticalPodAutoscaler for the daemonset + enabled: false + +deployment: + # -- Deploys Promtail as a Deployment + enabled: false + +config: + enabled: true + logLevel: info + logFormat: logfmt + serverPort: 3101 + clients: + - url: http://loki-loki-distributed-gateway/loki/api/v1/push + scrape_configs: + - job_name: authlog + static_configs: + - targets: + - authlog + labels: + job: authlog + __path__: /logs/auth.log + - job_name: syslog + static_configs: + - targets: + - syslog + labels: + job: syslog + __path__: /logs/syslog \ No newline at end of file diff --git a/template/stacks/ref-implementation/fibonacci-app.yaml b/template/stacks/ref-implementation/fibonacci-app.yaml new file mode 100644 index 0000000..bd5c94a --- /dev/null +++ b/template/stacks/ref-implementation/fibonacci-app.yaml @@ -0,0 +1,25 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: fibonacci-app + namespace: argocd + labels: + env: dev + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + project: default + source: + repoURL: https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder + targetRevision: HEAD + path: "stacks/ref-implementation/fibonacci-app" + destination: + server: "https://kubernetes.default.svc" + namespace: fibonacci-app + syncPolicy: + syncOptions: + - CreateNamespace=true + automated: + selfHeal: true + retry: + limit: -1 diff --git a/template/stacks/ref-implementation/fibonacci-app/deployment.yaml b/template/stacks/ref-implementation/fibonacci-app/deployment.yaml new file mode 100644 index 0000000..16f42e8 --- /dev/null +++ b/template/stacks/ref-implementation/fibonacci-app/deployment.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: fibonacci-deployment + namespace: fibonacci-app +spec: + replicas: 1 + selector: + matchLabels: + app: fibonacci-go + template: + metadata: + labels: + app: fibonacci-go + spec: + containers: + - name: fibonacci-go + image: forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/christopher.hase/fibonacci_http_go:1.0.0 + ports: + - containerPort: 9090 \ No newline at end of file diff --git a/template/stacks/ref-implementation/fibonacci-app/ingress.yaml b/template/stacks/ref-implementation/fibonacci-app/ingress.yaml new file mode 100644 index 0000000..c5d189b --- /dev/null +++ b/template/stacks/ref-implementation/fibonacci-app/ingress.yaml @@ -0,0 +1,18 @@ +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: fibonacci-service + namespace: fibonacci-app +spec: + ingressClassName: nginx + rules: + - host: {{ .Values.edfbuilderTargetDomain }} + http: + paths: + - backend: + service: + name: fibonacci-service + port: + number: 9090 + path: /fibonacci + pathType: Prefix \ No newline at end of file diff --git a/template/stacks/ref-implementation/fibonacci-app/service.yaml b/template/stacks/ref-implementation/fibonacci-app/service.yaml new file mode 100644 index 0000000..6458fa4 --- /dev/null +++ b/template/stacks/ref-implementation/fibonacci-app/service.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +kind: Service +metadata: + name: fibonacci-service + namespace: fibonacci-app +spec: + selector: + app: fibonacci-go + ports: + - protocol: TCP + port: 9090 + targetPort: 9090 + type: ClusterIP \ No newline at end of file diff --git a/template/stacks/ref-implementation/openbao.yaml b/template/stacks/ref-implementation/openbao.yaml new file mode 100644 index 0000000..b378351 --- /dev/null +++ b/template/stacks/ref-implementation/openbao.yaml @@ -0,0 +1,34 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: openbao + namespace: argocd + labels: + env: dev + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + project: default + syncPolicy: + automated: + selfHeal: false + syncOptions: + - CreateNamespace=true + destination: + name: in-cluster + namespace: openbao + sources: + - repoURL: https://github.com/openbao/openbao-helm.git + path: charts/openbao + targetRevision: HEAD + helm: + valueFiles: + - $values/stacks/ref-implementation/openbao/values.yaml + - repoURL: https://gitea.{{ .Values.edfbuilderTargetDomain }}/giteaAdmin/edfbuilder + targetRevision: HEAD + ref: values + ignoreDifferences: + - group: admissionregistration.k8s.io + kind: MutatingWebhookConfiguration + jqPathExpressions: + - .webhooks[]?.clientConfig.caBundle diff --git a/template/stacks/ref-implementation/openbao/values.yaml b/template/stacks/ref-implementation/openbao/values.yaml new file mode 100644 index 0000000..d929572 --- /dev/null +++ b/template/stacks/ref-implementation/openbao/values.yaml @@ -0,0 +1,12 @@ +server: + ingress: + enabled: true + ingressClassName: nginx + hosts: + - host: openbao.{{ .Values.edfbuilderTargetDomain }} + paths: [] + dev: + enabled: true + +ui: + enabled: true \ No newline at end of file