Compare commits
98 commits
main
...
developmen
Author | SHA1 | Date | |
---|---|---|---|
60cf9319c2 | |||
b67bba5b49 | |||
85c51ff021 | |||
6427bff915 | |||
69fb71b1e4 | |||
c9f55115cf | |||
90de5149ff | |||
![]() |
304fbdcdcb | ||
![]() |
df186af47c | ||
![]() |
8d17325bcd | ||
![]() |
13bd4343d9 | ||
![]() |
910ea62dee | ||
![]() |
dae67f17f1 | ||
d0e2f54eec | |||
09ff335665 | |||
534e6b4592 | |||
![]() |
7d562ec04c | ||
6ccc09dd38 | |||
58e32c69d6 | |||
2dcccd7922 | |||
a6f81c1d02 | |||
401aefe420 | |||
![]() |
0a73aa0870 | ||
6f0ac1777a | |||
![]() |
866dca1655 | ||
![]() |
22967405ab | ||
![]() |
eb4b56118a | ||
2c339a6b73 | |||
e694536136 | |||
d256801640 | |||
5450db4d3d | |||
559cff4d28 | |||
61e63e56ef | |||
a4f56a5c06 | |||
ce7929af30 | |||
602b64fe0b | |||
55f6d83606 | |||
a0d38791ed | |||
cd14b9946a | |||
![]() |
1a0383e284 | ||
![]() |
432f37aae7 | ||
![]() |
511f9f97e2 | ||
![]() |
48b60acdb7 | ||
![]() |
91ee6313cc | ||
![]() |
f86ce821bc | ||
![]() |
6c9d026c16 | ||
![]() |
2f8eac5452 | ||
![]() |
3ebe604150 | ||
c94c5cb994 | |||
![]() |
30023b23a0 | ||
02454a26a7 | |||
![]() |
9acc97880b | ||
![]() |
a0d65d1a09 | ||
![]() |
04289d11e1 | ||
![]() |
0888db9f29 | ||
![]() |
c291f80713 | ||
a0541bcd77 | |||
2f4290a09c | |||
ed1be1e805 | |||
26db80313d | |||
ba99303f16 | |||
ee1101fb3a | |||
661edcf761 | |||
932d607f31 | |||
c1ac6a0eb9 | |||
5442480c50 | |||
d6c6aacf6c | |||
6266f6d20f | |||
3cc5387ef3 | |||
58d33d406b | |||
14db0d608c | |||
8b9ffc08ed | |||
3c00bd5ac3 | |||
6ebca3250b | |||
1071d0727f | |||
3f22c2dcf8 | |||
b8908db873 | |||
a6dd5fe8a8 | |||
e8f4cfc363 | |||
3c01c0b199 | |||
![]() |
1baeac9b2a | ||
![]() |
ef2c7f20c1 | ||
![]() |
a7cec02db8 | ||
![]() |
930159ce62 | ||
![]() |
31f3f9f65d | ||
![]() |
3411feace7 | ||
14145a8855 | |||
c51039f34f | |||
24748fb636 | |||
3029029c4a | |||
eae136e65a | |||
6cf0c13cff | |||
fba7cb3749 | |||
fda1701434 | |||
983c08c090 | |||
18d7b976f8 | |||
d2b241afbd | |||
65722bbe6d |
22
.vscode/settings.json
vendored
Normal file
|
@ -0,0 +1,22 @@
|
|||
{
|
||||
"workbench.colorCustomizations": {
|
||||
"activityBar.activeBackground": "#ab307e",
|
||||
"activityBar.background": "#ab307e",
|
||||
"activityBar.foreground": "#e7e7e7",
|
||||
"activityBar.inactiveForeground": "#e7e7e799",
|
||||
"activityBarBadge.background": "#25320e",
|
||||
"activityBarBadge.foreground": "#e7e7e7",
|
||||
"commandCenter.border": "#e7e7e799",
|
||||
"sash.hoverBorder": "#ab307e",
|
||||
"statusBar.background": "#832561",
|
||||
"statusBar.foreground": "#e7e7e7",
|
||||
"statusBarItem.hoverBackground": "#ab307e",
|
||||
"statusBarItem.remoteBackground": "#832561",
|
||||
"statusBarItem.remoteForeground": "#e7e7e7",
|
||||
"titleBar.activeBackground": "#832561",
|
||||
"titleBar.activeForeground": "#e7e7e7",
|
||||
"titleBar.inactiveBackground": "#83256199",
|
||||
"titleBar.inactiveForeground": "#e7e7e799"
|
||||
},
|
||||
"peacock.color": "#832561"
|
||||
}
|
109
README.md
|
@ -1,46 +1,95 @@
|
|||
# eDF Release 'PoC' [WiP]
|
||||
# 🌟 EDP - EdgeDeveloperPlatform
|
||||
|
||||
This is a preliminary README in 'WiP' state - it's just sketching the story and structure of work.
|
||||
> * Owner: Telekom MMS & T-Systems
|
||||
> * Date: December 20, 2024
|
||||
> * Version: Release 1.0.0 ('PoC')
|
||||
|
||||
When we reach the 'PoC' milestone it will be replaced by a short and concise README as the main documentation entry point for our customers who want to consume the 'edf_poc' product.
|
||||
## About this Repository
|
||||
|
||||
## POC Content Structure
|
||||
This repo [`edp-doc`](https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW-CICD/edp-doc) is the documentation repository of the **EDP product**.
|
||||
|
||||
* [Part 1 - User documentation](https://jira.telekom-mms.com/browse/IPCEICIS-368)
|
||||
* Part 2 - Hands On
|
||||
* [Part 2.1 - Local IdP creation](https://jira.telekom-mms.com/browse/IPCEICIS-765)
|
||||
* [Part 2.2 - OSC IdP creation](https://jira.telekom-mms.com/browse/IPCEICIS-766)
|
||||
* Part 2.1 & 2.2 Use Cases
|
||||
* [Part 2.x.1 - Golden Path template for a Go application](https://jira.telekom-mms.com/browse/IPCEICIS-514)
|
||||
* [Part 2.x.2 - Fibonacci Go application](https://jira.telekom-mms.com/browse/IPCEICIS-759)
|
||||
* [Part 2.x.3 - Forgejo Actions with respect to the Fibonacci CI/CD](https://jira.telekom-mms.com/browse/IPCEICIS-760)
|
||||
* [Part 2.x.4 - Telemetry with respect to the Fibonacci workload](https://jira.telekom-mms.com/browse/IPCEICIS-761)
|
||||
* [Part 2.x.5 - OSC infrastructure with respect to the Fibonacci workload](https://jira.telekom-mms.com/browse/IPCEICIS-762)
|
||||
* [Part 2.x.6 - Additional bells and whistles](https://jira.telekom-mms.com/browse/IPCEICIS-763)
|
||||
* [Part 2.3 - Extended Local IdP creation and orchestration](https://jira.telekom-mms.com/browse/IPCEICIS-767)
|
||||
* Part 2.3 Use Cases
|
||||
* tbd
|
||||
* [Part 3 - User documentation](https://jira.telekom-mms.com/browse/IPCEICIS-768)
|
||||
## About the EDP Product
|
||||
|
||||
EDP is a product developed by the IPCEI-CIS subproject 'edge Developer Framework'. The goal is to provide a cutting edge developer experience for developing and delivering applications in the cloud edge continuum.
|
||||
|
||||
## Content and Story
|
||||
## What EDP contains
|
||||
|
||||
This is the Storybook of the PoC content structure (content is depicted top right):
|
||||
The EDP product consists of three parts:
|
||||
|
||||

|
||||
1. The platform orchestrator, processing declarative 'platform stack' descriptions
|
||||
1. The predefined stack for a Demo EDP instance
|
||||
1. The documentation
|
||||
|
||||
## Use case content of 2.1 and 2.2 'Hands On'
|
||||
### Platform Orchestrator `edpbuilder`
|
||||
|
||||
This is the illustration of the [use cases (aka 'functionality') in the PoC content](https://confluence.telekom-mms.com/display/IPCEICIS/Proof+of+Concept+2024):
|
||||
[edpbuilder](https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW/edpbuilder) is a tool to quickly instantiate and manage Internal Development Platforms (IDPs).
|
||||
The Edge Development Platform Builder (edpbuilder) can easily setup a Kubenetes cluster (local kind cluster or OSC instance) and deploy tools to manage the Kubernetes resources and the software lifecicle of an application.
|
||||
|
||||

|
||||
### Predefined `Demo EDP` Stack
|
||||
|
||||
## Technical composition
|
||||
There are predefined stack sets for deploying and orchestrating a whole platform.
|
||||
|
||||
The 'edf_poc'-product should be as self-contained as possible.
|
||||
At the time of writing (version PoC) we provide the [`Demo EDP`as PoC stack](https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW-CICD/stacks).
|
||||
|
||||
Thus all parts coming from upstream repos should have a physical copy in this repo with a reference to the upstream repo. Therefore we need a mapping table from parts here to references in upstreams.
|
||||
#### `Demo EDP` Stack
|
||||
|
||||
Alternative: Submodules
|
||||
The `Demo EDP` Stack contains the follwing application components:
|
||||
|
||||
Proposal: The final product will be a export of the repo
|
||||
* Version-Control: Forgejo
|
||||
* CI: Forgejo Actions
|
||||
* CD: ArgoCD
|
||||
* Monitoring: Grafana, Prometheus, Loki, Promtail
|
||||
* SSO: Keycloak
|
||||
* Developer Portal: Backstage
|
||||
* Secret-Management: OpenBao, external-secrets
|
||||
* Infrastructur-Provisioning: Crossplane
|
||||
|
||||
One usecase to demonstrate the capabilities and development lifecycle flow through all stack components is the [PetClinic Application](./docs/user-documentation/petclinic.md)
|
||||
|
||||
### Documentation
|
||||
|
||||
The EDP documentation is subject to this repo in folder [`docs`](./docs/). It is created in the [`mkdocs`](https://www.mkdocs.org/) documentation format and natively embedded in the [`Backstage TechDocs` documentation technology](https://backstage.io/docs/features/techdocs/).
|
||||
|
||||
Thus it can be read in four ways by
|
||||
|
||||
1. [accessing the Backstage portal in a running Demo EDP](#doc-in-backstage-in-a-running-demo-edp)
|
||||
2. [accessing the Backstage portal in a Demo EDP setup on your computer](#doc-in-backstage-in-a-demo-edp-setup-on-your-computer)
|
||||
3. [browsing the documentaion repository](#doc-in-the-repository)
|
||||
4. [browsing in a Backstage simulation on your computer](#doc-in-a-backstage-simulation)
|
||||
|
||||
## Getting Started
|
||||
|
||||
So get started by reading the doc in one of these ways, sorted from 'easy' to 'opinionated' accessibility:
|
||||
|
||||
### Doc in Backstage in a running Demo EDP
|
||||
|
||||
The documentation can be easily accessed through a Demo EDP instance hosted on our Open Sovereign Cloud (OSC).
|
||||
Thus there you have a Backstage-Developer Portal running, containing the EDP documentation.
|
||||
|
||||
Simply log in to Demo EDP Backstage to read through the documentation:
|
||||
|
||||
* [EDP Documentation URL](https://edf-cc1.cx.fg1.ffm.osc.live/docs/edppoc/component/edp-documentation)
|
||||
* Username: user1
|
||||
* Password: PpMpfZYICG9MRRF-3QBY2Zz1-+URYB6+-JRe
|
||||
|
||||
### Doc in the repository
|
||||
|
||||
> Hint: the following link only works when you are in the repo (not in Backstage), like [the central one](https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW-CICD/edp-doc)
|
||||
|
||||
If you prefer direct access, the content of our documentation is centrally defined and maintained within the [docs folder of this repository](./docs/contents.md).
|
||||
|
||||
### Doc in a Backstage simulation
|
||||
|
||||
> Hint: the following link only works when you are in the repo (not in Backstage), like [the central one](https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW-CICD/edp-doc)
|
||||
|
||||
The Backstage TechDocs embedding can also be [emulated in a Backstage portal frame running on your computer](./live-preview-integration/README-backstage-local.md).
|
||||
|
||||
### Doc in Backstage in a Demo EDP setup on your computer
|
||||
|
||||
As `edpbuilder` is a infrastructure agnostic platform orchestrator, you also can boostrap the EDP Demo on your own laptop!
|
||||
|
||||
Using the edpbuilder, you can set up a local Demo EDP that comes with a Backstage instance containing the documentation:
|
||||
|
||||
> Hint: the following link only works when you are in the repo (not in Backstage), like [the central one](https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW-CICD/edp-doc)
|
||||
|
||||
[How to set up a local IDP on a kind cluster](./docs/user-documentation/edpbuilder.md)
|
||||
|
|
46
design/README-poc-design.md
Normal file
|
@ -0,0 +1,46 @@
|
|||
# eDF Release 'PoC' [WiP]
|
||||
|
||||
This is a preliminary README in 'WiP' state - it's just sketching the story and structure of work.
|
||||
|
||||
When we reach the 'PoC' milestone it will be replaced by a short and concise README as the main documentation entry point for our customers who want to consume the 'edf_poc' product.
|
||||
|
||||
## POC Content Structure
|
||||
|
||||
* [Part 1 - User documentation](https://jira.telekom-mms.com/browse/IPCEICIS-368)
|
||||
* Part 2 - Hands On
|
||||
* [Part 2.1 - Local IdP creation](https://jira.telekom-mms.com/browse/IPCEICIS-765)
|
||||
* [Part 2.2 - OSC IdP creation](https://jira.telekom-mms.com/browse/IPCEICIS-766)
|
||||
* Part 2.1 & 2.2 Use Cases
|
||||
* [Part 2.x.1 - Golden Path template for a Go application](https://jira.telekom-mms.com/browse/IPCEICIS-514)
|
||||
* [Part 2.x.2 - Fibonacci Go application](https://jira.telekom-mms.com/browse/IPCEICIS-759)
|
||||
* [Part 2.x.3 - Forgejo Actions with respect to the Fibonacci CI/CD](https://jira.telekom-mms.com/browse/IPCEICIS-760)
|
||||
* [Part 2.x.4 - Telemetry with respect to the Fibonacci workload](https://jira.telekom-mms.com/browse/IPCEICIS-761)
|
||||
* [Part 2.x.5 - OSC infrastructure with respect to the Fibonacci workload](https://jira.telekom-mms.com/browse/IPCEICIS-762)
|
||||
* [Part 2.x.6 - Additional bells and whistles](https://jira.telekom-mms.com/browse/IPCEICIS-763)
|
||||
* [Part 2.3 - Extended Local IdP creation and orchestration](https://jira.telekom-mms.com/browse/IPCEICIS-767)
|
||||
* Part 2.3 Use Cases
|
||||
* tbd
|
||||
* [Part 3 - User documentation](https://jira.telekom-mms.com/browse/IPCEICIS-768)
|
||||
|
||||
|
||||
## Content and Story
|
||||
|
||||
This is the Storybook of the PoC content structure (content is depicted top right):
|
||||
|
||||

|
||||
|
||||
## Use case content of 2.1 and 2.2 'Hands On'
|
||||
|
||||
This is the illustration of the [use cases (aka 'functionality') in the PoC content](https://confluence.telekom-mms.com/display/IPCEICIS/Proof+of+Concept+2024):
|
||||
|
||||

|
||||
|
||||
## Technical composition
|
||||
|
||||
The 'edf_poc'-product should be as self-contained as possible.
|
||||
|
||||
Thus all parts coming from upstream repos should have a physical copy in this repo with a reference to the upstream repo. Therefore we need a mapping table from parts here to references in upstreams.
|
||||
|
||||
Alternative: Submodules
|
||||
|
||||
Proposal: The final product will be a export of the repo
|
Before Width: | Height: | Size: 295 KiB After Width: | Height: | Size: 295 KiB |
Before Width: | Height: | Size: 264 KiB After Width: | Height: | Size: 264 KiB |
0
docs/about/license.md
Normal file
133
docs/about/mkdocs-test.md
Normal file
|
@ -0,0 +1,133 @@
|
|||
# Mkdoc Test Doc
|
||||
|
||||
```plantuml
|
||||
@startuml
|
||||
title Login Sequence
|
||||
ComponentA->ComponentB: Login Request
|
||||
note right of ComponentB: ComponentB logs message
|
||||
ComponentB->ComponentA: Login Response
|
||||
@enduml
|
||||
```
|
||||
|
||||
## hello mock docs
|
||||
|
||||
!!! test
|
||||
Testing something
|
||||
|
||||
Abbreviations:
|
||||
Some text about MOCDOC
|
||||
|
||||
This is a paragraph.
|
||||
{: #test_id .test_class }
|
||||
|
||||
Apple
|
||||
: Pomaceous fruit of plants of the genus Malus in
|
||||
the family Rosaceae.
|
||||
|
||||
```javascript
|
||||
import { test } from 'something';
|
||||
|
||||
const addThingToThing = (a, b) a + b;
|
||||
```
|
||||
|
||||
- [abc](#abc)
|
||||
- [xyz](#xyz)
|
||||
|
||||
## abc
|
||||
|
||||
This is a b c.
|
||||
|
||||
## xyz
|
||||
|
||||
This is x y z.
|
||||
|
||||
# The attack plan
|
||||
|
||||
{% dot attack_plan.svg
|
||||
digraph G {
|
||||
rankdir=LR
|
||||
Earth [peripheries=2]
|
||||
Mars
|
||||
Earth -> Mars
|
||||
}
|
||||
%}
|
||||
|
||||
```graphviz dot attack_plan.svg
|
||||
digraph G {
|
||||
rankdir=LR
|
||||
Earth [peripheries=2]
|
||||
Mars
|
||||
Earth -> Mars
|
||||
}
|
||||
```
|
||||
|
||||
# PlantUML Samples
|
||||
|
||||
```plantuml classes="uml myDiagram" alt="Diagram placeholder" title="My diagram"
|
||||
@startuml
|
||||
Goofy -> MickeyMouse: calls
|
||||
Goofy <-- MickeyMouse: responds
|
||||
@enduml
|
||||
```
|
||||
|
||||
# Emojis
|
||||
|
||||
:bulb: :smile:
|
||||
|
||||
# Code blocks
|
||||
|
||||
```javascript
|
||||
import { test } from 'something';
|
||||
|
||||
const addThingToThing = (a, b) a + b;
|
||||
```
|
||||
|
||||
# Grouped Code blocks
|
||||
|
||||
=== "JavaScript"
|
||||
|
||||
```javascript
|
||||
import { test } from 'something';
|
||||
|
||||
const addThingToThing = (a, b) a + b;
|
||||
```
|
||||
|
||||
=== "Java"
|
||||
|
||||
```java
|
||||
public void function() {
|
||||
test();
|
||||
}
|
||||
```
|
||||
|
||||
```java tab="java"
|
||||
public void function() {
|
||||
test();
|
||||
}
|
||||
```
|
||||
|
||||
```java tab="java 2"
|
||||
public void function() {
|
||||
test();
|
||||
}
|
||||
```
|
||||
|
||||
# MDX truly sane lists
|
||||
|
||||
- attributes
|
||||
|
||||
- customer
|
||||
- first_name
|
||||
- test
|
||||
- family_name
|
||||
- email
|
||||
- person
|
||||
- first_name
|
||||
- family_name
|
||||
- birth_date
|
||||
- subscription_id
|
||||
|
||||
- request
|
||||
|
||||
<!-- prettier-ignore -->
|
||||
*[MOCDOC]: Mock Documentation
|
0
docs/about/release-notes.md
Normal file
35
docs/contents.md
Normal file
|
@ -0,0 +1,35 @@
|
|||
# EDP Documentation
|
||||
|
||||
The documentation consists of two parts
|
||||
|
||||
* User Documentation
|
||||
* Technical Documentation
|
||||
|
||||
Both are outlined in the following.
|
||||
|
||||
## User Documentation
|
||||
|
||||
A 'user' is someone who accesses a predeployed EDP and wants to drive developer use cases.
|
||||
|
||||
### 🧐 What is it?
|
||||
|
||||
When you enter a EDP as a development user the EDP is an Internal Development Platform.
|
||||
|
||||
### 📦 Main features
|
||||
|
||||
* Start new use cases
|
||||
* Search, browse, monitor and revoke already run use cases
|
||||
|
||||
### ⏱️ Quick start guide (WiP)
|
||||
|
||||
* Open the platform URL
|
||||
* Follow one of the use cases
|
||||
|
||||
## Technical Documentation
|
||||
|
||||
EDP is a product developed by the IPCEI-CIS subproject 'edge Developer Framework'. The goal is to provide a cutting edge developer experience for developing and delivering applications in the cloud edge continuum.
|
||||
|
||||
It consists of
|
||||
|
||||
* [edpbuilder](https://forgejo.edf-bootstrap.cx.fg1.ffm.osc.live/DevFW/edpbuilder) as a tool to quickly instantiate and manage Internal Development Platforms (IDPs).
|
||||
* declarative 'platform stack' descriptions which are processed by the orchestrator
|
1
docs/index.md
Symbolic link
|
@ -0,0 +1 @@
|
|||
../README.md
|
BIN
docs/ressources/1.png
Normal file
After Width: | Height: | Size: 503 KiB |
BIN
docs/ressources/2.png
Normal file
After Width: | Height: | Size: 292 KiB |
BIN
docs/ressources/3.png
Normal file
After Width: | Height: | Size: 217 KiB |
BIN
docs/ressources/4.png
Normal file
After Width: | Height: | Size: 303 KiB |
BIN
docs/ressources/5.png
Normal file
After Width: | Height: | Size: 319 KiB |
BIN
docs/ressources/6.png
Normal file
After Width: | Height: | Size: 535 KiB |
BIN
docs/ressources/7.png
Normal file
After Width: | Height: | Size: 591 KiB |
|
@ -0,0 +1,9 @@
|
|||
---
|
||||
title: 'Code: Software and Workloads'
|
||||
linktitle: Code
|
||||
weight: 1
|
||||
description: 'The center of everything else, the reason, driver and center of all being: Running Code'
|
||||
---
|
||||
|
||||
|
||||
> The center of everything else, the reason, driver and center of all being: Running Code
|
|
@ -0,0 +1,7 @@
|
|||
---
|
||||
title: Engineers
|
||||
weight: 2
|
||||
description: 'Our clients: People creating code and bringing it to life - and their habits and contexts'
|
||||
---
|
||||
|
||||
|
62
docs/technical-documentation/concepts/3_use-cases/_index.md
Normal file
|
@ -0,0 +1,62 @@
|
|||
---
|
||||
title: Use Cases
|
||||
weight: 2
|
||||
description: The golden paths in the engineers and product development domain
|
||||
---
|
||||
|
||||
## Rationale
|
||||
|
||||
The challenge of IPCEI-CIS Developer Framework is to provide value for DTAG customers, and more specifically: for Developers of DTAG customers.
|
||||
|
||||
That's why we need verifications - or test use cases - for the Developer Framework to develop.
|
||||
|
||||

|
||||
(source: https://tag-app-delivery.cncf.io/whitepapers/platforms/)
|
||||
|
||||
## Golden Paths as Use Cases
|
||||
|
||||
* https://platformengineering.org/blog/how-to-pave-golden-paths-that-actually-go-somewhere
|
||||
* https://thenewstack.io/using-an-internal-developer-portal-for-golden-paths/
|
||||
* https://nl.devoteam.com/expert-view/building-golden-paths-with-internal-developer-platforms/
|
||||
* https://www.redhat.com/en/blog/designing-golden-paths
|
||||
|
||||
## List of Use Cases
|
||||
|
||||
Here we have a collection of possible usage scenarios.
|
||||
|
||||
### Socksshop
|
||||
|
||||
Deploy and develop the famous socks shops:
|
||||
|
||||
* https://medium.com/@wadecharley703/socks-shop-microservices-application-deployment-on-the-cloud-cd1017cce1c0
|
||||
|
||||
* See also mkdev fork: https://github.com/mkdev-me/microservices-demo
|
||||
|
||||
### Humanitec Demos
|
||||
|
||||
* https://github.com/poc-template-org/node-js-sample
|
||||
|
||||
### Github Examples
|
||||
|
||||
* https://github.com/kezoo/nestjs-reactjs-graphql-typescript-boilerplate-example
|
||||
|
||||
### Telemetry Use Case with respect to the Fibonacci workload
|
||||
|
||||
The Fibonacci App on the cluster can be accessed on the path https://cnoe.localtest.me/fibonacci.
|
||||
It can be called for example by using the URL https://cnoe.localtest.me/fibonacci?number=5000000.
|
||||
|
||||
The resulting ressource spike can be observed one the Grafana dashboard "Kubernetes / Compute Resources / Cluster".
|
||||
The resulting visualization should look similar like this:
|
||||
|
||||
|
||||

|
||||
|
||||
|
||||
|
||||
## When and how to use the developer framework?
|
||||
|
||||
### e.g. an example
|
||||
|
||||
.... taken from https://cloud.google.com/blog/products/application-development/common-myths-about-platform-engineering?hl=en
|
||||
|
||||

|
After Width: | Height: | Size: 154 KiB |
BIN
docs/technical-documentation/concepts/3_use-cases/image.png
Normal file
After Width: | Height: | Size: 944 KiB |
After Width: | Height: | Size: 160 KiB |
|
@ -0,0 +1,10 @@
|
|||
---
|
||||
title: (Digital) Platforms
|
||||
linktitle: Platforms
|
||||
weight: 4
|
||||
description: Platforming is the discipline to provide full sophisticated golden paths to the engineers. It's the next level of DevOps.
|
||||
---
|
||||
|
||||
## Surveys
|
||||
|
||||
* [10-best-internal-developer-platforms-to-consider-in-2023/](https://www.qovery.com/blog/10-best-internal-developer-platforms-to-consider-in-2023/)
|
|
@ -0,0 +1,8 @@
|
|||
---
|
||||
title: "Platform Components"
|
||||
weight: 3
|
||||
description: What in terms of components or building blocks is needed in a platform?
|
||||
---
|
||||
|
||||
> This page is in work. Right now we have in the index a collection of links describing and listing typical components and building blocks of platforms. Also we have a growing number of subsections regarding special types of components.
|
||||
|
|
@ -0,0 +1,87 @@
|
|||
---
|
||||
title: CI/CD Pipeline
|
||||
---
|
||||
|
||||
This document describes the concept of pipelining in the context of the Edge Developer Framework.
|
||||
|
||||
## Overview
|
||||
|
||||
In order to provide a composable pipeline as part of the Edge Developer Framework (EDF), we have defined a set of concepts that can be used to create pipelines for different usage scenarios. These concepts are:
|
||||
|
||||
**Pipeline Contexts** define the context in which a pipeline execution is run. Typically, a context corresponds to a specific step within the software development lifecycle, such as building and testing code, deploying and testing code in staging environments, or releasing code. Contexts define which components are used, in which order, and the environment in which they are executed.
|
||||
|
||||
**Components** are the building blocks, which are used in the pipeline. They define specific steps that are executed in a pipeline such as compiling code, running tests, or deploying an application.
|
||||
|
||||

|
||||
|
||||
## Pipeline Contexts
|
||||
|
||||
We provide 4 Pipeline Contexts that can be used to create pipelines for different usage scenarios. The contexts can be described as the golden path, which is fully configurable and extenable by the users.
|
||||
|
||||
Pipeline runs with a given context can be triggered by different actions. For example, a pipeline run with the `Continuous Integration` context can be triggered by a commit to a repository, while a pipeline run with the `Continuous Delivery` context could be triggered by merging a pull request to a specific branch.
|
||||
|
||||
### Continuous Integration
|
||||
|
||||
This context is focused on running tests and checks on every commit to a repository. It is used to ensure that the codebase is always in a working state and that new changes do not break existing functionality. Tests within this context are typically fast and lightweight, and are used to catch simple errors such as syntax errors, typos, and basic logic errors. Static vulnerability and compliance checks can also be performed in this context.
|
||||
|
||||
### Continuous Delivery
|
||||
|
||||
This context is focused on deploying code to a (ephermal) staging environment after its static checks have been performed. It is used to ensure that the codebase is always deployable and that new changes can be easily reviewed by stakeholders. Tests within this context are typically more comprehensive than those in the Continuous Integration context, and handle more complex scenarios such as integration tests and end-to-end tests. Additionally, live security and compliance checks can be performed in this context.
|
||||
|
||||
### Continuous Deployment
|
||||
|
||||
This context is focused on deploying code to a production environment and/or publishing artefacts after static checks have been performed.
|
||||
|
||||
### Chore
|
||||
|
||||
This context focuses on measures that need to be carried out regularly (e.g. security or compliance scans). They are used to ensure the robustness, security and efficiency of software projects. They enable teams to maintain high standards of quality and reliability while minimizing risks and allowing developers to focus on more critical and creative aspects of development, increasing overall productivity and satisfaction.
|
||||
|
||||
## Components
|
||||
|
||||
Components are the composable and self-contained building blocks for the contexts described above. The aim is to cover most (common) use cases for application teams and make them particularly easy to use by following our golden paths. This way, application teams only have to include and configure the functionalities they actually need. An additional benefit is that this allows for easy extensibility. If a desired functionality has not been implemented as a component, application teams can simply add their own.
|
||||
|
||||
Components must be as small as possible and follow the same concepts of software development and deployment as any other software product. In particular, they must have the following characteristics:
|
||||
|
||||
- designed for a single task
|
||||
- provide a clear and intuitive output
|
||||
- easy to compose
|
||||
- easily customizable or interchangeable
|
||||
- automatically testable
|
||||
|
||||
In the EDF components are divided into different categories. Each category contains components that perform similar actions. For example, the `build` category contains components that compile code, while the `deploy` category contains components that automate the management of the artefacts created in a production-like system.
|
||||
|
||||
> **Note:** Components are comparable to interfaces in programming. Each component defines a certain behaviour, but the actual implementation of these actions depends on the specific codebase and environment.
|
||||
>
|
||||
> For example, the `build` component defines the action of compiling code, but the actual build process depends on the programming language and build tools used in the project. The `vulnerability scanning` component will likely execute different tools and interact with different APIs depending on the context in which it is executed.
|
||||
|
||||
### Build
|
||||
|
||||
Build components are used to compile code. They can be used to compile code written in different programming languages, and can be used to compile code for different platforms.
|
||||
|
||||
### Code Test
|
||||
|
||||
These components define tests that are run on the codebase. They are used to ensure that the codebase is always in a working state and that new changes do not break existing functionality. Tests within this category are typically fast and lightweight, and are used to catch simple errors such as syntax errors, typos, and basic logic errors. Tests must be executable in isolation, and do not require external dependencies such as databases or network connections.
|
||||
|
||||
### Application Test
|
||||
|
||||
Application tests are tests, which run the code in a real execution environment, and provide external dependencies. These tests are typically more comprehensive than those in the `Code Test` category, and handle more complex scenarios such as integration tests and end-to-end tests.
|
||||
|
||||
### Deploy
|
||||
|
||||
Deploy components are used to deploy code to different environments, but can also be used to publish artifacts. They are typically used in the `Continuous Delivery` and `Continuous Deployment` contexts.
|
||||
|
||||
### Release
|
||||
|
||||
Release components are used to create releases of the codebase. They can be used to create tags in the repository, create release notes, or perform other tasks related to releasing code. They are typically used in the `Continuous Deployment` context.
|
||||
|
||||
### Repo House Keeping
|
||||
|
||||
Repo house keeping components are used to manage the repository. They can be used to clean up old branches, update the repository's README file, or perform other maintenance tasks. They can also be used to handle issues, such as automatically closing stale issues.
|
||||
|
||||
### Dependency Management
|
||||
|
||||
Dependency management is used to automate the process of managing dependencies in a codebase. It can be used to create pull requests with updated dependencies, or to automatically update dependencies in a codebase.
|
||||
|
||||
### Security and Compliance
|
||||
|
||||
Security and compliance components are used to ensure that the codebase meets security and compliance requirements. They can be used to scan the codebase for vulnerabilities, check for compliance with coding standards, or perform other security and compliance checks. Depending on the context, different tools can be used to accomplish scanning. In the `Continuous Integration` context, static code analysis can be used to scan the codebase for vulnerabilities, while in the `Continuous Delivery` context, live security and compliance checks can be performed.
|
After Width: | Height: | Size: 732 KiB |
|
@ -0,0 +1,11 @@
|
|||
|
||||
|
||||
# Gitops changes the definition of 'Delivery' or 'Deployment'
|
||||
|
||||
We have Gitops these days .... so there is a desired state of an environment in a repo and a reconciling mechanism done by Gitops to enforce this state on the environemnt.
|
||||
|
||||
There is no continuous whatever step inbetween ... Gitops is just 'overwriting' (to avoid saying 'delivering' or 'deploying') the environment with the new state.
|
||||
|
||||
This means whatever quality ensuring steps have to take part before 'overwriting' have to be defined as state changer in the repos, not in the environments.
|
||||
|
||||
Conclusio: I think we only have three contexts, or let's say we don't have the contect 'continuous delivery'
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
title: "Developer Portals"
|
||||
weight: 2
|
||||
description: Developer portals are one part of the UI for developers to access platforms. The general idea is that the UI parts should be enough for a developer to th their work.
|
||||
---
|
||||
|
||||
> This page is in work. Right now we have in the index a collection of links describing developer portals.
|
||||
|
||||
|
||||
* Backstage (siehe auch https://nl.devoteam.com/expert-view/project-unox/)
|
||||
* [Port](https://www.getport.io/)
|
||||
* Cortex
|
||||
* Humanitec
|
||||
* [OpsLevel](https://docs.opslevel.com/docs/introducing-opslevel#what-is-opslevel)
|
||||
* https://www.configure8.io/
|
||||
* ... tbc ...
|
||||
|
||||
### Port's Comparison vs. Backstage
|
||||
|
||||
https://www.getport.io/compare/backstage-vs-port
|
||||
|
||||
### Cortex's Comparison vs. Backstage, Port, OpsLevel
|
||||
|
||||
* https://www.cortex.io/compare
|
||||
|
||||
### Service Catalogue
|
||||
|
||||
* https://humanitec.com/blog/service-catalogs-and-internal-developer-platforms
|
||||
* https://dzone.com/articles/the-differences-between-a-service-catalog-internal
|
||||
|
||||
## Links
|
||||
|
||||
* [port-vs-backstage-choosing-your-internal-developer-portal](https://medium.com/@vaibhavgupta0702/port-vs-backstage-choosing-your-internal-developer-portal-71c6a6acd979)
|
||||
* [idp-vs-self-service-portal-a-platform-engineering-showdown](https://thenewstack.io/idp-vs-self-service-portal-a-platform-engineering-showdown)
|
||||
* [portals-vs-platform-orchestrator](https://humanitec.com/portals-vs-platform-orchestrator)
|
||||
* [internal-developer-portal-vs-internal-developer-platform](https://www.cortex.io/post/internal-developer-portal-vs-internal-developer-platform)
|
|
@ -0,0 +1,23 @@
|
|||
---
|
||||
title: Platform Orchestrator
|
||||
weight: 3
|
||||
description: "The new kid on the block since 2023 ist 'Platform Orchestrating': Do the the magic declaratively cloud natively automated."
|
||||
---
|
||||
|
||||
'Platform Orchestration' is first mentionned by [Thoughtworks in Sept 2023](https://www.thoughtworks.com/en-de/radar/techniques/platform-orchestration)
|
||||
|
||||
## Links
|
||||
|
||||
* [portals-vs-platform-orchestrator](https://humanitec.com/portals-vs-platform-orchestrator)
|
||||
* [kratix.io](https://www.kratix.io/)
|
||||
* https://internaldeveloperplatform.org/platform-orchestrators/
|
||||
* [backstack.dev](https://backstack.dev/blog/)
|
||||
|
||||
### CNOE
|
||||
|
||||
* cnoe.io
|
||||
|
||||
#### Resources
|
||||
|
||||
* [CNOE IDPBuilder](https://cnoe.io/docs/reference-implementation/installations/idpbuilder)
|
||||
* https://github.com/csantanapr/cnoe-examples/tree/main
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
title: List of references
|
||||
weight: 10
|
||||
linktitle: References
|
||||
description: An currently uncurated list of references with respect to typical platform building components
|
||||
---
|
||||
|
||||
## CNCF
|
||||
|
||||
> [Here are capability domains to consider when building platforms for cloud-native computing](https://tag-app-delivery.cncf.io/whitepapers/platforms/#capabilities-of-platforms):
|
||||
|
||||
* Web portals for observing and provisioning products and capabilities
|
||||
* APIs (and CLIs) for automatically provisioning products and capabilities
|
||||
* “Golden path” templates and docs enabling optimal use of capabilities in products
|
||||
* Automation for building and testing services and products
|
||||
* Automation for delivering and verifying services and products
|
||||
* Development environments such as hosted IDEs and remote connection tools
|
||||
* Observability for services and products using instrumentation and dashboards, including observation of functionality, performance and costs
|
||||
* Infrastructure services including compute runtimes, programmable networks, and block and volume storage
|
||||
* Data services including databases, caches, and object stores
|
||||
* Messaging and event services including brokers, queues, and event fabrics
|
||||
* Identity and secret management services such as service and user identity and authorization, certificate and key issuance, and static secret storage
|
||||
* Security services including static analysis of code and artifacts, runtime analysis, and policy enforcement
|
||||
* Artifact storage including storage of container image and language-specific packages, custom binaries and libraries, and source code
|
||||
|
||||
## IDP
|
||||
|
||||
> [An Internal Developer Platform (IDP) should be built to cover 5 Core Components:](https://internaldeveloperplatform.org/core-components/)
|
||||
|
||||
| Core Component | Short Description |
|
||||
| ---- | --- |
|
||||
| Application Configuration Management | Manage application configuration in a dynamic, scalable and reliable way. |
|
||||
| Infrastructure Orchestration | Orchestrate your infrastructure in a dynamic and intelligent way depending on the context. |
|
||||
| Environment Management | Enable developers to create new and fully provisioned environments whenever needed. |
|
||||
| Deployment Management | Implement a delivery pipeline for Continuous Delivery or even Continuous Deployment (CD). |
|
||||
| Role-Based Access Control | Manage who can do what in a scalable way. |
|
After Width: | Height: | Size: 554 KiB |
|
@ -0,0 +1,72 @@
|
|||
---
|
||||
title: Platform Engineering
|
||||
weight: 1
|
||||
description: Theory and general blue prints of the platform engineering discipline
|
||||
---
|
||||
|
||||
|
||||
## Rationale
|
||||
|
||||
IPCEI-CIS Developer Framework is part of a cloud native technology stack. To design the capabilities and architecture of the Developer Framework we need to define the surounding context and internal building blocks, both aligned with cutting edge cloud native methodologies and research results.
|
||||
|
||||
In CNCF the discipline of building stacks to enhance the developer experience is called 'Platform Engineering'
|
||||
|
||||
## CNCF Platforms White Paper
|
||||
|
||||
[CNCF first asks](https://tag-app-delivery.cncf.io/whitepapers/platforms/) why we need platform engineering:
|
||||
|
||||
> The desire to refocus delivery teams on their core focus and reduce duplication of effort across the organisation has motivated enterprises to implement platforms for cloud-native computing. By investing in platforms, enterprises can:
|
||||
> * Reduce the cognitive load on product teams and thereby accelerate product development and delivery
|
||||
> * Improve reliability and resiliency of products relying on platform capabilities by dedicating experts to configure and manage them
|
||||
> * Accelerate product development and delivery by reusing and sharing platform tools and knowledge across many teams in an enterprise
|
||||
> * Reduce risk of security, regulatory and functional issues in products and services by governing platform capabilities and the users, tools and processes surrounding them
|
||||
> * Enable cost-effective and productive use of services from public clouds and other managed offerings by enabling delegation of implementations to those providers while maintaining control over user experience
|
||||
|
||||
## `platformengineering.org`'s Definition of Platform Engineering
|
||||
|
||||
> [Platform engineering is the discipline](https://platformengineering.org/blog/what-is-platform-engineering) of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era. Platform engineers provide an integrated product most often referred to as an “Internal Developer Platform” covering the operational necessities of the entire lifecycle of an application.
|
||||
|
||||
## Reference Architecture aka 'Even more wording': Internal Developer Platform vs. Developer Portal vs. Platform
|
||||
|
||||
https://humanitec.com/blog/wtf-internal-developer-platform-vs-internal-developer-portal-vs-paas
|
||||
|
||||

|
||||
|
||||
## Platform Engineering as running a restaurant
|
||||
|
||||
* https://technologyconversations.com/2024/04/29/how-platform-engineering-compares-to-running-a-restaurant/
|
||||
|
||||

|
||||
|
||||
## Internal Developer Platform
|
||||
|
||||
> In IPCEI-CIS right now (July 2024) we are primarily interested in understanding how IDPs are built as one option to implement an IDP is to build it ourselves.
|
||||
|
||||
The outcome of the Platform Engineering discipline is - created by the platform engineering team - a so called 'Internal Developer Platform'.
|
||||
|
||||
One of the first sites focusing on this discipline was [internaldeveloperplatform.org](https://internaldeveloperplatform.org/)
|
||||
|
||||
* [The golden path](https://engineering.atspotify.com/2020/08/how-we-use-golden-paths-to-solve-fragmentation-in-our-software-ecosystem/)
|
||||
|
||||
## Examples of existing IDPs
|
||||
|
||||
The amount of available IDPs as product is rapidly growing.
|
||||
|
||||
[TODO] LIST OF IDPs
|
||||
|
||||
* [internaldeveloperplatform.org - 'Ecosystem'](https://internaldeveloperplatform.org/)
|
||||
* Typical market overview: https://medium.com/@rphilogene/10-best-internal-developer-portals-to-consider-in-2023-c780fbf8ab12
|
||||
* Another one: https://www.qovery.com/blog/10-best-internal-developer-platforms-to-consider-in-2023/
|
||||
* Just found as another example: [platformplane](https://www.platformplane.com/)
|
||||
|
||||
## Additional links
|
||||
|
||||
* [how-to-fail-at-platform-engineering](https://www.cncf.io/blog/2024/03/08/how-to-fail-at-platform-engineering/)
|
||||
* [8-real-world-reasons-to-adopt-platform-engineering](https://thenewstack.io/8-real-world-reasons-to-adopt-platform-engineering/)
|
||||
* [7-core-elements-of-an-internal-developer-platform](https://thenewstack.io/7-core-elements-of-an-internal-developer-platform/)
|
||||
* [internal-developer-platform-vs-internal-developer-portal](https://www.getport.io/blog/internal-developer-platform-vs-internal-developer-portal)
|
||||
|
||||
## Platform 'Initiatives' aka Use Cases
|
||||
|
||||
Cortex is [talking about Use Cases (aka Initiatives):](https://www.youtube.com/watch?v=LrEC-fkBbQo) (or https://www.brighttalk.com/webcast/20257/601901)
|
||||

|
After Width: | Height: | Size: 1.3 MiB |
After Width: | Height: | Size: 51 KiB |
|
@ -0,0 +1,40 @@
|
|||
+++
|
||||
archetype = "sub-chapter"
|
||||
title = "Reference Architecture"
|
||||
weight = 1
|
||||
[params]
|
||||
author = 'stephan.lo@telekom.de'
|
||||
date = '2024-07-30'
|
||||
+++
|
||||
|
||||
## [The Structure of a Successful Internal Developer Platform](https://platformengineering.org/blog/create-your-own-platform-engineering-reference-architectures)
|
||||
|
||||
In a platform reference architecture there are five main planes that make up an IDP:
|
||||
|
||||
1. Developer Control Plane – this is the primary configuration layer and interaction point for the platform users. Components include Workload specifications such as Score and a portal for developers to interact with.
|
||||
2. Integration and Delivery Plane – this plane is about building and storing the image, creating app and infra configs, and deploying the final state. It usually contains a CI pipeline, an image registry, a Platform Orchestrator, and the CD system.
|
||||
3. Resource Plane – this is where the actual infrastructure exists including clusters, databases, storage or DNS services.
|
||||
4, Monitoring and Logging Plane – provides real-time metrics and logs for apps and infrastructure.
|
||||
5. Security Plane – manages secrets and identity to protect sensitive information, e.g., storing, managing, and security retrieving API keys and credentials/secrets.
|
||||
|
||||

|
||||
|
||||
(source: https://humanitec.com/blog/wtf-internal-developer-platform-vs-internal-developer-portal-vs-paas)
|
||||
|
||||
|
||||
## Humanitec
|
||||
|
||||
https://github.com/humanitec-architecture
|
||||
|
||||
https://humanitec.com/reference-architectures
|
||||
|
||||
|
||||
|
||||
## Create a reference architecture
|
||||
|
||||
[Create your own platform reference architecture](https://platformengineering.org/blog/create-your-own-platform-engineering-reference-architectures)
|
||||
|
||||
[Reference arch slide deck](https://docs.google.com/presentation/d/1yAf_FSjiA0bAFukgu5p1DRMvvGGE1fF4KhvZbb7gn2I/edit?pli=1#slide=id.g1ef66f3349b_3_3)
|
||||
|
||||
|
||||
|
157
docs/technical-documentation/concepts/5_platforms/CNOE/_index.md
Normal file
|
@ -0,0 +1,157 @@
|
|||
---
|
||||
title: CNOE
|
||||
weight: 4
|
||||
---
|
||||
|
||||
|
||||
* https://cnoe.io/docs/intro
|
||||
|
||||
> The goal for the CNOE framework is to bring together a cohort of enterprises operating at the same scale so that they can navigate their operational technology decisions together, de-risk their tooling bets, coordinate contribution, and offer guidance to large enterprises on which CNCF technologies to use together to achieve the best cloud efficiencies.
|
||||
|
||||
### Aussprache
|
||||
|
||||
* Englisch Kuh.noo,
|
||||
* also 'Kanu' im Deutschen
|
||||
|
||||
|
||||
## Architecture
|
||||
|
||||

|
||||
|
||||
## Run the CNOEs reference implementation
|
||||
|
||||
See https://cnoe.io/docs/reference-implementation/integrations/reference-impl:
|
||||
|
||||
```bash
|
||||
# in a local terminal with docker and kind
|
||||
idpbuilder create --use-path-routing --log-level debug --package-dir https://github.com/cnoe-io/stacks//ref-implementation
|
||||
```
|
||||
### Output
|
||||
|
||||
```bash
|
||||
time=2024-08-05T14:48:33.348+02:00 level=INFO msg="Creating kind cluster" logger=setup
|
||||
time=2024-08-05T14:48:33.371+02:00 level=INFO msg="Runtime detected" logger=setup provider=docker
|
||||
########################### Our kind config ############################
|
||||
# Kind kubernetes release images https://github.com/kubernetes-sigs/kind/releases
|
||||
kind: Cluster
|
||||
apiVersion: kind.x-k8s.io/v1alpha4
|
||||
nodes:
|
||||
- role: control-plane
|
||||
image: "kindest/node:v1.29.2"
|
||||
kubeadmConfigPatches:
|
||||
- |
|
||||
kind: InitConfiguration
|
||||
nodeRegistration:
|
||||
kubeletExtraArgs:
|
||||
node-labels: "ingress-ready=true"
|
||||
extraPortMappings:
|
||||
- containerPort: 443
|
||||
hostPort: 8443
|
||||
protocol: TCP
|
||||
|
||||
containerdConfigPatches:
|
||||
- |-
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."gitea.cnoe.localtest.me:8443"]
|
||||
endpoint = ["https://gitea.cnoe.localtest.me"]
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.configs."gitea.cnoe.localtest.me".tls]
|
||||
insecure_skip_verify = true
|
||||
|
||||
######################### config end ############################
|
||||
time=2024-08-05T14:48:33.394+02:00 level=INFO msg="Creating kind cluster" logger=setup cluster=localdev
|
||||
time=2024-08-05T14:48:53.680+02:00 level=INFO msg="Done creating cluster" logger=setup cluster=localdev
|
||||
time=2024-08-05T14:48:53.905+02:00 level=DEBUG+3 msg="Getting Kube config" logger=setup
|
||||
time=2024-08-05T14:48:53.908+02:00 level=DEBUG+3 msg="Getting Kube client" logger=setup
|
||||
time=2024-08-05T14:48:53.908+02:00 level=INFO msg="Adding CRDs to the cluster" logger=setup
|
||||
time=2024-08-05T14:48:53.948+02:00 level=DEBUG+3 msg="crd not yet established, waiting." "crd name"=custompackages.idpbuilder.cnoe.io
|
||||
time=2024-08-05T14:48:53.954+02:00 level=DEBUG+3 msg="crd not yet established, waiting." "crd name"=custompackages.idpbuilder.cnoe.io
|
||||
time=2024-08-05T14:48:53.957+02:00 level=DEBUG+3 msg="crd not yet established, waiting." "crd name"=custompackages.idpbuilder.cnoe.io
|
||||
time=2024-08-05T14:48:53.981+02:00 level=DEBUG+3 msg="crd not yet established, waiting." "crd name"=gitrepositories.idpbuilder.cnoe.io
|
||||
time=2024-08-05T14:48:53.985+02:00 level=DEBUG+3 msg="crd not yet established, waiting." "crd name"=gitrepositories.idpbuilder.cnoe.io
|
||||
time=2024-08-05T14:48:54.734+02:00 level=DEBUG+3 msg="Creating controller manager" logger=setup
|
||||
time=2024-08-05T14:48:54.737+02:00 level=DEBUG+3 msg="Created temp directory for cloning repositories" logger=setup dir=/tmp/idpbuilder-localdev-2865684949
|
||||
time=2024-08-05T14:48:54.737+02:00 level=INFO msg="Setting up CoreDNS" logger=setup
|
||||
time=2024-08-05T14:48:54.798+02:00 level=INFO msg="Setting up TLS certificate" logger=setup
|
||||
time=2024-08-05T14:48:54.811+02:00 level=DEBUG+3 msg="Creating/getting certificate" logger=setup host=cnoe.localtest.me sans="[cnoe.localtest.me *.cnoe.localtest.me]"
|
||||
time=2024-08-05T14:48:54.825+02:00 level=DEBUG+3 msg="Creating secret for certificate" logger=setup host=cnoe.localtest.me
|
||||
time=2024-08-05T14:48:54.832+02:00 level=DEBUG+3 msg="Running controllers" logger=setup
|
||||
time=2024-08-05T14:48:54.833+02:00 level=DEBUG+3 msg="starting manager"
|
||||
time=2024-08-05T14:48:54.833+02:00 level=INFO msg="Creating localbuild resource" logger=setup
|
||||
time=2024-08-05T14:48:54.834+02:00 level=INFO msg="Starting EventSource" controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage source="kind source: *v1alpha1.CustomPackage"
|
||||
time=2024-08-05T14:48:54.834+02:00 level=INFO msg="Starting EventSource" controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository source="kind source: *v1alpha1.GitRepository"
|
||||
time=2024-08-05T14:48:54.834+02:00 level=INFO msg="Starting Controller" controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
|
||||
time=2024-08-05T14:48:54.834+02:00 level=INFO msg="Starting Controller" controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
|
||||
time=2024-08-05T14:48:54.834+02:00 level=INFO msg="Starting EventSource" controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild source="kind source: *v1alpha1.Localbuild"
|
||||
time=2024-08-05T14:48:54.834+02:00 level=INFO msg="Starting Controller" controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
|
||||
time=2024-08-05T14:48:54.937+02:00 level=INFO msg="Starting workers" controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository "worker count"=1
|
||||
time=2024-08-05T14:48:54.937+02:00 level=INFO msg="Starting workers" controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage "worker count"=1
|
||||
time=2024-08-05T14:48:54.937+02:00 level=INFO msg="Starting workers" controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild "worker count"=1
|
||||
time=2024-08-05T14:48:56.863+02:00 level=DEBUG+3 msg=Reconciling controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild Localbuild.name=localdev namespace="" name=localdev reconcileID=cc0e5b9d-4952-4fd1-9d62-6d9821f180be resource=/localdev
|
||||
time=2024-08-05T14:48:56.863+02:00 level=DEBUG+3 msg="Create or update namespace" controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild Localbuild.name=localdev namespace="" name=localdev reconcileID=cc0e5b9d-4952-4fd1-9d62-6d9821f180be resource="&Namespace{ObjectMeta:{idpbuilder-localdev 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:NamespaceSpec{Finalizers:[],},Status:NamespaceStatus{Phase:,Conditions:[]NamespaceCondition{},},}"
|
||||
time=2024-08-05T14:48:56.983+02:00 level=DEBUG+3 msg="installing core packages" controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild Localbuild.name=localdev namespace="" name=localdev reconcileID=cc0e5b9d-4952-4fd1-9d62-6d9821f180be
|
||||
time=2024-08-05T14:
|
||||
|
||||
...
|
||||
|
||||
time=2024-08-05T14:51:04.166+02:00 level=INFO msg="Stopping and waiting for webhooks"
|
||||
time=2024-08-05T14:51:04.166+02:00 level=INFO msg="Stopping and waiting for HTTP servers"
|
||||
time=2024-08-05T14:51:04.166+02:00 level=INFO msg="Wait completed, proceeding to shutdown the manager"
|
||||
|
||||
########################### Finished Creating IDP Successfully! ############################
|
||||
|
||||
|
||||
Can Access ArgoCD at https://cnoe.localtest.me:8443/argocd
|
||||
Username: admin
|
||||
Password can be retrieved by running: idpbuilder get secrets -p argocd
|
||||
```
|
||||
|
||||
## Outcome
|
||||
|
||||
Nach ca. 10 minuten sind alle applications ausgerollt (am längsten dauert Backstage):
|
||||
|
||||

|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~$ kubectl get applications -A
|
||||
NAMESPACE NAME SYNC STATUS HEALTH STATUS
|
||||
argocd argo-workflows Synced Healthy
|
||||
argocd argocd Synced Healthy
|
||||
argocd backstage Synced Healthy
|
||||
argocd included-backstage-templates Synced Healthy
|
||||
argocd coredns Synced Healthy
|
||||
argocd external-secrets Synced Healthy
|
||||
argocd gitea Synced Healthy
|
||||
argocd keycloak Synced Healthy
|
||||
argocd metric-server Synced Healthy
|
||||
argocd nginx Synced Healthy
|
||||
argocd spark-operator Synced Healthy
|
||||
stl@ubuntu-vpn:~$ idpbuilder get secrets
|
||||
---------------------------
|
||||
Name: argocd-initial-admin-secret
|
||||
Namespace: argocd
|
||||
Data:
|
||||
password : sPMdWiy0y0jhhveW
|
||||
username : admin
|
||||
---------------------------
|
||||
Name: gitea-credential
|
||||
Namespace: gitea
|
||||
Data:
|
||||
password : |iJ+8gG,(Jj?cc*G>%(i'OA7@(9ya3xTNLB{9k'G
|
||||
username : giteaAdmin
|
||||
---------------------------
|
||||
Name: keycloak-config
|
||||
Namespace: keycloak
|
||||
Data:
|
||||
KC_DB_PASSWORD : ES-rOE6MXs09r+fAdXJOvaZJ5I-+nZ+hj7zF
|
||||
KC_DB_USERNAME : keycloak
|
||||
KEYCLOAK_ADMIN_PASSWORD : BBeMUUK1CdmhKWxZxDDa1c5A+/Z-dE/7UD4/
|
||||
POSTGRES_DB : keycloak
|
||||
POSTGRES_PASSWORD : ES-rOE6MXs09r+fAdXJOvaZJ5I-+nZ+hj7zF
|
||||
POSTGRES_USER : keycloak
|
||||
USER_PASSWORD : RwCHPvPVMu+fQM4L6W/q-Wq79MMP+3CN-Jeo
|
||||
```
|
||||
|
||||
### login to backstage
|
||||
|
||||
login geht mit den Creds, siehe oben:
|
||||
|
||||

|
BIN
docs/technical-documentation/concepts/5_platforms/CNOE/cnoe.png
Normal file
After Width: | Height: | Size: 113 KiB |
After Width: | Height: | Size: 364 KiB |
After Width: | Height: | Size: 208 KiB |
|
@ -0,0 +1,7 @@
|
|||
---
|
||||
title: Humanitec
|
||||
weight: 4
|
||||
---
|
||||
|
||||
|
||||
tbd
|
12
docs/technical-documentation/concepts/5_platforms/_index.md
Normal file
|
@ -0,0 +1,12 @@
|
|||
---
|
||||
title: Platform Orchestrators
|
||||
linktitle: Orchestrators
|
||||
weight: 5
|
||||
description: Platform automation is the next level of platforming
|
||||
---
|
||||
|
||||
|
||||
* [CNOE](./CNOE/_index.md)
|
||||
* [Humanitec](./Humanitec/_index.md)
|
||||
* [Kratix](./Kratix/_index.md)
|
||||
* [BACKstack]()
|
7
docs/technical-documentation/concepts/_index.md
Normal file
|
@ -0,0 +1,7 @@
|
|||
---
|
||||
title: Concepts
|
||||
weight: 1
|
||||
description: The underlying platforming concepts of the Edge Developer Framework (EDF) solution, i.e. the problem domain
|
||||
---
|
||||
|
||||
|
126
docs/technical-documentation/decisions/0001-pipeline-tools.md
Normal file
|
@ -0,0 +1,126 @@
|
|||
# CI/CD pipeline tools for composable pipeline
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
In order to build a composable pipeline that provides a golden path and reusable components, we need to define the tools that will be used to execute the pipeline.
|
||||
|
||||
ArgoCD is considered set in stone as the tool to manage the deployment of applications. However, the tools to compose and execute the pipeline are still up for debate.
|
||||
|
||||
> Note: The pipeline will use many other tools to perform certain actions such as testing, building, and deploying. This ADR is focused on the tools that will be used to compose and execute the pipeline itself.
|
||||
|
||||
In general, there are 2 decisions to make:
|
||||
|
||||
- What tools should we use to execute the pipeline?
|
||||
- What tools should we use to compose the pipeline?
|
||||
|
||||
The following use-cases should be considered for this decision:
|
||||
|
||||
- **User who wants to manage their own runners (???)**
|
||||
- User who only wants to use our golden path
|
||||
- User who wants to use our golden path and add custom actions
|
||||
- User who wants to use their own templates and import some of our actions
|
||||
- User who wants to import an existing GitHub repository with a pipeline
|
||||
|
||||
## Considered Options
|
||||
|
||||
- Argo Workflows + Events
|
||||
- Argo Workflows + Events + Additional Composition tool
|
||||
- Forgejo Actions
|
||||
- Forgejo Actions + Additional Composition tool
|
||||
- Dagger (as Engine)
|
||||
- Shuttle (as Engine)
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
TBD
|
||||
|
||||
## Pros and Cons of the Options
|
||||
|
||||
### Argo Workflows + Events
|
||||
|
||||
**Pro**
|
||||
|
||||
- integration with ArgoCD
|
||||
- ability to trigger additional workflows based on events.
|
||||
- level of maturity and community support.
|
||||
|
||||
**Con**
|
||||
|
||||
- Ability to self-host runners?
|
||||
- way how composition for pipelines works (based on Kubernetes CRDs)
|
||||
- Templates must be available in the cluster where the pipelines are executed, so any imported templates must be applied into the cluster before the pipeline can be executed and cannot simply reference a repository
|
||||
- This makes it difficult to import existing templates from other repositories when using self-hosted runners
|
||||
- This also makes it difficult to use our golden path, or at least we will need to provide a way to import our golden path into the cluster
|
||||
- This also makes the split of every component has its own repo very difficult
|
||||
- additional UI to manage the pipeline
|
||||
- Additional complexity
|
||||
|
||||
### Argo Workflows + Events + Additional Composition tool
|
||||
|
||||
**Pro**
|
||||
|
||||
- Composability can be offloaded to another tool
|
||||
|
||||
**Con**
|
||||
|
||||
- All cons of the previous option (except composability)
|
||||
- Additional complexity by adding another tool
|
||||
|
||||
### Forgejo Actions
|
||||
|
||||
**Pro**
|
||||
|
||||
- tight integration with GitHub Actions providing a familiar interface for developers and a vast catalog of actions to choose from
|
||||
- ability to compose pipelines without relying on another tool
|
||||
- Self-hosting of runners possible
|
||||
- every component can have its own repository and use different tools (e.g. written in go, bash, python etc.)
|
||||
|
||||
**Con**
|
||||
|
||||
- level of maturity - will require additional investments to provide a production-grade system
|
||||
|
||||
### Forgejo Actions + Additional Tool
|
||||
|
||||
**Pro**
|
||||
|
||||
- may be possible to use GitHub actions alongside another tool
|
||||
|
||||
**Con**
|
||||
|
||||
- additional complexity by adding another tool
|
||||
|
||||
### Shuttle
|
||||
|
||||
**Pro**
|
||||
|
||||
- Possibility to clearly define interfaces for pipeline steps
|
||||
- Relatively simple
|
||||
|
||||
**Con**
|
||||
|
||||
- basically backed by only one company
|
||||
- **centralized templates**, so no mechanism for composing pipelines from multiple repositories
|
||||
|
||||
### Dagger
|
||||
|
||||
**Pro**
|
||||
|
||||
- Pipeline as code
|
||||
- if it runs it should run anywhere and produce the "same" / somewhat stable results
|
||||
- build environments are defined within containers / the dagger config. Dagger is the only dependency one has to install on a machine
|
||||
- DX is extremely nice, especially if you have to debug (image) builds, also type safety due to the ability to code your build in a strong language
|
||||
- additional tooling, like trivy, is added to a build pipeline with low effort due to containers and existing plugin/wrappers
|
||||
- you can create complex test environments similar to test containers and docker compose
|
||||
|
||||
**Con**
|
||||
|
||||
- relies heavily containers, which might not be available some environments (due to policy etc), it also has an effect on reproducibility and verifiability
|
||||
- as a dev you need to properly understand containers
|
||||
- dagger engine has to run privileged locally and/or in the cloud which might be a blocker or at least a big pain in the ...
|
||||
|
||||
**Suggestion Patrick**
|
||||
|
||||
- dagger is a heavy weight and might not be as productive in a dev workflow as it seems (setup lsp etc)
|
||||
- it might be too opinionated to force on teams, especially since it is not near mainstream enough, community might be too small
|
||||
- it feels like dagger gets you 95% of the way, but the remaining 5% are a real struggle
|
||||
- if we like it, we should check the popularity in the dev community before further considering as it has a direct impact on teams and their preferences
|
5
docs/technical-documentation/decisions/README.md
Normal file
|
@ -0,0 +1,5 @@
|
|||
# ADRs
|
||||
|
||||
Architecture Decision Records (ADRs) are a way to capture the important architectural decisions made during the development of a project. They are a way to document the context, the decision, and the consequences of the decision. They are a way to keep track of the architectural decisions made in a project and to communicate them to the team.
|
||||
|
||||
The [Markdown Architectural Decision Records](https://adr.github.io/madr/) (MADR) format is a simple and easy-to-use format for writing ADRs in Markdown.
|
67
docs/technical-documentation/decisions/_adr-template.md
Normal file
|
@ -0,0 +1,67 @@
|
|||
<!-- we need to disable MD025, because we use the different heading "ADR Template" in the homepage (see above) than it is foreseen in the template -->
|
||||
<!-- markdownlint-disable-next-line MD025 -->
|
||||
# {short title, representative of solved problem and found solution}
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
{Describe the context and problem statement, e.g., in free form using two to three sentences or in the form of an illustrative story. You may want to articulate the problem in form of a question and add links to collaboration boards or issue management systems.}
|
||||
|
||||
<!-- This is an optional element. Feel free to remove. -->
|
||||
## Decision Drivers
|
||||
|
||||
* {decision driver 1, e.g., a force, facing concern, …}
|
||||
* {decision driver 2, e.g., a force, facing concern, …}
|
||||
* … <!-- numbers of drivers can vary -->
|
||||
|
||||
## Considered Options
|
||||
|
||||
* {title of option 1}
|
||||
* {title of option 2}
|
||||
* {title of option 3}
|
||||
* … <!-- numbers of options can vary -->
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
Chosen option: "{title of option 1}", because {justification. e.g., only option, which meets k.o. criterion decision driver | which resolves force {force} | … | comes out best (see below)}.
|
||||
|
||||
<!-- This is an optional element. Feel free to remove. -->
|
||||
### Consequences
|
||||
|
||||
* Good, because {positive consequence, e.g., improvement of one or more desired qualities, …}
|
||||
* Bad, because {negative consequence, e.g., compromising one or more desired qualities, …}
|
||||
* … <!-- numbers of consequences can vary -->
|
||||
|
||||
<!-- This is an optional element. Feel free to remove. -->
|
||||
### Confirmation
|
||||
|
||||
{Describe how the implementation of/compliance with the ADR can/will be confirmed. Are the design that was decided for and its implementation in line with the decision made? E.g., a design/code review or a test with a library such as ArchUnit can help validate this. Not that although we classify this element as optional, it is included in many ADRs.}
|
||||
|
||||
<!-- This is an optional element. Feel free to remove. -->
|
||||
## Pros and Cons of the Options
|
||||
|
||||
### {title of option 1}
|
||||
|
||||
<!-- This is an optional element. Feel free to remove. -->
|
||||
{example | description | pointer to more information | …}
|
||||
|
||||
* Good, because {argument a}
|
||||
* Good, because {argument b}
|
||||
<!-- use "neutral" if the given argument weights neither for good nor bad -->
|
||||
* Neutral, because {argument c}
|
||||
* Bad, because {argument d}
|
||||
* … <!-- numbers of pros and cons can vary -->
|
||||
|
||||
### {title of other option}
|
||||
|
||||
{example | description | pointer to more information | …}
|
||||
|
||||
* Good, because {argument a}
|
||||
* Good, because {argument b}
|
||||
* Neutral, because {argument c}
|
||||
* Bad, because {argument d}
|
||||
* …
|
||||
|
||||
<!-- This is an optional element. Feel free to remove. -->
|
||||
## More Information
|
||||
|
||||
{You might want to provide additional evidence/confidence for the decision outcome here and/or document the team agreement on the decision and/or define when/how this decision the decision should be realized and if/when it should be re-visited. Links to other decisions and resources might appear here as well.}
|
6
docs/technical-documentation/project/_index.md
Normal file
|
@ -0,0 +1,6 @@
|
|||
---
|
||||
title: Project
|
||||
weight: 5
|
||||
description: How we organize work and proceed as team, which decisions we made, what outputs and outcomes we have
|
||||
---
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
---
|
||||
title: Bootstrapping Infrastructure
|
||||
weight: 30
|
||||
description: The cluster and the installed applications in the bootstrapping cluster
|
||||
---
|
||||
|
||||
In order to be able to do useful work, we do need a number of applications right away. We're deploying these manually so we have the necessary basis for our work. Once the framework has been developed far enough, we will deploy this infrastructure with the framework itself.
|
|
@ -0,0 +1,84 @@
|
|||
---
|
||||
title: Backup of the Bootstrapping Cluster
|
||||
weight: 30
|
||||
description: Backup and Restore of the Contents of the Bootstrapping Cluster
|
||||
---
|
||||
|
||||
## Velero
|
||||
|
||||
We are using [Velero](https://velero.io/) for backup and restore of the deployed applications.
|
||||
|
||||
## Installing Velero Tools
|
||||
|
||||
To manage a Velero install in a cluster, you need to have Velero command line tools installed locally. Please follow the instructions for [Basic Install](https://velero.io/docs/v1.9/basic-install).
|
||||
|
||||
## Setting Up Velero For a Cluster
|
||||
|
||||
Installing and configuring Velero for a cluster can be accomplished with the CLI.
|
||||
|
||||
1. Create a file with the credentials for the S3 compatible bucket that is storing the backups, for example `credentials.ini`.
|
||||
|
||||
```ini
|
||||
[default]
|
||||
aws_access_key_id = "Access Key Value"
|
||||
aws_secret_access_key = "Secret Key Value"
|
||||
```
|
||||
|
||||
2. Install Velero in the cluster
|
||||
|
||||
```
|
||||
velero install \
|
||||
--provider aws \
|
||||
--plugins velero/velero-plugin-for-aws:v1.2.1 \
|
||||
--bucket osc-backup \
|
||||
--secret-file ./credentials.ini \
|
||||
--use-volume-snapshots=false \
|
||||
--use-node-agent=true \
|
||||
--backup-location-config region=minio,s3ForcePathStyle="true",s3Url=https://obs.eu-de.otc.t-systems.com
|
||||
```
|
||||
|
||||
3. Delete `credentials.ini`, it is not needed anymore (a secret has been created in the cluster).
|
||||
4. Create a schedule to back up the relevant resources in the cluster:
|
||||
```
|
||||
velero schedule create devfw-bootstrap --schedule="23 */2 * * *" "--include-namespaces=forgejo"
|
||||
```
|
||||
|
||||
## Working with Velero
|
||||
|
||||
You can now use Velero to create backups, restore them, or perform other operations. Please refer to the [Velero Documentation](https://velero.io/docs/main/backup-reference/).
|
||||
|
||||
To list all currently available backups:
|
||||
```
|
||||
velero backup get
|
||||
```
|
||||
|
||||
## Setting up a Service Account for Access to the OTC Object Storage Bucket
|
||||
|
||||
We are using the S3-compatible Open Telekom Cloud Object Storage service to make available some storage for the backups. We picked OTC specifically because we're not using it for anything else, so no matter what catastrophy we create in Open Sovereign Cloud, the backups should be safe.
|
||||
|
||||
### Create an Object Storage Service Bucket
|
||||
|
||||
1. Log in to the [OTC Console with the correct tenant](https://auth.otc.t-systems.com/authui/federation/websso?domain_id=81e7dbd7ec9f4b03a58120dfaa61d3db&idp=ADFS_MMS_OTC00000000001000113497&protocol=saml).
|
||||
1. Navigate to [Object Storage Service](https://console.otc.t-systems.com/obs/?agencyId=WEXsFwkkVdGYULIrZT1icF4nmHY1dgX2®ion=eu-de&locale=en-us#/obs/manager/buckets).
|
||||
1. Click Create Bucket in the upper right hand corner. Give your bucket a name. No further configuration should be necessary.
|
||||
|
||||
### Create a Service User to Access the Bucket
|
||||
|
||||
1. Log in to the [OTC Console with the correct tenant](https://auth.otc.t-systems.com/authui/federation/websso?domain_id=81e7dbd7ec9f4b03a58120dfaa61d3db&idp=ADFS_MMS_OTC00000000001000113497&protocol=saml).
|
||||
1. Navigate to [Identity and Access Management](https://console.otc.t-systems.com/iam/?agencyId=WEXsFwkkVdGYULIrZT1icF4nmHY1dgX2®ion=eu-de&locale=en-us#/iam/users).
|
||||
1. Navigate to User Groups, and click Create User Group in the upper right hand corner.
|
||||
1. Enter a suitable name ("OSC Cloud Backup") and click OK.
|
||||
1. In the group list, locate the group just created and click its name.
|
||||
1. Click Authorize to add the necessary roles. Enter "OBS" in the search box to filter for Object Storage roles.
|
||||
1. Select "OBS OperateAccess", if there are two roles, select them both.
|
||||
1. **2024-10-15** Also select the "OBS Administrator" role. It is unclear why the "OBS OperateAccess" role is not sufficient, but without the admin role, the service user will not have write access to the bucket.
|
||||
1. Click Next to save the roles, then click OK to confirm, then click Finish.
|
||||
1. Navigate to Users, and click Create User in the upper right hand corner.
|
||||
1. Give the user a sensible name ("ipcei-cis-devfw-osc-backups").
|
||||
1. Disable Management console access
|
||||
1. Enable Access key, disable Password, disable Login protection.
|
||||
1. Click Next
|
||||
1. Pick the group created earlier.
|
||||
1. Download the access key when prompted.
|
||||
|
||||
The access key is a CSV file with the Access Key and the Secret Key listed in the second line.
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
title: Introduction
|
||||
weight: 1
|
||||
description: The 5-step storyflow of this Onboarding chapter
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
This onboarding section is for you when are new to IPCEI-CIS subproject 'Edge Developer Framework (EDF)' and you want to know about
|
||||
* its context to 'Platform Engineering'
|
||||
* and why we think it's the stuff we need to care about in the EDF
|
||||
|
||||
## Storyline of our current project plan (2024)
|
||||
|
||||
1. We have the ['Edge Developer Framework'](./edgel-developer-framework/)
|
||||
2. We think the solution for EDF is in relation to ['Platforming' (Digital Platforms)](./platforming/)
|
||||
1. The next evolution after DevOps
|
||||
2. Gartner predicts 80% of SWE companies to have platforms in 2026
|
||||
3. Platforms have a history since roundabout 2019
|
||||
4. CNCF has a working group which created capabilities and a maturity model
|
||||
3. Platforms evolve - nowadys there are [Platform Orchestrators](./orchestrators/)
|
||||
1. Humanitec set up a Reference Architecture
|
||||
2. There is this 'Orchestrator' thing - declaratively describe, customize and change platforms!
|
||||
4. Mapping our assumptions to the [CNOE solution](./cnoe/)
|
||||
1. CNOE is a hot candidate to help and fulfill our platform building
|
||||
2. CNOE aims to embrace change and customization!
|
||||
5. [Showtime CNOE](./cnoe-showtime/)
|
||||
|
||||
|
||||
## Please challenge this story!
|
||||
|
||||
Please do not think this story and the underlying assumptions are carved in stone!
|
||||
|
||||
1. Don't miss to further investigate and truely understand [**EDF specification needs**](./edgel-developer-framework/)
|
||||
2. Don't miss to further investigate and truely understand [**Platform capabilities on top of DevOps**](./platforming/)
|
||||
3. Don't miss to further investigate and truely understand [**Platform orchestration**](./orchestrators/)
|
||||
3. Don't miss to further investigate and truely understand [**specific orchestratiing solutions like CNOE**](./cnoe/)
|
||||
|
||||
## Your role as 'Framework Engineer' in the Domain Architecture
|
||||
|
||||
Pls be aware of the the following domain and task structure of our mission:
|
||||
|
||||

|
||||
|
||||
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
---
|
||||
title: Edge Developer Framework
|
||||
weight: 2
|
||||
description: Driving requirements for a platform
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
The 'Edge Developer Framework' is both the project and the product we are working for. Out of the leading 'Portfolio Document'
|
||||
we derive requirements which are ought to be fulfilled by Platform Engineering.
|
||||
|
||||
**This is our claim!**
|
||||
|
||||
|
||||
## What are the specifications we know from the IPCEI-CIS Project Portfolio document
|
||||
|
||||
> Reference: IPCEI-CIS Project Portfolio
|
||||
> Version 5.9, November 17, 2023
|
||||
|
||||
### DTAG´s IPCEI-CIS Project Portfolio (p.12)
|
||||
|
||||
e. Development of DTAG/TSI Edge Developer Framework
|
||||
|
||||
* Goal: All developed innovations must be accessible to developer communities in a **highly user-friendly and easy way**
|
||||
|
||||
### Development of DTAG/TSI Edge Developer Framework (p.14)
|
||||
| capability | major novelties |||
|
||||
| -- | -- | -- | -- |
|
||||
| e.1. Edge Developer full service framework (SDK + day1 +day2 support for edge installations) | Adaptive CI/CD pipelines for heterogeneous edge environments | Decentralized and self healing deployment and management | edge-driven monitoring and analytics |
|
||||
| e.2. Built-in sustainability optimization in Edge developer framework | sustainability optimized edge-aware CI/CD tooling | sustainability-optimized configuration management | sustainability-optimized efficient deployment strategies |
|
||||
| e.3. Sustainable-edge management-optimized user interface for edge developers | sustainability optimized User Interface | Ai-Enabled intelligent experience | AI/ML-based automated user experience testing and optimization |
|
||||
|
||||
### DTAG objectives & contributions (p.27)
|
||||
|
||||
DTAG will also focus on developing an easy-to-use **Edge Developer framework for software
|
||||
developers** to **manage the whole lifecycle of edge applications**, i.e. for **day-0-, day-1- and up to day-2-
|
||||
operations**. With this DTAG will strongly enable the ecosystem building for the entire IPCEI-CIS edge to
|
||||
cloud continuum and ensure openness and accessibility for anyone or any company to make use and
|
||||
further build on the edge to cloud continuum. Providing the use of the tool framework via an open-source approach will further reduce entry barriers and enhance the openness and accessibility for anyone or
|
||||
any organization (see innovations e.).
|
||||
|
||||
### WP Deliverables (p.170)
|
||||
|
||||
e.1 Edge developer full-service framework
|
||||
|
||||
This tool set and related best practices and guidelines will **adapt, enhance and further innovate DevOps principles** and
|
||||
their related, necessary supporting technologies according to the specific requirements and constraints associated with edge or edge cloud development, in order to keep the healthy and balanced innovation path on both sides,
|
||||
the (software) development side and the operations side in the field of DevOps.
|
||||
|
||||
{{% pageinfo color="info" %}}
|
||||
### What comes next?
|
||||
|
||||
[Next](../platforming/) we'll see how these requirements seem to be fulfilled by platforms!
|
||||
{{% /pageinfo %}}
|
|
@ -0,0 +1,109 @@
|
|||
---
|
||||
title: Platform Engineering aka Platforming
|
||||
linktitle: Platforming
|
||||
weight: 3
|
||||
description: DevOps is dead - long live next level DevOps in platforms
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Since 2010 we have DevOps. This brings increasing delivery speed and efficiency at scale.
|
||||
But next we got high 'cognitive loads' for developers and production congestion due to engineering lifecycle complexity.
|
||||
So we need on top of DevOps an instrumentation to ensure and enforce speed, quality, security in modern, cloud native software development. This instrumentation is called 'golden paths' in intenal develoepr platforms (IDP).
|
||||
|
||||
|
||||
## History of Platform Engineering
|
||||
|
||||
Let's start with a look into the history of platform engineering. A good starting point is [Humanitec](https://humanitec.com/), as they nowadays are one of the biggest players (['the market leader in IDPs.'](https://internaldeveloperplatform.org/#how-we-curate-this-site)) in platform engineering.
|
||||
|
||||
They create lots of [beautiful articles and insights](https://humanitec.com/blog), their own [platform products](https://humanitec.com/products/) and [basic concepts for the platform architecture](https://humanitec.com/platform-engineering) (we'll meet this later on!).
|
||||
|
||||
<img src="./humanitec-history.png" width="600" alt="https://platformengineering.org/blog/the-story-of-platform-engineering">
|
||||
|
||||
|
||||
### Further nice reference to the raise of platforms
|
||||
|
||||
* [What we **call** a Platform](https://martinfowler.com/articles/talk-about-platforms.html)
|
||||
* [Platforms and the **cloud native** connection](https://developers.redhat.com/articles/2024/05/06/)what-platform-engineering-and-why-do-we-need-it#why_we_need_platform_engineering
|
||||
* [Platforms and **microservices**](https://orkohunter.net/blog/a-brief-history-of-platform-engineering)
|
||||
* [Platforms in the **product** perspective](https://softwareengineeringdaily.com/2020/02/13/setting-the-stage-for-platform-engineering/)
|
||||
|
||||
|
||||
## Benefit of Platform Engineering, Capabilities
|
||||
|
||||
In [The Evolution of Platform Engineering](https://www.linkedin.com/pulse/evolution-platform-engineering-gaurav-goel) the interconnection inbetween DevOps, Cloud Native, and the Rise of Platform Engineering is nicely presented and summarizes:
|
||||
|
||||
> Platform engineering can be broadly defined as the discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations
|
||||
|
||||
When looking at these 'capabilities', we have CNCF itself:
|
||||
|
||||
### CNCF Working group / White paper defines layerwed capabilities
|
||||
|
||||
There is a CNCF working group which provides the definition of [Capabilities of platforms](https://tag-app-delivery.cncf.io/whitepapers/platforms/#capabilities-of-platforms) and shows a first idea of the layered architecture of platforms as **service layer for developers** ("product and application teams"):
|
||||
|
||||
<img src="./platforms-def.drawio.png" width="600">
|
||||
|
||||
|
||||
> Important: As Platform engineer also notice the [platform-eng-maturity-model](https://tag-app-delivery.cncf.io/whitepapers/platform-eng-maturity-model/)
|
||||
|
||||
### Platform Engineering Team
|
||||
|
||||
Or, in another illustration for the platform as a developer service interface, which also defines the **'Platform Engineering Team'** inbetween:
|
||||
|
||||
<img src="./platform-self-services.webp" width="600" alt="https://medium.com/@bijit211987/what-is-platform-engineering-and-how-it-reduce-cognitive-load-on-developers-ac7805603925">
|
||||
|
||||
## How to set up Platforms
|
||||
|
||||
As we now have evidence about the nescessity of platforms, their capabilities and benefit as self service layer for developers, we want to thin about how to build them.
|
||||
|
||||
First of all some important wording to motivate the important term 'internal developer platfoms' (we will go into this deeper in the next section [with the general architecture](../orchestrators/)), which is clear today, but took years to evolve and [is still some amount if effort to jump in](https://humanitec.com/blog/wtf-internal-developer-platform-vs-internal-developer-portal-vs-paas):
|
||||
|
||||
* Platform: As defined above: A product which serves software engineering teams
|
||||
* Platform Engineering: Creating such a product
|
||||
* Internal Developer Platforms (IDP): A platform for developers :-)
|
||||
* Internal Developer Portal: The entry point of a developer to such an IDP
|
||||
|
||||
### CNCF mapping from capabilities to (CNCF) projects/tools
|
||||
|
||||
[Capabilities of platforms](https://tag-app-delivery.cncf.io/whitepapers/platforms/#capabilities-of-platforms)
|
||||
|
||||
### Ecosystems in InternalDeveloperPlatform
|
||||
|
||||
Build or buy - this is also in pltaform engineering a tweaked discussion, which one of the oldest player answers like this with some oppinioated internal capability structuring:
|
||||
|
||||
[internaldeveloperplatform.org[(https://internaldeveloperplatform.org/platform-tooling/)
|
||||
|
||||
|
||||
{{% pageinfo color="info" %}}
|
||||
### What comes next?
|
||||
|
||||
[Next](../orchestrators/) we'll see how these concepts got structured!
|
||||
{{% /pageinfo %}}
|
||||
|
||||
## Addendum
|
||||
|
||||
### Digital Platform defintion from [What we **call** a Platform](https://martinfowler.com/articles/talk-about-platforms.html)
|
||||
|
||||
> Words are hard, it seems. ‘Platform’ is just about the most ambiguous term we could use for an approach that is super-important for increasing delivery speed and efficiency at scale. Hence the title of this article, here is what I’ve been talking about most recently.
|
||||
\
|
||||
Definitions for software and hardware platforms abound, generally describing an operating environment upon which applications can execute and which provides reusable capabilities such as file systems and security.
|
||||
\
|
||||
Zooming out, at an organisational level a ‘digital platform’ has similar characteristics - an operating environment which teams can build upon to deliver product features to customers more quickly, supported by reusable capabilities.
|
||||
\
|
||||
A digital platform is a foundation of self-service APIs, tools, services, knowledge and support which are arranged as a compelling internal product. Autonomous delivery teams can make use of the platform to deliver product features at a higher pace, with reduced co-ordination.
|
||||
|
||||
### Myths :-) - What are platforms _not_
|
||||
|
||||
[common-myths-about-platform-engineering](https://cloud.google.com/blog/products/application-development/common-myths-about-platform-engineering?hl=en)
|
||||
|
||||
### Platform Teams
|
||||
|
||||
This is about you :-), the platform engineering team:
|
||||
|
||||
[how-to-build-your-platform-engineering-team](https://platformengineering.org/blog/how-to-build-your-platform-engineering-team)
|
||||
|
||||
#### in comparison: devops vs sre vs platform
|
||||
|
||||
https://www.qovery.com/blog/devops-vs-platform-engineering-is-there-a-difference/
|
||||
|
||||

|
After Width: | Height: | Size: 904 KiB |
After Width: | Height: | Size: 37 KiB |
After Width: | Height: | Size: 160 KiB |
After Width: | Height: | Size: 600 KiB |
|
@ -0,0 +1,51 @@
|
|||
---
|
||||
title: Orchestrators
|
||||
weight: 4
|
||||
description: Next level platforming is orchestrating platforms
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
When defining and setting up platforms next two intrinsic problems arise:
|
||||
1. it is not declarative and automated
|
||||
2. it is not or least not easily changable
|
||||
|
||||
Thus the technology of 'Platform Orchestrating' emerged recently, in late 2023.
|
||||
|
||||
|
||||
## Platform reference architecture
|
||||
|
||||
An interesting difference between the CNCF white paper building blocks and them from Internaldeveloperplatforms is the part [**orchestrators**](https://internaldeveloperplatform.org/platform-orchestrators/).
|
||||
|
||||
This is something extremely new since late 2023 - the rise of the automation of platform engineering!
|
||||
|
||||
It was Humanitec creating a definition of platform architecture, as they needed to defien what they are building with their 'orchestrator':
|
||||
|
||||
<img src="./vendor-neutral-idp-final.gif" width="600" alt="https://developer.humanitec.com/introduction/overview/">
|
||||
|
||||
## Declarative Platform Orchestration
|
||||
|
||||
Based on the refence architecture you next can build - or let's say 'orchestrate' - different platform implementations, based on declarative definitions of the platform design.
|
||||
|
||||
https://humanitec.com/reference-architectures
|
||||
|
||||
<img src="./platform-architectures.webp" width="600" alt="https://humanitec.com/blog/aws-azure-and-gcp-open-source-reference-architectures-to-start-your-mvp">
|
||||
|
||||
> Hint: There is a [slides tool provided by McKinsey](https://platformengineering.org/blog/create-your-own-platform-engineering-reference-architectures) to set up your own platform deign based on the reference architecture
|
||||
|
||||
|
||||
{{% pageinfo color="info" %}}
|
||||
### What comes next?
|
||||
|
||||
[Next](../cnoe/) we'll see how we are going to do platform orchestration with CNOE!
|
||||
{{% /pageinfo %}}
|
||||
|
||||
## Addendum
|
||||
|
||||
## Building blocks from Humanitecs 'state-of-platform-engineering-report-volume-2'
|
||||
|
||||
You remember the [capability mappings from the time before orchestration](../platforming)? Here we have a [similar setup based on Humanitecs platform engineering status ewhite paper](https://humanitec.com/whitepapers/state-of-platform-engineering-report-volume-2):
|
||||
|
||||
<img src="./platform-tooling-humanitec-platform-report-2024.PNG" width="600" alt="https://humanitec.com/whitepapers/state-of-platform-engineering-report-volume-2 Whitepaper_ State of Platform Engineering Report.pdf">
|
||||
|
||||
|
After Width: | Height: | Size: 98 KiB |
After Width: | Height: | Size: 723 KiB |
After Width: | Height: | Size: 397 KiB |
|
@ -0,0 +1,59 @@
|
|||
---
|
||||
title: CNOE
|
||||
weight: 5
|
||||
description: Our top candidate for a platform orchestrator
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
In late 2023 platform orchestration raised - the discipline of declarativley dinfing, building, orchestarting and reconciling building blocks of (digital) platforms.
|
||||
|
||||
The famost one ist the platform orchestrator from Humanitec. They provide lots of concepts and access, also open sourced tools and schemas. But they do not have open sourced the ocheastartor itself.
|
||||
|
||||
Thus we were looking for open source means for platform orchestrating and found [CNOE](https://cnoe.io).
|
||||
|
||||
## Requirements for an Orchestrator
|
||||
|
||||
When we want to set up a [complete platform](../platforming/platforms-def.drawio.png) we expect to have
|
||||
* a **schema** which defines the platform, its ressources and internal behaviour
|
||||
* a **dynamic configuration or templating mechanism** to provide a concrete specification of a platform
|
||||
* a **deployment mechanism** to deploy and reconcile the platform
|
||||
|
||||
This is what [CNOE delivers](https://cnoe.io/docs/intro/approach):
|
||||
|
||||
> For seamless transition into a CNOE-compliant delivery pipeline, CNOE will aim at delivering **"packaging specifications"**, **"templating mechanisms"**, as well as **"deployer technologies"**, an example of which is enabled via the idpBuilder tool we have released. The combination of templates, specifications, and deployers allow for bundling and then unpacking of CNOE recommended tools into **a user's DevOps environment**. This enables teams to share and deliver components that are deemed to be the best tools for the job.
|
||||
|
||||
## CNOE (capabilities) architecture
|
||||
|
||||
### Architecture
|
||||
|
||||
CNOE architecture looks a bit different than the reference architecture from Humanitec, but this just a matter of details and arrangement:
|
||||
|
||||
> Hint: **This has to be further investigated!** This is subject to an Epic.
|
||||
|
||||
<img src="./cnoe-architecture.png" width="600" alt="https://cnoe.io/">
|
||||
|
||||
### Capabilities
|
||||
|
||||
You have a [definition of all the capabilities here](https://cnoe.io/docs/category/technology-capabilities):
|
||||
|
||||
> Hint: **This has to be further investigated!** This is subject to an Epic.
|
||||
|
||||
<img src="./cnoe-capabilities.png" width="600" alt="https://cnoe.io/docs/category/technology-capabilities">
|
||||
|
||||
## Stacks
|
||||
|
||||
CNOE calls the [schema and templating mechnanism 'stacks'](https://github.com/cnoe-io/stacks):
|
||||
|
||||
> Hint: **This has to be further investigated!** This is subject to an Epic.
|
||||
|
||||
There are already some example stacks:
|
||||
|
||||
<img src="./cnoe-stacks.png" width="600">
|
||||
|
||||
|
||||
{{% pageinfo color="info" %}}
|
||||
### What comes next?
|
||||
|
||||
[Next](../cnoe-showtime/) we'll see how a CNOE stacked Internal Developer Platform is deployed on you local laptop!
|
||||
{{% /pageinfo %}}
|
After Width: | Height: | Size: 63 KiB |
After Width: | Height: | Size: 167 KiB |
After Width: | Height: | Size: 120 KiB |
|
@ -0,0 +1,579 @@
|
|||
---
|
||||
title: CNOE Showtime
|
||||
weight: 6
|
||||
description: CNOE hands on
|
||||
---
|
||||
|
||||
|
||||
## Summary
|
||||
|
||||
CNOE is a 'Platform Engineering Framework' (Danger: Our wording!) - it is open source and locally runnable.
|
||||
|
||||
It consists of the orchestrator 'idpbuilder' and both of some predefined building blocks and also some predefined platform configurations.
|
||||
|
||||
|
||||
|
||||
## Orchestrator 'idpbuilder', initial run
|
||||
|
||||
The orchestrator in CNOE is called 'idpbuilder'. It is [locally installable binary](https://cnoe.io/docs/reference-implementation/installations/idpbuilder/quick-start)
|
||||
|
||||
A typipcal first setup ist described here: https://cnoe.io/docs/reference-implementation/technology
|
||||
|
||||
```bash
|
||||
# this is a local linux shell
|
||||
|
||||
# check local installation
|
||||
type idpbuilder
|
||||
idpbuilder is /usr/local/bin/idpbuilder
|
||||
|
||||
# check version
|
||||
idpbuilder version
|
||||
idpbuilder 0.8.0-nightly.20240914 go1.22.7 linux/amd64
|
||||
|
||||
# do some completion and aliasing
|
||||
source <(idpbuilder completion bash)
|
||||
alias ib=idpbuilder
|
||||
complete -F __start_idpbuilder ib
|
||||
|
||||
# check and remove all existing kind clusters
|
||||
kind delete clusters --all
|
||||
kind get clusters
|
||||
# sth. like 'No kind clusters found.'
|
||||
|
||||
# run
|
||||
$ib create --use-path-routing --log-level debug --package-dir https://github.com/cnoe-io/stacks//ref-implementation
|
||||
```
|
||||
|
||||
You get output like
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/ipceicis-developerframework$ idpbuilder create
|
||||
Oct 1 10:09:18 INFO Creating kind cluster logger=setup
|
||||
Oct 1 10:09:18 INFO Runtime detected logger=setup provider=docker
|
||||
########################### Our kind config ############################
|
||||
# Kind kubernetes release images https://github.com/kubernetes-sigs/kind/releases
|
||||
kind: Cluster
|
||||
apiVersion: kind.x-k8s.io/v1alpha4
|
||||
nodes:
|
||||
- role: control-plane
|
||||
image: "kindest/node:v1.30.0"
|
||||
labels:
|
||||
ingress-ready: "true"
|
||||
extraPortMappings:
|
||||
- containerPort: 443
|
||||
hostPort: 8443
|
||||
protocol: TCP
|
||||
|
||||
containerdConfigPatches:
|
||||
- |-
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."gitea.cnoe.localtest.me:8443"]
|
||||
endpoint = ["https://gitea.cnoe.localtest.me"]
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.configs."gitea.cnoe.localtest.me".tls]
|
||||
insecure_skip_verify = true
|
||||
|
||||
######################### config end ############################
|
||||
```
|
||||
|
||||
## Show time steps
|
||||
|
||||
> Goto https://cnoe.io/docs/reference-implementation/installations/idpbuilder/usage, and follow the flow
|
||||
|
||||
### Prepare a k8s cluster with kind
|
||||
|
||||
You may have seen: when starting `idpbuilder` without an existing cluster named `localdev` it first creates this cluster by calling `kind`with an internally defined config.
|
||||
|
||||
It's an important feature of idpbuilder that it will set up on an existing cluster `localdev` when called several times in a row e.g. to append new packages to the cluster.
|
||||
|
||||
That's why we here first create the kind cluster `localdev`itself:
|
||||
|
||||
```bash
|
||||
cat << EOF | kind create cluster --name localdev --config=-
|
||||
# Kind kubernetes release images https://github.com/kubernetes-sigs/kind/releases
|
||||
kind: Cluster
|
||||
apiVersion: kind.x-k8s.io/v1alpha4
|
||||
nodes:
|
||||
- role: control-plane
|
||||
image: "kindest/node:v1.30.0"
|
||||
labels:
|
||||
ingress-ready: "true"
|
||||
extraPortMappings:
|
||||
- containerPort: 443
|
||||
hostPort: 8443
|
||||
protocol: TCP
|
||||
|
||||
containerdConfigPatches:
|
||||
- |-
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."gitea.cnoe.localtest.me:8443"]
|
||||
endpoint = ["https://gitea.cnoe.localtest.me"]
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.configs."gitea.cnoe.localtest.me".tls]
|
||||
insecure_skip_verify = true
|
||||
```
|
||||
|
||||
```bash
|
||||
# alternatively, if you have the kind config as file:
|
||||
kind create cluster --name localdev --config kind-config.yaml
|
||||
```
|
||||
|
||||
#### Output
|
||||
|
||||
A typical raw kind setup kubernetes cluster would look like this with respect to running pods:
|
||||
|
||||
> be sure all pods are in status 'running'
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/idpbuilder$ k get pods -A
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
kube-system coredns-76f75df574-lb7jz 1/1 Running 0 15s
|
||||
kube-system coredns-76f75df574-zm2wp 1/1 Running 0 15s
|
||||
kube-system etcd-localdev-control-plane 1/1 Running 0 27s
|
||||
kube-system kindnet-8qhd5 1/1 Running 0 13s
|
||||
kube-system kindnet-r4d6n 1/1 Running 0 15s
|
||||
kube-system kube-apiserver-localdev-control-plane 1/1 Running 0 27s
|
||||
kube-system kube-controller-manager-localdev-control-plane 1/1 Running 0 27s
|
||||
kube-system kube-proxy-vrh64 1/1 Running 0 15s
|
||||
kube-system kube-proxy-w8ks9 1/1 Running 0 13s
|
||||
kube-system kube-scheduler-localdev-control-plane 1/1 Running 0 27s
|
||||
local-path-storage local-path-provisioner-6f8956fb48-6fvt2 1/1 Running 0 15s
|
||||
```
|
||||
|
||||
### First run: Start with core applications, 'core package'
|
||||
|
||||
Now we run idpbuilder the first time:
|
||||
|
||||
```
|
||||
# now idpbuilder reuses the already existing cluster
|
||||
# pls apply '--use-path-routing' otherwise as we discovered currently the service resolving inside the cluster won't work
|
||||
ib create --use-path-routing
|
||||
```
|
||||
|
||||
#### Output
|
||||
|
||||
##### idpbuilder log
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/idpbuilder$ ib create --use-path-routing
|
||||
Oct 1 10:32:50 INFO Creating kind cluster logger=setup
|
||||
Oct 1 10:32:50 INFO Runtime detected logger=setup provider=docker
|
||||
Oct 1 10:32:50 INFO Cluster already exists logger=setup cluster=localdev
|
||||
Oct 1 10:32:50 INFO Adding CRDs to the cluster logger=setup
|
||||
Oct 1 10:32:51 INFO Setting up CoreDNS logger=setup
|
||||
Oct 1 10:32:51 INFO Setting up TLS certificate logger=setup
|
||||
Oct 1 10:32:51 INFO Creating localbuild resource logger=setup
|
||||
Oct 1 10:32:51 INFO Starting EventSource controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository source=kind source: *v1alpha1.GitRepository
|
||||
Oct 1 10:32:51 INFO Starting Controller controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
|
||||
Oct 1 10:32:51 INFO Starting EventSource controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild source=kind source: *v1alpha1.Localbuild
|
||||
Oct 1 10:32:51 INFO Starting Controller controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
|
||||
Oct 1 10:32:51 INFO Starting EventSource controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage source=kind source: *v1alpha1.CustomPackage
|
||||
Oct 1 10:32:51 INFO Starting Controller controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
|
||||
Oct 1 10:32:51 INFO Starting workers controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild worker count=1
|
||||
Oct 1 10:32:51 INFO Starting workers controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage worker count=1
|
||||
Oct 1 10:32:51 INFO Starting workers controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository worker count=1
|
||||
Oct 1 10:32:54 INFO Waiting for Deployment my-gitea to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
|
||||
Oct 1 10:32:54 INFO Waiting for Deployment ingress-nginx-controller to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
|
||||
Oct 1 10:33:24 INFO Waiting for Deployment my-gitea to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
|
||||
Oct 1 10:33:24 INFO Waiting for Deployment ingress-nginx-controller to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
|
||||
Oct 1 10:33:54 INFO Waiting for Deployment my-gitea to become ready controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
|
||||
Oct 1 10:34:24 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
|
||||
Oct 1 10:34:24 INFO expected annotation, cnoe.io/last-observed-cli-start-time, not found controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
|
||||
Oct 1 10:34:24 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=6fc396d4-e0d5-4c80-aaee-20c1bbffea54
|
||||
Oct 1 10:34:25 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0667fa85-af1c-403f-bcd9-16ff8f2fad7e
|
||||
Oct 1 10:34:25 INFO expected annotation, cnoe.io/last-observed-cli-start-time, not found controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0667fa85-af1c-403f-bcd9-16ff8f2fad7e
|
||||
Oct 1 10:34:25 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0667fa85-af1c-403f-bcd9-16ff8f2fad7e
|
||||
Oct 1 10:34:40 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=ec720aeb-02cd-4974-a991-cf2f19ce8536
|
||||
Oct 1 10:34:40 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=ec720aeb-02cd-4974-a991-cf2f19ce8536
|
||||
Oct 1 10:34:40 INFO Shutting Down controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=ec720aeb-02cd-4974-a991-cf2f19ce8536
|
||||
Oct 1 10:34:40 INFO Stopping and waiting for non leader election runnables
|
||||
Oct 1 10:34:40 INFO Stopping and waiting for leader election runnables
|
||||
Oct 1 10:34:40 INFO Shutdown signal received, waiting for all workers to finish controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
|
||||
Oct 1 10:34:40 INFO Shutdown signal received, waiting for all workers to finish controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
|
||||
Oct 1 10:34:40 INFO All workers finished controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
|
||||
Oct 1 10:34:40 INFO Shutdown signal received, waiting for all workers to finish controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
|
||||
Oct 1 10:34:40 INFO All workers finished controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
|
||||
Oct 1 10:34:40 INFO All workers finished controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
|
||||
Oct 1 10:34:40 INFO Stopping and waiting for caches
|
||||
Oct 1 10:34:40 INFO Stopping and waiting for webhooks
|
||||
Oct 1 10:34:40 INFO Stopping and waiting for HTTP servers
|
||||
Oct 1 10:34:40 INFO Wait completed, proceeding to shutdown the manager
|
||||
|
||||
|
||||
########################### Finished Creating IDP Successfully! ############################
|
||||
|
||||
|
||||
Can Access ArgoCD at https://cnoe.localtest.me:8443/argocd
|
||||
Username: admin
|
||||
Password can be retrieved by running: idpbuilder get secrets -p argocd
|
||||
```
|
||||
|
||||
##### ArgoCD applications
|
||||
|
||||
When running idpbuilder 'barely' (without package option) you get the 'core applications' deployed in your cluster:
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/ipceicis-developerframework$ k get applications -A
|
||||
NAMESPACE NAME SYNC STATUS HEALTH STATUS
|
||||
argocd argocd Synced Healthy
|
||||
argocd gitea Synced Healthy
|
||||
argocd nginx Synced Healthy
|
||||
```
|
||||
|
||||
##### ArgoCD UI
|
||||
|
||||
Open ArgoCD locally:
|
||||
|
||||
https://cnoe.localtest.me:8443/argocd
|
||||
|
||||

|
||||
|
||||
Next find the provided credentials for ArgoCD (here: argocd-initial-admin-secret):
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/idpbuilder$ ib get secrets
|
||||
---------------------------
|
||||
Name: argocd-initial-admin-secret
|
||||
Namespace: argocd
|
||||
Data:
|
||||
password : 2MoMeW30wSC9EraF
|
||||
username : admin
|
||||
---------------------------
|
||||
Name: gitea-credential
|
||||
Namespace: gitea
|
||||
Data:
|
||||
password : LI$T?o>N{-<|{^dm$eTg*gni1(2:Y0@q344yqQIS
|
||||
username : giteaAdmin
|
||||
```
|
||||
|
||||
In ArgoCD you will see the deployed three applications of the core package:
|
||||
|
||||

|
||||
|
||||
### Second run: Append 'package1' from the CNOE-stacks repo
|
||||
|
||||
CNOE provides example packages in `https://github.com/cnoe-io/stacks.git`. Having cloned this repo you can locally refer to theses packages:
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ git remote -v
|
||||
origin https://github.com/cnoe-io/stacks.git (fetch)
|
||||
origin https://github.com/cnoe-io/stacks.git (push)
|
||||
```
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ls -al
|
||||
total 64
|
||||
drwxr-xr-x 12 stl stl 4096 Sep 28 13:55 .
|
||||
drwxr-xr-x 26 stl stl 4096 Sep 30 11:53 ..
|
||||
drwxr-xr-x 8 stl stl 4096 Sep 28 13:56 .git
|
||||
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 .github
|
||||
-rw-r--r-- 1 stl stl 11341 Sep 28 09:12 LICENSE
|
||||
-rw-r--r-- 1 stl stl 1079 Sep 28 13:55 README.md
|
||||
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 basic
|
||||
drwxr-xr-x 4 stl stl 4096 Sep 14 15:54 crossplane-integrations
|
||||
drwxr-xr-x 3 stl stl 4096 Aug 13 14:52 dapr-integration
|
||||
drwxr-xr-x 3 stl stl 4096 Sep 14 15:54 jupyterhub
|
||||
drwxr-xr-x 6 stl stl 4096 Aug 16 14:36 local-backup
|
||||
drwxr-xr-x 3 stl stl 4096 Aug 16 14:36 localstack-integration
|
||||
drwxr-xr-x 8 stl stl 4096 Sep 28 13:02 ref-implementation
|
||||
drwxr-xr-x 2 stl stl 4096 Aug 16 14:36 terraform-integrations
|
||||
|
||||
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ls -al basic/
|
||||
total 20
|
||||
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 .
|
||||
drwxr-xr-x 12 stl stl 4096 Sep 28 13:55 ..
|
||||
-rw-r--r-- 1 stl stl 632 Jul 29 10:57 README.md
|
||||
drwxr-xr-x 3 stl stl 4096 Jul 29 10:57 package1
|
||||
drwxr-xr-x 2 stl stl 4096 Jul 29 10:57 package2
|
||||
|
||||
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ls -al basic/package1
|
||||
total 16
|
||||
drwxr-xr-x 3 stl stl 4096 Jul 29 10:57 .
|
||||
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 ..
|
||||
-rw-r--r-- 1 stl stl 655 Jul 29 10:57 app.yaml
|
||||
drwxr-xr-x 2 stl stl 4096 Jul 29 10:57 manifests
|
||||
|
||||
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ls -al basic/package2
|
||||
total 16
|
||||
drwxr-xr-x 2 stl stl 4096 Jul 29 10:57 .
|
||||
drwxr-xr-x 4 stl stl 4096 Jul 29 10:57 ..
|
||||
-rw-r--r-- 1 stl stl 498 Jul 29 10:57 app.yaml
|
||||
-rw-r--r-- 1 stl stl 500 Jul 29 10:57 app2.yaml
|
||||
```
|
||||
|
||||
#### Output
|
||||
|
||||
Now we run idpbuilder the second time with `-p basic/package1`
|
||||
|
||||
##### idpbuilder log
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ib create --use-path-routing -p basic/package1
|
||||
Oct 1 12:09:27 INFO Creating kind cluster logger=setup
|
||||
Oct 1 12:09:27 INFO Runtime detected logger=setup provider=docker
|
||||
Oct 1 12:09:27 INFO Cluster already exists logger=setup cluster=localdev
|
||||
Oct 1 12:09:28 INFO Adding CRDs to the cluster logger=setup
|
||||
Oct 1 12:09:28 INFO Setting up CoreDNS logger=setup
|
||||
Oct 1 12:09:28 INFO Setting up TLS certificate logger=setup
|
||||
Oct 1 12:09:28 INFO Creating localbuild resource logger=setup
|
||||
Oct 1 12:09:28 INFO Starting EventSource controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild source=kind source: *v1alpha1.Localbuild
|
||||
Oct 1 12:09:28 INFO Starting Controller controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
|
||||
Oct 1 12:09:28 INFO Starting EventSource controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage source=kind source: *v1alpha1.CustomPackage
|
||||
Oct 1 12:09:28 INFO Starting Controller controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
|
||||
Oct 1 12:09:28 INFO Starting EventSource controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository source=kind source: *v1alpha1.GitRepository
|
||||
Oct 1 12:09:28 INFO Starting Controller controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
|
||||
Oct 1 12:09:28 INFO Starting workers controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild worker count=1
|
||||
Oct 1 12:09:28 INFO Starting workers controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository worker count=1
|
||||
Oct 1 12:09:28 INFO Starting workers controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage worker count=1
|
||||
Oct 1 12:09:29 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0ed7ccc2-dec7-4ab8-909c-791a7d1b67a8
|
||||
Oct 1 12:09:29 INFO unknown field "status.history[0].initiatedBy" logger=KubeAPIWarningLogger
|
||||
Oct 1 12:09:29 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=0ed7ccc2-dec7-4ab8-909c-791a7d1b67a8
|
||||
Oct 1 12:09:29 ERROR failed updating repo status controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage name=app-my-app namespace=idpbuilder-localdev namespace=idpbuilder-localdev name=app-my-app reconcileID=f9873560-5dcd-4e59-b6f7-ce5d1029ef3d err=Operation cannot be fulfilled on custompackages.idpbuilder.cnoe.io "app-my-app": the object has been modified; please apply your changes to the latest version and try again
|
||||
Oct 1 12:09:29 ERROR Reconciler error controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage name=app-my-app namespace=idpbuilder-localdev namespace=idpbuilder-localdev name=app-my-app reconcileID=f9873560-5dcd-4e59-b6f7-ce5d1029ef3d err=updating argocd application object my-app: Operation cannot be fulfilled on applications.argoproj.io "my-app": the object has been modified; please apply your changes to the latest version and try again
|
||||
Oct 1 12:09:31 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=531cc2d4-6250-493a-aca8-fecf048a608d
|
||||
Oct 1 12:09:31 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=531cc2d4-6250-493a-aca8-fecf048a608d
|
||||
Oct 1 12:09:44 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=022b9813-8708-4f4e-90d7-38f1e114c46f
|
||||
Oct 1 12:09:44 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=022b9813-8708-4f4e-90d7-38f1e114c46f
|
||||
Oct 1 12:10:00 INFO installing bootstrap apps to ArgoCD controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=79a85c21-42c1-41ec-ad03-2bb77aeae027
|
||||
Oct 1 12:10:00 INFO Checking if we should shutdown controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=79a85c21-42c1-41ec-ad03-2bb77aeae027
|
||||
Oct 1 12:10:00 INFO Shutting Down controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild name=localdev name=localdev reconcileID=79a85c21-42c1-41ec-ad03-2bb77aeae027
|
||||
Oct 1 12:10:00 INFO Stopping and waiting for non leader election runnables
|
||||
Oct 1 12:10:00 INFO Stopping and waiting for leader election runnables
|
||||
Oct 1 12:10:00 INFO Shutdown signal received, waiting for all workers to finish controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
|
||||
Oct 1 12:10:00 INFO Shutdown signal received, waiting for all workers to finish controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
|
||||
Oct 1 12:10:00 INFO All workers finished controller=custompackage controllerGroup=idpbuilder.cnoe.io controllerKind=CustomPackage
|
||||
Oct 1 12:10:00 INFO Shutdown signal received, waiting for all workers to finish controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
|
||||
Oct 1 12:10:00 INFO All workers finished controller=localbuild controllerGroup=idpbuilder.cnoe.io controllerKind=Localbuild
|
||||
Oct 1 12:10:00 INFO All workers finished controller=gitrepository controllerGroup=idpbuilder.cnoe.io controllerKind=GitRepository
|
||||
Oct 1 12:10:00 INFO Stopping and waiting for caches
|
||||
Oct 1 12:10:00 INFO Stopping and waiting for webhooks
|
||||
Oct 1 12:10:00 INFO Stopping and waiting for HTTP servers
|
||||
Oct 1 12:10:00 INFO Wait completed, proceeding to shutdown the manager
|
||||
|
||||
|
||||
########################### Finished Creating IDP Successfully! ############################
|
||||
|
||||
|
||||
Can Access ArgoCD at https://cnoe.localtest.me:8443/argocd
|
||||
Username: admin
|
||||
Password can be retrieved by running: idpbuilder get secrets -p argocd
|
||||
```
|
||||
|
||||
##### ArgoCD applications
|
||||
|
||||
Now we have additionally the 'my-app' deployed in the cluster:
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~$ k get applications -A
|
||||
NAMESPACE NAME SYNC STATUS HEALTH STATUS
|
||||
argocd argocd Synced Healthy
|
||||
argocd gitea Synced Healthy
|
||||
argocd my-app Synced Healthy
|
||||
argocd nginx Synced Healthy
|
||||
```
|
||||
|
||||
##### ArgoCD UI
|
||||
|
||||

|
||||
|
||||
### Third run: Finally we append 'ref-implementation' from the CNOE-stacks repo
|
||||
|
||||
We finally append the so called ['reference-implementation'](https://cnoe.io/docs/reference-implementation/integrations/reference-impl), which provides a real basic IDP:
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ib create --use-path-routing -p ref-implementation
|
||||
```
|
||||
|
||||
##### ArgoCD applications
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~$ k get applications -A
|
||||
NAMESPACE NAME SYNC STATUS HEALTH STATUS
|
||||
argocd argo-workflows Synced Healthy
|
||||
argocd argocd Synced Healthy
|
||||
argocd backstage Synced Healthy
|
||||
argocd included-backstage-templates Synced Healthy
|
||||
argocd external-secrets Synced Healthy
|
||||
argocd gitea Synced Healthy
|
||||
argocd keycloak Synced Healthy
|
||||
argocd metric-server Synced Healthy
|
||||
argocd my-app Synced Healthy
|
||||
argocd nginx Synced Healthy
|
||||
argocd spark-operator Synced Healthy
|
||||
```
|
||||
|
||||
##### ArgoCD UI
|
||||
|
||||
ArgoCD shows all provissioned applications:
|
||||
|
||||

|
||||
|
||||
##### Keycloak UI
|
||||
|
||||
In our cluster there is also keycloak as IAM provisioned.
|
||||
Login into Keycloak with 'cnoe-admin' and the KEYCLOAK_ADMIN_PASSWORD.
|
||||
|
||||
These credentails are defined in the package:
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ cat ref-implementation/keycloak/manifests/keycloak-config.yaml | grep -i admin
|
||||
group-admin-payload.json: |
|
||||
{"name":"admin"}
|
||||
"/admin"
|
||||
ADMIN_PASSWORD=$(cat /var/secrets/KEYCLOAK_ADMIN_PASSWORD)
|
||||
--data-urlencode "username=cnoe-admin" \
|
||||
--data-urlencode "password=${ADMIN_PASSWORD}" \
|
||||
```
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/cnoe-stacks$ ib get secrets
|
||||
---------------------------
|
||||
Name: argocd-initial-admin-secret
|
||||
Namespace: argocd
|
||||
Data:
|
||||
password : 2MoMeW30wSC9EraF
|
||||
username : admin
|
||||
---------------------------
|
||||
Name: gitea-credential
|
||||
Namespace: gitea
|
||||
Data:
|
||||
password : LI$T?o>N{-<|{^dm$eTg*gni1(2:Y0@q344yqQIS
|
||||
username : giteaAdmin
|
||||
---------------------------
|
||||
Name: keycloak-config
|
||||
Namespace: keycloak
|
||||
Data:
|
||||
KC_DB_PASSWORD : k3-1kgxxd/X2Cw//pX-uKMsmgWogEz5YGnb5
|
||||
KC_DB_USERNAME : keycloak
|
||||
KEYCLOAK_ADMIN_PASSWORD : zMSjv5eA0l/+0-MDAaaNe+rHRMrB2q0NssP-
|
||||
POSTGRES_DB : keycloak
|
||||
POSTGRES_PASSWORD : k3-1kgxxd/X2Cw//pX-uKMsmgWogEz5YGnb5
|
||||
POSTGRES_USER : keycloak
|
||||
USER_PASSWORD : Kd+0+/BqPRAvnLPZO-L2o/6DoBrzUeMsr29U
|
||||
```
|
||||
|
||||

|
||||
|
||||
|
||||
##### Backstage UI
|
||||
|
||||
As Backstage login you either can use the 'user1' with `USER_PASSWORD : Kd+0+/BqPRAvnLPZO-L2o/6DoBrzUeMsr29U` or you create a new user in keycloak
|
||||
|
||||

|
||||
|
||||
We create user 'ipcei' and also set a password (in tab 'Credentials'):
|
||||
|
||||

|
||||
|
||||
Now we can log into backstage (rember: you could have already existing usr 'user1'):
|
||||
|
||||

|
||||
|
||||
and see the basic setup of the Backstage portal:
|
||||
|
||||

|
||||
|
||||
### Use a Golden Path: 'Basic Deployment'
|
||||
|
||||
Now we want to use the Backstage portal as a developer. We create in Backstage our own platform based activity by using the golden path template 'Basic Deployment:
|
||||
|
||||

|
||||
|
||||
When we run it, we see 'golden path activities'
|
||||
|
||||

|
||||
|
||||
which finally result in a new catalogue entry:
|
||||
|
||||

|
||||
|
||||
#### Software development lifecycle
|
||||
|
||||
When we follow the 'view source' link we are directly linked to the git repo of our newly created application:
|
||||
|
||||

|
||||
|
||||
Check it out by cloning into a local git repo (watch the GIT_SSL_NO_VERIFY=true env setting):
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/idp-temporary$ GIT_SSL_NO_VERIFY=true git clone https://cnoe.localtest.me:8443/gitea/giteaAdmin/basicdeployment.git
|
||||
Cloning into 'basicdeployment'...
|
||||
remote: Enumerating objects: 10, done.
|
||||
remote: Counting objects: 100% (10/10), done.
|
||||
remote: Compressing objects: 100% (8/8), done.
|
||||
remote: Total 10 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
|
||||
Receiving objects: 100% (10/10), 47.62 KiB | 23.81 MiB/s, done.
|
||||
|
||||
stl@ubuntu-vpn:~/git/mms/idp-temporary$ cd basicdeployment/
|
||||
|
||||
stl@ubuntu-vpn:~/git/mms/idp-temporary/basicdeployment$ ll
|
||||
total 24
|
||||
drwxr-xr-x 5 stl stl 4096 Oct 1 13:00 ./
|
||||
drwxr-xr-x 4 stl stl 4096 Oct 1 13:00 ../
|
||||
drwxr-xr-x 8 stl stl 4096 Oct 1 13:00 .git/
|
||||
-rw-r--r-- 1 stl stl 928 Oct 1 13:00 catalog-info.yaml
|
||||
drwxr-xr-x 3 stl stl 4096 Oct 1 13:00 docs/
|
||||
drwxr-xr-x 2 stl stl 4096 Oct 1 13:00 manifests/
|
||||
```
|
||||
|
||||
#### Edit and change
|
||||
|
||||
Change some things, like the decription and the replicas:
|
||||
|
||||

|
||||
|
||||
#### Push
|
||||
|
||||
Push your changes, use the giteaAdmin user to authenticate:
|
||||
|
||||
```bash
|
||||
stl@ubuntu-vpn:~/git/mms/idp-temporary/basicdeployment$ ib get secrets
|
||||
---------------------------
|
||||
Name: argocd-initial-admin-secret
|
||||
Namespace: argocd
|
||||
Data:
|
||||
password : 2MoMeW30wSC9EraF
|
||||
username : admin
|
||||
---------------------------
|
||||
Name: gitea-credential
|
||||
Namespace: gitea
|
||||
Data:
|
||||
password : LI$T?o>N{-<|{^dm$eTg*gni1(2:Y0@q344yqQIS
|
||||
username : giteaAdmin
|
||||
---------------------------
|
||||
Name: keycloak-config
|
||||
Namespace: keycloak
|
||||
Data:
|
||||
KC_DB_PASSWORD : k3-1kgxxd/X2Cw//pX-uKMsmgWogEz5YGnb5
|
||||
KC_DB_USERNAME : keycloak
|
||||
KEYCLOAK_ADMIN_PASSWORD : zMSjv5eA0l/+0-MDAaaNe+rHRMrB2q0NssP-
|
||||
POSTGRES_DB : keycloak
|
||||
POSTGRES_PASSWORD : k3-1kgxxd/X2Cw//pX-uKMsmgWogEz5YGnb5
|
||||
POSTGRES_USER : keycloak
|
||||
USER_PASSWORD : Kd+0+/BqPRAvnLPZO-L2o/6DoBrzUeMsr29U
|
||||
stl@ubuntu-vpn:~/git/mms/idp-temporary/basicdeployment$ GIT_SSL_NO_VERIFY=true git push
|
||||
Username for 'https://cnoe.localtest.me:8443': giteaAdmin
|
||||
Password for 'https://giteaAdmin@cnoe.localtest.me:8443':
|
||||
Enumerating objects: 5, done.
|
||||
Counting objects: 100% (5/5), done.
|
||||
Delta compression using up to 8 threads
|
||||
Compressing objects: 100% (3/3), done.
|
||||
Writing objects: 100% (3/3), 382 bytes | 382.00 KiB/s, done.
|
||||
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
|
||||
remote: . Processing 1 references
|
||||
remote: Processed 1 references in total
|
||||
To https://cnoe.localtest.me:8443/gitea/giteaAdmin/basicdeployment.git
|
||||
69244d6..1269617 main -> main
|
||||
```
|
||||
|
||||
#### Wait for gitops magic: deployment into the 'production' cluster
|
||||
|
||||
Next wait a bit until Gitops does its magic and our 'wanted' state in the repo gets automatically deployed to the 'production' cluster:
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
{{% pageinfo color="info" %}}
|
||||
### What comes next?
|
||||
|
||||
The showtime of CNOE high level behaviour and usage scenarios is now finished. We setup an initial IDP and used a backstage golden path to init and deploy a simple application.
|
||||
|
||||
[Last not least](../conclusio/) we want to sum up the whole way from Devops to 'Frameworking' (is this the correct wording???)
|
||||
{{% /pageinfo %}}
|
After Width: | Height: | Size: 174 KiB |
After Width: | Height: | Size: 196 KiB |
After Width: | Height: | Size: 188 KiB |
After Width: | Height: | Size: 243 KiB |
After Width: | Height: | Size: 79 KiB |
After Width: | Height: | Size: 275 KiB |
After Width: | Height: | Size: 190 KiB |
After Width: | Height: | Size: 66 KiB |
After Width: | Height: | Size: 207 KiB |
After Width: | Height: | Size: 397 KiB |
After Width: | Height: | Size: 137 KiB |
After Width: | Height: | Size: 85 KiB |
After Width: | Height: | Size: 85 KiB |
After Width: | Height: | Size: 118 KiB |
After Width: | Height: | Size: 152 KiB |
After Width: | Height: | Size: 372 KiB |
After Width: | Height: | Size: 766 KiB |
|
@ -0,0 +1,18 @@
|
|||
// how to create/export c4 images:
|
||||
// see also https://likec4.dev/tooling/cli/
|
||||
|
||||
docker run -it --rm --name likec4 --user node -v $PWD:/app node bash
|
||||
npm install likec4
|
||||
exit
|
||||
|
||||
docker commit likec4 likec4
|
||||
docker run -it --rm --user node -v $PWD:/app -p 5173:5173 likec4 bash
|
||||
|
||||
// as root
|
||||
npx playwright install-deps
|
||||
npx playwright install
|
||||
|
||||
npm install likec4
|
||||
|
||||
// render
|
||||
node@e20899c8046f:/app/content/en/docs/project/onboarding$ ./node_modules/.bin/likec4 export png -o ./images .
|
|
@ -0,0 +1,31 @@
|
|||
---
|
||||
title: Conclusio
|
||||
weight: 7
|
||||
description: 'Summary and final thoughts: Always challenge theses concepts, accumptions and claims!'
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
In the project 'Edge Developer Framework' we start with DevOps, set platforms on top to automate golden paths, and finally set 'frameworks' (aka Orchestrators') on top to have declarative,automated and reconcilable platforms.
|
||||
|
||||
|
||||
|
||||
## From Devops over Platform to Framework Engineering
|
||||
|
||||
We come along from a quite well known, but already complex discipline called 'Platform Engineering', which is the next level devops.
|
||||
On top of these two domains we now have 'Framework Engineering', i.e. buildung dynamic, orchestrated and reconciling platforms:
|
||||
|
||||
| Classic Platform engineering | New: Framework Orchestration on top of Platforms | Your job: Framework Engineer |
|
||||
| -- | -- | -- |
|
||||
| <img src="./images/layers-and-platform-engineer.png" width="200"> |<img src="./images/layers.png" width="200"> | <img src="./images/layers-and-framework-engineer.png" width="200"> |
|
||||
|
||||
## The whole picture of engineering
|
||||
|
||||
So always keep in mind that as as 'Framework Engineer' you
|
||||
* include the skills of a platform and a devops engineer,
|
||||
* you do Framework, Platform and Devops Engineering at the same time
|
||||
* and your results have impact on Frameworks, Platforms and Devops tools, layers, processes.
|
||||
|
||||
The following diamond is illustrating this: on top is you, on the bottom is our baseline 'DevOps'
|
||||
|
||||
<img src="./images/modern.png" width="600">
|
|
@ -0,0 +1,102 @@
|
|||
specification {
|
||||
tag engineering
|
||||
element domain
|
||||
element engineer {
|
||||
style {
|
||||
shape person
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
model {
|
||||
|
||||
engineer framework-engineer 'Framework Engineer' 'Build and maintain one platform orchestrating framework'{
|
||||
style {
|
||||
color: sky
|
||||
}
|
||||
-> framework-engineering
|
||||
-> platform-engineer
|
||||
}
|
||||
|
||||
domain framework-engineering 'Framework Engineering' 'Building and maintaining frameworks'{
|
||||
#engineering
|
||||
style {
|
||||
color: sky
|
||||
}
|
||||
-> framework
|
||||
-> platform-engineering
|
||||
}
|
||||
|
||||
domain framework '"Framework" (IPCEI wording!)' 'A platform defining system' {
|
||||
style {
|
||||
color: sky
|
||||
}
|
||||
-> platform
|
||||
}
|
||||
|
||||
engineer platform-engineer 'Platform Engineer' {
|
||||
style {
|
||||
color: indigo
|
||||
}
|
||||
-> platform-engineering
|
||||
-> devops-engineer
|
||||
}
|
||||
|
||||
domain platform-engineering 'Platform Engineering' 'Building and maintaining platforms' {
|
||||
#engineering
|
||||
style {
|
||||
color: indigo
|
||||
}
|
||||
-> platform
|
||||
-> devops-engineering
|
||||
}
|
||||
|
||||
domain platform 'Platform' 'A Devops defining system' {
|
||||
style {
|
||||
color: indigo
|
||||
}
|
||||
-> devops
|
||||
}
|
||||
|
||||
engineer devops-engineer 'Devops Engineer' {
|
||||
style {
|
||||
color: amber
|
||||
}
|
||||
-> devops-engineering
|
||||
}
|
||||
|
||||
domain devops-engineering 'Devops Engineering' 'Building and maintaining devops means' {
|
||||
#engineering
|
||||
style {
|
||||
color: amber
|
||||
}
|
||||
-> devops
|
||||
}
|
||||
domain devops 'Devops' 'A software lifecycle enabling tool and process setup' {
|
||||
style {
|
||||
color: amber
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
views {
|
||||
view modern {
|
||||
title 'Modern Devops'
|
||||
description 'Devops is abstarcted by Platforms, Platforms are abstracted by Frameworks (IPCEI wording!)'
|
||||
include element.kind==domain, element.kind==engineer
|
||||
|
||||
}
|
||||
|
||||
view layers {
|
||||
include devops, platform, framework
|
||||
}
|
||||
|
||||
view layers-and-framework-engineer {
|
||||
include devops, platform, framework, framework-engineering, framework-engineer
|
||||
}
|
||||
|
||||
view layers-and-platform-engineer {
|
||||
include devops, platform, platform-engineering, platform-engineer
|
||||
}
|
||||
}
|
After Width: | Height: | Size: 100 KiB |
After Width: | Height: | Size: 67 KiB |
After Width: | Height: | Size: 55 KiB |
After Width: | Height: | Size: 224 KiB |
|
@ -0,0 +1,9 @@
|
|||
---
|
||||
title: 'Platform 101: Conceptual Onboarding '
|
||||
linktitle: Conceptual Onboarding
|
||||
weight: 20
|
||||
description: How to conceptually onboard onto the Edge Developer Framework (EDF) requirements and the designed solution
|
||||
---
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
|
||||
## Storyline
|
||||
|
||||
1. We have the 'Developer Framework'
|
||||
2. We think the solution for DF is 'Platforming' (Digital Platforms)
|
||||
1. The next evolution after DevOps
|
||||
2. Gartner predicts 80% of SWE companies to have platforms in 2026
|
||||
3. Platforms have a history since roundabout 2019
|
||||
4. CNCF has a working group which created capabilities and a maturity model
|
||||
3. Platforms evolve - nowadys there are Platform Orchestrators
|
||||
1. Humanitec set up a Reference Architecture
|
||||
2. There is this 'Orchestrator' thing - declaratively describe, customize and change platforms!
|
||||
4. Mapping our assumptions to solutions
|
||||
1. CNOE is a hot candidate to help and fulfill our platform building
|
||||
2. CNOE aims to embrace change and customization!
|
||||
5. Showtime CNOE
|
||||
|
||||
|
||||
## Challenges
|
||||
|
||||
1. Don't miss to further investigate and truely understand **DF needs**
|
||||
2. Don't miss to further investigate and truely understand **Platform capabilities**
|
||||
3. Don't miss to further investigate and truely understand **Platform orchestration**
|
||||
3. Don't miss to further investigate and truely understand **CNOE solution**
|
||||
|
||||
## Architecture
|
||||
|
||||
|
After Width: | Height: | Size: 114 KiB |
After Width: | Height: | Size: 114 KiB |
|
@ -0,0 +1,95 @@
|
|||
---
|
||||
title: Stakeholder Workshop Intro
|
||||
weight: 50
|
||||
description: An overall eDF introduction for stakeholders
|
||||
linktitle: Stakeholder Workshops
|
||||
---
|
||||
|
||||
|
||||
## Edge Developer Framework Solution Overview
|
||||
|
||||
> This section is derived from [conceptual-onboarding-intro](../conceptual-onboarding/1_intro/)
|
||||
|
||||
1. As presented in the introduction: We have the ['Edge Developer Framework'](./edgel-developer-framework/). \
|
||||
In short the mission is:
|
||||
* Build a european edge cloud IPCEI-CIS
|
||||
* which contains typical layers infrastructure, platform, application
|
||||
* and on top has a new layer 'developer platform'
|
||||
* which delivers a **cutting edge developer experience** and enables **easy deploying** of applications onto the IPCEI-CIS
|
||||
2. We think the solution for EDF is in relation to ['Platforming' (Digital Platforms)](../conceptual-onboarding/3_platforming/)
|
||||
1. The next evolution after DevOps
|
||||
2. Gartner predicts 80% of SWE companies to have platforms in 2026
|
||||
3. Platforms have a history since roundabout 2019
|
||||
4. CNCF has a working group which created capabilities and a maturity model
|
||||
3. Platforms evolve - nowadys there are [Platform Orchestrators](../conceptual-onboarding/4_orchestrators/)
|
||||
1. Humanitec set up a Reference Architecture
|
||||
2. There is this 'Orchestrator' thing - declaratively describe, customize and change platforms!
|
||||
4. Mapping our assumptions to the [CNOE solution](../conceptual-onboarding/5_cnoe/)
|
||||
1. CNOE is a hot candidate to help and fulfill our platform building
|
||||
2. CNOE aims to embrace change and customization!
|
||||
|
||||
|
||||
## 2. Platforming as the result of DevOps
|
||||
|
||||
### DevOps since 2010
|
||||
|
||||

|
||||
|
||||
* from 'left' to 'right' - plan to monitor
|
||||
* 'leftshift'
|
||||
* --> turns out to be a right shift for developers with cognitive overload
|
||||
* 'DevOps isd dead' -> we need Platforms
|
||||
|
||||
### Platforming to provide 'golden paths'
|
||||
|
||||
> don't mix up 'golden paths' with pipelines or CI/CD
|
||||
|
||||

|
||||
|
||||
#### Short list of platform using companies
|
||||
|
||||
As [Gartner states](https://www.gartner.com/en/newsroom/press-releases/2023-11-28-gartner-hype-cycle-shows-ai-practices-and-platform-engineering-will-reach-mainstream-adoption-in-software-engineering-in-two-to-five-years): "By 2026, 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components and tools for application delivery."
|
||||
|
||||
Here is a small list of companies alrteady using IDPs:
|
||||
|
||||
* Spotify
|
||||
* Airbnb
|
||||
* Zalando
|
||||
* Uber
|
||||
* Netflix
|
||||
* Salesforce
|
||||
* Google
|
||||
* Booking.com
|
||||
* Amazon
|
||||
* Autodesk
|
||||
* Adobe
|
||||
* Cisco
|
||||
* ...
|
||||
|
||||
## 3 Platform building by 'Orchestrating'
|
||||
|
||||
So the goal of platforming is to build a 'digital platform' which fits [this architecture](https://www.gartner.com/en/infrastructure-and-it-operations-leaders/topics/platform-engineering) ([Ref. in German)](https://www.gartner.de/de/artikel/was-ist-platform-engineering):
|
||||
|
||||

|
||||
|
||||
### Digital Platform blue print: Reference Architecture
|
||||
|
||||
The blue print for such a platform is given by the reference architecture from Humanitec:
|
||||
|
||||
[Platform Orchestrators](../conceptual-onboarding/4_orchestrators/)
|
||||
|
||||
### Digital Platform builder: CNOE
|
||||
|
||||
Since 2023 this is done by 'orchestrating' such platforms. One orchestrator is the [CNOE solution](../conceptual-onboarding/5_cnoe/), which highly inspired our approach.
|
||||
|
||||
In our orchestartion engine we think in 'stacks' of 'packages' containing platform components.
|
||||
|
||||
|
||||
## 4 Sticking all together: Our current platform orchestrating generated platform
|
||||
|
||||
Sticking together the platforming orchestration concept, the reference architecture and the CNOE stack solution, [this is our current running platform minimum viable product](../plan-in-2024/image-2024-8-14_10-50-27.png).
|
||||
|
||||
This will now be presented! Enjoy!
|
||||
|
||||
|
||||
|
After Width: | Height: | Size: 212 KiB |
After Width: | Height: | Size: 96 KiB |
45
docs/technical-documentation/project/plan-in-2024/_index.md
Normal file
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
title: Plan in 2024
|
||||
weight: 30
|
||||
description: The planned project workload in 2024
|
||||
---
|
||||
|
||||
|
||||
## First Blue Print in 2024
|
||||
|
||||
Our first architectural blue print for the IPCEI-CIS Developer Framework derives from Humanitecs Reference Architecture, see links in [Blog](../../blog/240823-archsession.md)
|
||||
|
||||

|
||||
|
||||
## C4 Model
|
||||
|
||||
> (sources see in ./ressources/architecture-c4)
|
||||
|
||||
> How to use: install C4lite VSC exension and/or C4lite cli - then open *.c4 files in ./ressources/architecture-c4
|
||||
|
||||
First system landscape C4 model:
|
||||
|
||||

|
||||
|
||||
## In Confluence
|
||||
|
||||
https://confluence.telekom-mms.com/display/IPCEICIS/Architecture
|
||||
|
||||
|
||||
## Dimensionierung Cloud für initiales DevFramework
|
||||
|
||||
### 28.08.24, Stefan Bethke, Florian Fürstenberg, Stephan Lo
|
||||
|
||||
1) zuerst viele DevFrameworkPlatformEngineers arbeiten lokal, mit zentralem Deployment nach OTC in **einen/max zwei** Control-Cluster
|
||||
2) wir gehen anfangs von ca. 5 clustern aus
|
||||
3) jeder cluster mit 3 Knoten/VM (in drei AvailabilityZones)
|
||||
4) pro VM 4 CPU, 16 GB Ram, 50 GB Storage read/write once, PVCs 'ohne limit'
|
||||
5) public IPs, plus Loadbalancer
|
||||
6) Keycloak vorhanden
|
||||
7) Wildcard Domain ?? --> Eher ja
|
||||
|
||||
Next Steps: (Vorschlag: in den nächsten 2 Wochen)
|
||||
1. Florian spezifiziert an Tobias
|
||||
2. Tobias stellt bereit, kubeconfig kommt an uns
|
||||
3. wir deployen
|
||||
|
After Width: | Height: | Size: 243 KiB |
BIN
docs/technical-documentation/project/plan-in-2024/planes.png
Executable file
After Width: | Height: | Size: 302 KiB |
After Width: | Height: | Size: 264 KiB |