# K3s cluster | Name | Usage | Accessibility | Host | DB type | Additional data | Backup configuration | Loki integration | Prometheus integration | Status | |-------------------------|--------------------------------------|------------------|-------------------------|------------|----------------------|----------------------|------------------|------------------------|-----------------------------------| | Traefik | Reverse proxy and load balancer | Public & Private | Socrates & Pythagoras-b | - | - | - | Configured | Configured | Completed5 | | Vaultwarden | Password manager | Public | Pythagoras-b | MariaDB | - | 4AM K8s CronJob | Configured | Not available | Completed | | Gitlab | Version control system | Public | Pythagoras-b | PostgreSQL | User created content | 5AM internal CronJob | Configured | Configured | Completed4 | | Prometheus | Metrics aggregator | Private | Pythagoras-b | TBD | - | Not configured | Configured | Configured | Partial | | Loki | Log aggregator | Private | Pythagoras-b | TBD | - | Not configured | Configured | Configured | Partial | | Grafana | Graph visualizer | Public | Pythagoras-b | - | - | Not configured | Configured | Configured | Partial | | Adguard | DNS ad blocker and custom DNS server | Private | Socrates | - | - | - | Not configured | Not configured | Pending configuration1 | | Owncloud Infinity Scale | File hosting webUI | Public | Plato | ? | Drive files | Not configured | Configured | Not available | Pending configuration2 | | Synapse | Matrix server - Message centralizer | Public | Pythagoras-b | PostgreSQL | User medias | 4AM K8s CronJob | Configured | Configured | Pending configuration3 | | therbron.com | Personal website | Public | Socrates | - | - | - | Not configured | Not configured | Awaiting configuration | | Home assistant | Home automation and monitoring | Private | Pythagoras-a | MariaDB | - | Not configured | Not configured | Not configured | Awaiting configuration | | Vikunja | To-do and Kanban boards | Public | Pythagoras-b | - | - | - | Not configured | Not configured | Migrate to Gitlab | | Wiki | Documentation manager | Public | Pythagoras-b | - | - | - | Not configured | Not configured | Migrate to VuePress and Gitlab | | PaperlessNG | PDF viewer and organiser | Public | Pythagoras-b | PostgreSQL | - | - | Not configured | Not configured | Research migration into OCIS | | Jellyfin | Media streaming | Public | Archimedes | - | - | - | Not configured | Not configured | Awaiting configuration | | Sonarr | TV shows collection manager | Private | Plato | SQLite** | Internal backups | Not configured | Not configured | Not configured | Awaiting configuration | | Radarr | Movie collection manager | Private | Plato | SQLite** | Internal backups | Not configured | Not configured | Not configured | Awaiting configuration | | Jackett | Torrent indexer | Private | Plato | - |
?
| Not configured | Not configured | Not configured | Awaiting configuration | | Deluge | Torrent client | Private | Plato | - |
?
| - | Not configured | Not configured | Awaiting configuration | | Minecraft | Vanilla minecraft server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Awaiting configuration | | Satisfactory | Satisfactory server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Not needed for v1 | | Space engineers | Space engineers server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Not needed for v1 | | Raspsnir | Bachelor memorial website | Public | Pythagoras-b | PostgreSQL | - | Not configured | Not configured | Not configured | Not needed for v1 | \* Configuration panel only available internally
** Current implementation only support SQLite, making manual backups a necessity
1 Missing automated configuration pipeline for environment variable injection
2 Missing configuration for NAS volume mounting (over network)
3 Missing Longhorn scheduling for saving media_store and secret management
4 Backup management is not handled by k3s but by an internal cronjob rule (Change image name when putting to production)
5 Missing dashboard configuration
## Backup management ### Databases All services needing a database to function come with a sidecar pod running a crontab to automate individual database backups. These backups are saved into a longhorn volume, to benefit from general snapshots later one. Each sidecar pod can only mount the backup folder it has been linked with, and cannot see other services' backups. ### Additional data All additional data needing to be backed up is mounted to a longhorn volume, to also benefit from scheduled backups. Example : ``` longhorn └───backups └───vaultwarden │ └───.sql │ │ ... └───gitlab └───.sql │ ... ``` ## TODO - Change host/deployment specific variables to use environment variables - Write CI/CD pipeline to create environment loaded files - Write CI/CD pipeline to deploy cluster - ~~Setup internal traefik with nodeport as reverse proxy for internal only services~~ Done through double ingress class and LB - ~~Setup DB container sidecars for automated backups to Longhorn volume~~ - Setup secrets configuration through CI/CD variable injection - Explore permission issues when issuing OVH API keys (not working for wildcard and `beta.halia.dev` subdomain) - Setup default users for deployments - ~~Setup log and metric monitoring~~ - ~~Define namespaces through yaml files~~ - ~~Look into CockroachDB for redundant database~~ Judged too complicated, moving to a 1 to 1 relationship between services and databases - ~~Configure IP range accessibility through Traefik (Internal vs external services)~~ Impossible because of flannel ip-masq - Schedule longhorn S3 backups - Move to new naming conventions for files - Move secrets to separate, private Git repository ? ## Notes ### Cluster base setup Setup the cluster's backbone ``` kubectl apply -k . ``` ### Convert helm chart to k3s manifest `helm template chart stable/chart --output-dir ./chart` ### Gitlab backup process Because gitlab does not offer the possibility to backup a container's data from an external container, a cronjob has been implemented in the custom image used for deployment. ### VPN configuration for Deluge Instead of adding an extra networking layer to the whole cluster, it seems like a better idea to just integrate a wireguard connection inside of the deluge image, and self-build everything within Gitlab registry. This image could utilize kubernetes secrets, including a "torrent-vpn" secret produces by the initial wireguard configuration done via Ansible. This ansible script could create one (or more) additional client(s) depending on the inventory configuration, and keep the "torrent-vpn" configuration file within a k3s formated file, inside of the auto-applied directory on CP. Cf : https://docs.k3s.io/advanced#auto-deploying-manifests ### Development domains To access a service publicly when developing, the domain name should be *.beta.halia.dev To only expose a service internally, the domain name should be *.k3s.beta ### Ingresses To split between external and internal services, two traefik ingresses are implemented through the `ingressclass` annotation. `traefik-external` will only allow external access to a given service, while `traefik-internal` restrict to an internal only access. ### Naming convention As the project grows in size, all files must be renamed the following norm for easier readability and maintability : ``` {priority}-{optional-detail}-{service-name}.yaml ``` The priority here gives an execution order when applying files in bulk. This means that, for example, to avoid the `namespace doesn't exist` error, the following order can be followed : ``` - 0-namespace.yaml - 1-pvc.yaml - 2-deployment.yaml - 2-service.yaml - 2-ingress.yaml ``` This also applies to directories, this behaviour can be seen in the `traefik` directory, where RBAC are prioritized before other files, as they are the first dependency.