Go to file
2023-03-14 21:39:03 +01:00
.gitlab/agents Add dev-env kubernetes agent 2022-05-27 22:27:30 +02:00
adguard fix(domain name): Fix domain name for internal services 2022-12-31 00:31:31 +01:00
dashboard Update kubernetes dashboard to be accessible from localhost, waiting for TLS 2022-05-15 21:43:20 +02:00
environments feat(env): Update dev base environment kustomization file 2023-02-07 09:50:15 +01:00
gitlab feat(gitlab): Move secrets to proper secret manifests 2023-02-07 09:53:52 +01:00
longhorn fix(domain name): Fix domain name for internal services 2022-12-31 00:31:31 +01:00
metallb feat(kustomization): Add kustomization for deployment 2022-12-30 23:48:09 +01:00
minecraft Add working minecraft configuration 2022-06-06 18:41:15 +02:00
monitoring feat(gitlab): Add monitoring, registry and external database 2022-10-18 00:50:25 +02:00
nginx Holyday tmp 2022-06-24 20:43:24 +02:00
ocis feat(ocis): Add basic drive configuration 2022-09-21 18:24:46 +02:00
res feat(env): Add development environment 2023-01-03 01:09:08 +01:00
streaming feat(stream): Add jellyfin/seerr configuration bases 2023-03-14 21:39:03 +01:00
synapse feat(synapse): Add basic Matrix configuration 2022-09-24 01:03:07 +02:00
traefik feat(env): Add development environment 2023-01-03 01:09:08 +01:00
vaultwarden feat(vaultwarden): Configure local files 2022-09-24 01:04:58 +02:00
.gitlab-ci.yml Test gitlab integration 2022-05-28 14:05:44 +02:00
Makefile feat(env): Add production value 2023-01-03 01:27:06 +01:00
README.md docs(readme): Update setup documentation and notes 2023-02-07 09:47:24 +01:00

K3s cluster

Name Usage Accessibility Host DB type Additional data Backup configuration Loki integration Prometheus integration Secret management Status
Traefik Reverse proxy and load balancer Public & Private Socrates & Pythagoras-b - - - Configured Configured - Completed5
Vaultwarden Password manager Public Pythagoras-b MariaDB - 4AM K8s CronJob Configured Not available - Completed
Gitlab Version control system Public Pythagoras-b PostgreSQL User created content 5AM internal CronJob Configured Configured Configured Completed4
Radarr Movie collection manager Private Plato PostgreSQL - - Configured Configured Configured Completed
Flaresolverr Cloudflare proxy Private Plato - - - - - - Completed
Prometheus Metrics aggregator Private Pythagoras-b TBD - Not configured Configured Configured - Partial
Loki Log aggregator Private Pythagoras-b TBD - Not configured Configured Configured - Partial
Grafana Graph visualizer Public Pythagoras-b - - Not configured Configured Configured Configured Partial
Sonarr TV shows collection manager Private Plato SQLite - Not configured Configured Configured Configured Partial
Prowlarr Torrent indexer Private Plato PostgreSQL - Not configured Configured Not available Configured Partial
Jellyfin Media streaming Public Archimedes SQLite** - - Configured Not configured Not configured Awaiting configuration
Jellyseerr Media requesting WebUI Public Pythagoras-b - - - Not configured Not configured Not configured Awaiting configuration
Adguard DNS ad blocker and custom DNS server Private Socrates - - - Not configured Not configured Not configured Pending configuration1
Owncloud Infinity Scale File hosting webUI Public Plato ? Drive files Not configured Configured Not available Not configured Pending configuration2
Synapse Matrix server - Message centralizer Public Pythagoras-b PostgreSQL User medias 4AM K8s CronJob Configured Configured Not configured Pending configuration3
therbron.com Personal website Public Socrates - - - Not configured Not configured - Awaiting configuration
Home assistant Home automation and monitoring Private Pythagoras-a MariaDB - Not configured Not configured Not configured Not configured Awaiting configuration
Vikunja To-do and Kanban boards Public Pythagoras-b - - - Not configured Not configured - Migrate to Gitlab
Wiki Documentation manager Public Pythagoras-b - - - Not configured Not configured - Migrate to VuePress and Gitlab
PaperlessNG PDF viewer and organiser Public Pythagoras-b PostgreSQL - - Not configured Not configured - Research migration into OCIS
Deluge Torrent client Private Plato - ? - Not configured Not configured Not configured Awaiting configuration
Minecraft Vanilla minecraft server for friends Public Archimedes - Game map Not configured Not configured Not configured - Awaiting configuration
Satisfactory Satisfactory server for friends Public Archimedes - Game map Not configured Not configured Not configured - Not needed for v1
Space engineers Space engineers server for friends Public Archimedes - Game map Not configured Not configured Not configured - Not needed for v1
Raspsnir Bachelor memorial website Public Pythagoras-b PostgreSQL - Not configured Not configured Not configured - Not needed for v1

* Configuration panel only available internally
** Current implementation only support SQLite, making manual backups a necessity
1 Missing automated configuration pipeline for environment variable injection
2 Missing configuration for NAS volume mounting (over network)
3 Missing Longhorn scheduling for saving media_store and secret management
4 Backup management is not handled by k3s but by an internal cronjob rule (Change image name when putting to production)
5 Missing dashboard configuration

Backup management

Databases

All services needing a database to function come with a sidecar pod running a crontab to automate individual database backups. These backups are saved into a longhorn volume, to benefit from general snapshots later one. Each sidecar pod can only mount the backup folder it has been linked with, and cannot see other services' backups.

Additional data

All additional data needing to be backed up is mounted to a longhorn volume, to also benefit from scheduled backups.

Example :

longhorn
└───backups
    └───vaultwarden
    │   └───<backup_date>.sql
    │   │   ...
    └───gitlab
        └───<backup_date>.sql
        │   ...

TODO

  • Migrate Vaultwarden & Homeassistant to PostgreSQL instead of MariaDB
  • Deploy PostgresQL cluster using operator for database HA and easy maintenance
  • Change host/deployment specific variables to use environment variables (using Kustomize)
  • Write CI/CD pipeline to create environment loaded files Done with Kustomize migration
  • Write CI/CD pipeline to deploy cluster
  • Setup internal traefik with nodeport as reverse proxy for internal only services Done through double ingress class and LB
  • Setup DB container sidecars for automated backups to Longhorn volume
  • Setup secrets configuration through CI/CD variable injection (using Kustomzie)
  • Explore permission issues when issuing OVH API keys (not working for wildcard and beta.halia.dev subdomain)
  • Setup default users for deployments
  • Setup log and metric monitoring
  • Define namespaces through yaml files
  • Look into CockroachDB for redundant database Judged too complicated, moving to a 1 to 1 relationship between services and databases
  • Configure IP range accessibility through Traefik (Internal vs external services) Impossible because of flannel ip-masq
  • Schedule longhorn S3 backups
  • Move secrets to separate, private Git repository ?
  • Configure NFS connection for media library
  • Research IPv6 configuration for outsider node Impossible in Denmark while using YouSee as an ISP for now (no IPv6 support)

Notes

Cluster base setup

Setup the cluster's backbone

kubectl apply -k environment/dev

NOTE: It might be required to update the metallb IP range as well as traefik LoadBalancerIPs

Convert helm chart to k3s manifest

helm template chart stable/chart --output-dir ./chart

Gitlab backup process

Because gitlab does not offer the possibility to backup a container's data from an external container, a cronjob has been implemented in the custom image used for deployment.

VPN configuration for Deluge

Instead of adding an extra networking layer to the whole cluster, it seems like a better idea to just integrate a wireguard connection inside of the deluge image, and self-build everything within Gitlab registry. This image could utilize kubernetes secrets, including a "torrent-vpn" secret produces by the initial wireguard configuration done via Ansible. This ansible script could create one (or more) additional client(s) depending on the inventory configuration, and keep the "torrent-vpn" configuration file within a k3s formated file, inside of the auto-applied directory on CP.
Cf : https://docs.k3s.io/advanced#auto-deploying-manifests

Development domains

To access a service publicly when developing, the domain name should be *.beta.halia.dev To only expose a service internally, the domain name should be *.beta.entos

Ingresses

To split between external and internal services, two traefik ingresses are implemented through the ingressclass annotation. traefik-external will only allow external access to a given service, while traefik-internal restrict to an internal only access.