Tanguy Herbron
802eef042e
Add OwnCloud Infinity Scale as drive to the cluster. The current configuration only includes enough for a beta environment. |
||
---|---|---|
.gitlab/agents | ||
adguard | ||
dashboard | ||
longhorn | ||
minecraft | ||
monitoring | ||
nginx | ||
ocis | ||
res | ||
traefik | ||
vaultwarden | ||
.gitlab-ci.yml | ||
README.md |
K3s cluster
Name | Usage | Accessibility | Host | DB type | Additional data | Backup configuration | Loki integration | Prometheus integration | Status |
---|---|---|---|---|---|---|---|---|---|
therbron.com | Personal website | Public | Socrates | - | - | - | Not configured | Not configured | Awaiting configuration |
Traefik | Reverse proxy and load balancer | Public* | Socrates | - | - | - | Configured | Configured | Completed |
Adguard | DNS ad blocker and custom DNS server | Private | Socrates | - | - | - | Not configured | Not configured | Completed |
Owncloud Infinity Scale | File hosting webUI | Public | Plato | ? | Drive files | Managed by Longhorn | Configured | Not available | Completed |
Home assistant | Home automation and monitoring | Private | Pythagoras-a | PostgreSQL | - | Not configured | Not configured | Not configured | Awaiting configuration |
Vikunja | To-do and Kanban boards | Public | Pythagoras-b | - | - | - | Not configured | Not configured | Migrate to Gitlab |
Gitlab | Version control system | Public | Pythagoras-b | PostgreSQL | User created content | Not configured | Not configured | Not configured | Awaiting configuration |
Wiki | Documentation manager | Public | Pythagoras-b | - | - | - | Not configured | Not configured | Migrate to VuePress and Gitlab |
Vaultwarden | Password manager | Public | Pythagoras-b | PostgreSQL | - | 4AM K8s CronJob | Configured | Not available | Completed |
Synapse | Matrix server - Message centralizer | Public | Pythagoras-b | PostgreSQL | User medias | Not configured | Not configured | Not configured | Awaiting configuration |
PaperlessNG | PDF viewer and organiser | Public | Pythagoras-b | PostgreSQL | - | - | Not configured | Not configured | Research migration into OCIS |
Raspsnir | Bachelor memorial website | Public | Pythagoras-b | PostgreSQL | - | Not configured | Not configured | Not configured | Awaiting configuration |
Jellyfin | Media streaming | Public | Archimedes | - | - | - | Not configured | Not configured | Awaiting configuration |
Sonarr | TV shows collection manager | Private | Plato | SQLite** | Internal backups | Not configured | Not configured | Not configured | Awaiting configuration |
Radarr | Movie collection manager | Private | Plato | SQLite** | Internal backups | Not configured | Not configured | Not configured | Awaiting configuration |
Jackett | Torrent indexer | Private | Plato | - | ? | Not configured | Not configured | Not configured | Awaiting configuration |
Deluge | Torrent client | Private | Plato | - | ? | - | Not configured | Not configured | Awaiting configuration |
Minecraft | Vanilla minecraft server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Awaiting configuration |
Satisfactory | Satisfactory server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Awaiting configuration |
Space engineers | Space engineers server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Awaiting configuration |
Prometheus | Metrics aggregator | Private | Pythagoras-b | TBD | - | Not configured | Configured | Configured | Partial |
Loki | Log aggregator | Private | Pythagoras-b | TBD | - | Not configured | Configured | Configured | Partial |
Grafana | Graph visualizer | Public | Pythagoras-b | - | - | Not configured | Configured | Configured | Partial |
* Configuration panel only available internally
** Current implementation only support SQLite, making manual backups a necessity
Backup management
Databases
All services needing a database to function come with a sidecar pod running a crontab to automate individual database backups. These backups are saved into a longhorn volume, to benefit from general snapshots later one. Each sidecar pod can only mount the backup folder it has been linked with, and cannot see other services' backups.
Additional data
All additional data needing to be backed up is mounted to a longhorn volume, to also benefit from scheduled backups.
Example :
longhorn
└───backups
└───vaultwarden
│ └───<backup_date>.sql
│ │ ...
└───gitlab
└───<backup_date>.sql
│ ...
Completed
- Add TLS certificates for every http applications
TODO
- Change host/deployment specific variables to use environment variables
- Write CI/CD pipeline to create environment loaded files
- Write CI/CD pipeline to deploy cluster
- Setup internal traefik with nodeport as reverse proxy for internal only services
Setup DB container sidecars for automated backups to Longhorn volume- Setup secrets configuration through CI/CD variable injection
- Explore permission issues when issuing OVH API keys (not working for wildcard and
beta.halia.dev
subdomain) - Setup default users for services
Setup log and metric monitoringDefine namespaces through yaml filesLook into CockroachDB for redundant databaseJudged too complicated, moving to a 1 to 1 relationship between services and databasesConfigure IP range accessibility through Traefik (Internal vs external services)Impossible because of flannel ip-masq
Notes
Cluster base setup
Add node to the list of available load balancer
kubectl label node <node-name> svccontroller.k3s.cattle.io/enablelb=true
NOTE: For development, don't forget to also add the cp
to the LB list, in order to access local only services
Setup OVH configuration
kubectl apply -f ovh-config.yaml
Install traefik
kubectl apply -f traefik
Install longhorn
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
Remove local-path
from default StorageClass
kubectl patch storageclass standard -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
Add longhorn storage classes
kubectl apply -f res
Convert helm chart to k3s manifest
helm template chart stable/chart --output-dir ./chart