athens-school/K3s-cluster
3
0
Fork 0
Code Issues 11 Pull Requests Packages Projects Releases Wiki Activity

feat(LB): Add MetalLB and Traefik managed LBs

Browse Source 
Add custom IP range for LBs to split internal and external access
This commit is contained in:
Tanguy Herbron 2022-11-16 23:52:06 +01:00
parent dcc76f2cab
commit aa9e372c98
7 changed files with 80 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
README.md
View File

@ -4,7 +4,10 @@
|-------------------------|--------------------------------------|---------------|--------------|------------|----------------------|----------------------|------------------|------------------------|-----------------------------------|
| Traefik | Reverse proxy and load balancer | Public* | Socrates | - | - | - | Configured | Configured | Completed |
| Vaultwarden | Password manager | Public | Pythagoras-b | MariaDB | - | 4AM K8s CronJob | Configured | Not available | Completed |
| Gitlab | Version control system | Public | Pythagoras-b | PostgreSQL | User created content | 5AM internal CronJob | Configured | Configured | Completed<sup>4</sup> |
| Gitlab | Version control system | Public | Pythagoras-b | PostgreSQL | User created content | 5AM internal CronJob | Configured | Configured | Completed<sup>4</sup> |
| Prometheus | Metrics aggregator | Private | Pythagoras-b | TBD | - | Not configured | Configured | Configured | Partial |
| Loki | Log aggregator | Private | Pythagoras-b | TBD | - | Not configured | Configured | Configured | Partial |
| Grafana | Graph visualizer | Public | Pythagoras-b | - | - | Not configured | Configured | Configured | Partial |
| Adguard | DNS ad blocker and custom DNS server | Private | Socrates | - | - | - | Not configured | Not configured | Pending configuration<sup>1</sup> |
| Owncloud Infinity Scale | File hosting webUI | Public | Plato | ? | Drive files | Not configured | Configured | Not available | Pending configuration<sup>2</sup> |
| Synapse | Matrix server - Message centralizer | Public | Pythagoras-b | PostgreSQL | User medias | 4AM K8s CronJob | Configured | Configured | Pending configuration<sup>3</sup> |
@ -13,18 +16,15 @@
| Vikunja | To-do and Kanban boards | Public | Pythagoras-b | - | - | - | Not configured | Not configured | Migrate to Gitlab |
| Wiki | Documentation manager | Public | Pythagoras-b | - | - | - | Not configured | Not configured | Migrate to VuePress and Gitlab |
| PaperlessNG | PDF viewer and organiser | Public | Pythagoras-b | PostgreSQL | - | - | Not configured | Not configured | Research migration into OCIS |
| Raspsnir | Bachelor memorial website | Public | Pythagoras-b | PostgreSQL | - | Not configured | Not configured | Not configured | Awaiting configuration |
| Jellyfin | Media streaming | Public | Archimedes | - | - | - | Not configured | Not configured | Awaiting configuration |
| Sonarr | TV shows collection manager | Private | Plato | SQLite** | Internal backups | Not configured | Not configured | Not configured | Awaiting configuration |
| Radarr | Movie collection manager | Private | Plato | SQLite** | Internal backups | Not configured | Not configured | Not configured | Awaiting configuration |
| Jackett | Torrent indexer | Private | Plato | - | <center> ? </center> | Not configured | Not configured | Not configured | Awaiting configuration |
| Deluge | Torrent client | Private | Plato | - | <center> ? </center> | - | Not configured | Not configured | Awaiting configuration |
| Minecraft | Vanilla minecraft server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Awaiting configuration |
| Satisfactory | Satisfactory server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Awaiting configuration |
| Space engineers | Space engineers server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Awaiting configuration |
| Prometheus | Metrics aggregator | Private | Pythagoras-b | TBD | - | Not configured | Configured | Configured | Partial |
| Loki | Log aggregator | Private | Pythagoras-b | TBD | - | Not configured | Configured | Configured | Partial |
| Grafana | Graph visualizer | Public | Pythagoras-b | - | - | Not configured | Configured | Configured | Partial |
| Satisfactory | Satisfactory server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Not needed for v1 |
| Space engineers | Space engineers server for friends | Public | Archimedes | - | Game map | Not configured | Not configured | Not configured | Not needed for v1 |
| Raspsnir | Bachelor memorial website | Public | Pythagoras-b | PostgreSQL | - | Not configured | Not configured | Not configured | Not needed for v1 |
\* Configuration panel only available internally<br>
** Current implementation only support SQLite, making manual backups a necessity<br>
@ -57,7 +57,7 @@ longhorn
- Change host/deployment specific variables to use environment variables
- Write CI/CD pipeline to create environment loaded files
- Write CI/CD pipeline to deploy cluster
- Setup internal traefik with nodeport as reverse proxy for internal only services
- ~~Setup internal traefik with nodeport as reverse proxy for internal only services~~ Done through internal LB
- ~~Setup DB container sidecars for automated backups to Longhorn volume~~
- Setup secrets configuration through CI/CD variable injection
- Explore permission issues when issuing OVH API keys (not working for wildcard and `beta.halia.dev` subdomain)
@ -95,3 +95,13 @@ Add longhorn storage classes
### Gitlab backup process
Because gitlab does not offer the possibility to backup a container's data from an external container, a cronjob has been implemented in the custom image used for deployment.
### VPN configuration for Deluge
Instead of adding an extra networking layer to the whole cluster, it seems like a better idea to just integrate a wireguard connection inside of the deluge image, and self-build everything within Gitlab registry.
This image could utilize kubernetes secrets, including a "torrent-vpn" secret produces by the initial wireguard configuration done via Ansible.
This ansible script could create one (or more) additional client(s) depending on the inventory configuration, and keep the "torrent-vpn" configuration file within a k3s formated file, inside of the auto-applied directory on CP.
Cf : https://docs.k3s.io/advanced#auto-deploying-manifests
### Development domains
To access a service publicly when developping, the domain name should be *.beta.halia.dev
To only expose a service internally, the domain name should be *.k3s.beta

12
metallb/configmap.yaml Normal file
View File

@ -0,0 +1,12 @@
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
ipaddress-pools:
- name: default
protocol: layer2
addresses:
- 10.20.0.1-10.20.0.100

9
metallb/ipaddresspool.yaml Normal file
View File

@ -0,0 +1,9 @@
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: base
namespace: metallb-system
spec:
addresses:
- 10.19.66.247/32
- 10.10.0.64-10.10.0.69

14
traefik/dashboard.yaml
View File

@ -1,18 +1,4 @@
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: internal-ipwhitelist
spec:
ipWhiteList:
sourceRange:
- 10.10.0.1/24
- 10.20.0.1/24
- 10.42.1.1/24
ipStrategy:
depth: 0
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik-dashboard

2
traefik/deployment.yaml
View File

@ -94,7 +94,6 @@ spec:
mountPath: /tmp
args:
- "--global.checknewversion"
- "--global.sendanonymoususage"
- "--entrypoints.admin.address=:8080/tcp"
- "--entrypoints.metrics.address=:9100/tcp"
- "--entrypoints.minecrafttcp.address=:25565/tcp"
@ -110,6 +109,7 @@ spec:
- "--entrypoints.web.http.redirections.entryPoint.to=:443"
- "--entrypoints.web.http.redirections.entryPoint.scheme=https"
- "--log.level=DEBUG"
- "--accesslog=true"
- "--entrypoints.websecure.http.tls=true"
- "--entrypoints.websecure.http.tls.certresolver=letsencrypt"
- "--entrypoints.websecure.http.tls.domains[0].main=beta.halia.dev"

11
traefik/service.yaml → traefik/service-external.yaml
View File

@ -3,28 +3,25 @@
apiVersion: v1
kind: List
metadata:
name: traefik
name: traefik-external
items:
- apiVersion: v1
kind: Service
metadata:
name: traefik
name: traefik-external
labels:
app.kubernetes.io/name: traefik
app.kubernetes.io/name: traefik-external
helm.sh/chart: traefik-10.24.2
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/instance: traefik
annotations:
spec:
type: LoadBalancer
loadBalancerIP: 10.19.66.247
selector:
app.kubernetes.io/name: traefik
app.kubernetes.io/instance: traefik
ports:
- port: 8080
name: admin
targetPort: "admin"
protocol: TCP
- port: 25565
name: minecrafttcp
targetPort: "minecrafttcp"

36
traefik/service-internal.yaml Normal file
View File

@ -0,0 +1,36 @@
---
# Source: traefik/templates/service.yaml
apiVersion: v1
kind: List
metadata:
name: traefik-internal
items:
- apiVersion: v1
kind: Service
metadata:
name: traefik-internal
labels:
app.kubernetes.io/name: traefik-internal
helm.sh/chart: traefik-10.24.2
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/instance: traefik
annotations:
spec:
type: LoadBalancer
loadBalancerIP: 10.10.0.64
selector:
app.kubernetes.io/name: traefik
app.kubernetes.io/instance: traefik
ports:
- port: 25565
name: minecrafttcp
targetPort: "minecrafttcp"
protocol: TCP
- port: 80
name: web
targetPort: "web"
protocol: TCP
- port: 443
name: websecure
targetPort: "websecure"
protocol: TCP