Here we will understand how to deploy GitLab over a Kubernetes cluster.
For this installation we will need:
What I'm using:
Create the namespace for our installation
kubectl create ns gitlab
Select the destination namespace (in this way we will avoid to specify the namespace at each command)
kubens gitlab
According to the GitLab official documentation the Helm chart is the best way to deploy GitLab on a Kubernetes cluster. At this repository we can find the full chart, specially the default value file. We can get these configuration also with this command:
helm inspect gitlab/gitlab
Before to deploy the chart we will write our override file to change some parameter contained in the file above.
Here a commented version of the override file override.yaml that I'm using in this tutorial
# Domain settings, this is the root domain, with this configuration I'll get the following endpoints:
# - gitlab.tanuki-demo.com
# - minio.tanuki-demo.com
# - registry.tanuki-demo.com
# You have to setup this records on your DNS to point to the Ingress that we are going to create
global:
hosts:
domain: tanuki-demo.com
https: true
# Certmanager issuer email to use Let's Encrypt for SSL certificates
certmanager-issuer:
email: admin@tanuki-demo.com
# Prometheus configuration to enable more metrics in our deployment
prometheus:
kubeStateMetrics:
enabled: true
nodeExporter:
enabled: true
# The GitLab runners will be executed in privileged mode, this allow the Docker-in-Docker executor -> https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-executor
gitlab-runner:
runners:
privileged: true
All the other values will be inherited from the default values that we found in the repo.
Now we can proceed with the deployment
helm repo add gitlab https://charts.gitlab.io/
helm repo update
helm upgrade --install -f override.yaml gitlab gitlab/gitlab
If we need to change some values in our override we can just update the file and relaunch the last command..
During the installation an Ingress endpoint will be created, you have to take note of the public IP address and update your DNS records according this.
After the update the DNS will take a while for the propagation, only after that the CertManager can complete its tasks.
At the end of this process your installation will be available, to get the default admin password you can use:
kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo
To verify that all our services are up and running we can use this preflight list:
Some components in our deployment will autoscale basing on the load of the relative container, to discover for which of our services the autoscaling has been configured we can use this command:
kubectl get horizontalpodautoscalers.autoscaling
Here the output that I get:
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
gitlab-gitlab-shell Deployment/gitlab-gitlab-shell 1m/100m 2 10 2 3h44m
gitlab-registry Deployment/gitlab-registry 2%/75% 2 10 2 3h44m
gitlab-sidekiq-all-in-1 Deployment/gitlab-sidekiq-all-in-1 35m/350m 1 10 1 3h44m
gitlab-unicorn Deployment/gitlab-unicorn 7m/1 2 10 2 3h44m
As you can see for each service you have the actual load and the target expected for the autoscale, with the min and max replicas allowed for that service. We can change these values according to our needs.
Now we have a working GitLab and Prometheus installation, some metrics are available inside the GitLab UI but if we want to access to all metrics we need a dedicated UI, like Grafana.
Also in this case we will proceed using Helm and creating before an override file, we will deploy Grafana inside the same namespace of GitLab.
Here my grafana-override.yaml
grafana.ini:
server:
domain: grafana.tanuki-demo.com
We can set on the DNS that grafana.tanuki-demo.com is a CNAME of gitlab.tanuki-demo.com.
Now we can proceed with the installation:
helm upgrade --install -f grafana-override.yaml grafana stable/grafana
With these parameters an Ingress for Grafana will not be created, we will create manually a rule to add this endpoint to the existing Ingress.
Here the ingress yaml grafana-ingress-rule.yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
certmanager.k8s.io/issuer: gitlab-issuer
kubernetes.io/ingress.class: gitlab-nginx
kubernetes.io/ingress.provider: nginx
generation: 1
labels:
app: grafana
name: gitlab-grafana
namespace: gitlab
spec:
rules:
- host: grafana.tanuki-demo.com
http:
paths:
- backend:
serviceName: grafana
servicePort: 80
path: /
tls:
- hosts:
- grafana.tanuki-demo.com
secretName: gitlab-grafana-tls
And we can proceed with the deployment:
kubectl create -f grafana-ingress-rule.yaml
Now a new SSL certificate will be generated from the CertManager and our endpoint will be available, to get the default credentials:
kubectl get secret grafana -ojsonpath='{.data.admin-user}' | base64 --decode ; echo
kubectl get secret grafana -ojsonpath='{.data.admin-password}' | base64 --decode ; echo
After the login we can start adding our Prometheus endpoint, in the Datasource setting page we create the new Prometheus datasource and we add in the URL http://gitlab-prometheus-server.
This is the service name of our Prometheus server, we can find it with:
kubectl get services | grep prometheus-server | awk '{ print "http://"$1 }'
And leaving at the default the other settings we can click on Save & Test.
From the Dashboards tab we can import the defaults dashboards for this datasource.
On the Grafana website you can find a lot of interesting dashboard, like this one that it's working by default on my environment.
For the other Grafana settings I suggest to give a look to the default values file, like we made for GitLab.
With this tutorial we deployed PostGreSQL and Redis on containers, all our data are stored on 5 different PV on Google Cloud, be sure that you are confident managing this kind of storage and database installation. If not many providers let you have PostGreSQL and Redis as a service, in this way you don't have to take care of this part of the infrastructure, but new configurations in our override are required.