🇬🇧 GitLab + K3S: Deployment and Deploy Boards

In a previous blog post, I explained how I installed a "devops toolchain" on my laptop, with a registry, a GitLab instance and a K3S cluster (read it here: GitLab, K3S and Kubernetes executor on my laptop). I was very "short" on how to deploy a webapplication on this new cluster from GitLab CI.

So, I did some gardening and improved my sample: https://gitlab.com/k33g/little-local-devops-toolchain/-/tree/master/webapp-sample.

The .gitlab-ci.yml is pretty simple (open an issue if you have any problem: https://gitlab.com/k33g/little-local-devops-toolchain/-/issues).

There are 2 stages:

stages:
  - 🐳build
  - 🚢deploy

and 4 jobs:

# build the docker image
📦kaniko-build:
  stage: 🐳build

# deploy to cluster from master branch
🚀production-deploy:
  stage: 🚢deploy

# deploy to cluster on a different namespace from feature branch
🎉preview-deploy:
  stage: 🚢deploy

# remove the review application from the cluster
😢stop-preview-deploy:
  stage: 🚢deploy

To be able to deploy the application to the cluster, you need to create 2 CI Variables in your project (or in the group of your project):

  • KUBECONFIG, define it as a file and use the content of k3s.yaml to fill the field
  • CLUSTER_IP: it's the IP of the Cluster's VM

👋 Be sure to use the correct registry (check the Dockerfile too)

Deploy Boards

If you are operating a Premium GitLab instance or a Silver organisation, you can use the Deploy Boards on Kubernetes displaying the status of the pods in the deployment.

To enable the Deploy Boards you need to use:

  • This variable KUBE_NAMESPACE, its content is generated by GitLab. The GitLab runner will create a namespace automatically with KUBE_NAMESPACE and deploy your application inside this namespace. See Deployment variables
  • This 2 kubernetes annotations:
    • app.gitlab.com/env: $CI_ENVIRONMENT_SLUG
    • app.gitlab.com/app: $CI_PROJECT_PATH_SLUG

The annotations will applied to the deployments, replica sets, and pods.

  • the KUBE_NAMESPACE's format looks like this: <project_name>-<project_id>-<environment>
  • $CI_ENVIRONMENT_SLUG and $CI_PROJECT_PATH_SLUG are the values of the CI variables

Where to use the annotations?

It's easy. Go to webapp-sample/kube/deploy.template.yaml and add the annotations like that in the Deployment section:

for the Deployment

# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${APPLICATION_NAME}
  annotations:
    app.gitlab.com/app: ${CI_PROJECT_PATH_SLUG} # here 👋
    app.gitlab.com/env: ${CI_ENVIRONMENT_SLUG}  # here 👋
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ${APPLICATION_NAME}
  template:
    metadata:
      labels:
        app: ${APPLICATION_NAME}
      annotations:
        app.gitlab.com/app: ${CI_PROJECT_PATH_SLUG} # here 👋
        app.gitlab.com/env: ${CI_ENVIRONMENT_SLUG}  # here 👋
    spec:
      containers:
        - name: ${APPLICATION_NAME}
          image: ${IMAGE}
          ports:
            - containerPort: ${CONTAINER_PORT}
          imagePullPolicy: Always

btw, I already committed the file.

Deploy the WebApp

If you look at the .gitlab-ci.yml, you have all what you need to deploy on master or to deploy review application thanks to a MR:

stages:
  - 🐳build
  - 🚢deploy

variables:
  REGISTRY: "little-registry.test:5000"
  DOCKER_USER: "little-registry.test:5000"

#----------------------------
# Build Docker image
#----------------------------
📦kaniko-build:
  image:
    name: ${REGISTRY}/gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
  stage: 🐳build
  script: |
    IMAGE_NAME="${CI_PROJECT_NAME}-img"
    TAG="${CI_COMMIT_SHORT_SHA}"
    DOCKER_USER="${DOCKER_USER}"
    IMAGE="${DOCKER_USER}/${IMAGE_NAME}:${TAG}"

    echo '{"auths":{"${REGISTRY}":{"username":"","password":""}}}'  > /kaniko/.docker/config.json
    cat /kaniko/.docker/config.json
    
    /kaniko/executor \
    --context $CI_PROJECT_DIR \
    --dockerfile $CI_PROJECT_DIR/Dockerfile \
    --destination ${IMAGE} \
    --insecure --skip-tls-verify --insecure-pull
   
#----------------------------
# Deploy
#----------------------------
#----------------------------
# YAML Anchors
#----------------------------
.environment-variables: &environment-variables
- |
  export CONTAINER_PORT=${CONTAINER_PORT:-8080}
  export EXPOSED_PORT=${EXPOSED_PORT:-80}
  export APPLICATION_NAME=${CI_PROJECT_NAME}
  # 🖐️ KUBE_NAMESPACE is automatically generated by GitLab
  export NAMESPACE=${KUBE_NAMESPACE} 
  export TAG=${CI_COMMIT_SHORT_SHA}
  export IMAGE_NAME="${CI_PROJECT_NAME}-img"
  export DOCKER_USER="${DOCKER_USER}"
  export IMAGE=${DOCKER_USER}/${IMAGE_NAME}:${TAG}
  export CLUSTER_IP="${CLUSTER_IP}"
  export BRANCH=${CI_COMMIT_REF_SLUG}
  export HOST="${APPLICATION_NAME}.${BRANCH}.${CLUSTER_IP}.nip.io"

.environment-variables-substitution: &environment-variables-substitution
- |
  envsubst < ./kube/deploy.template.yaml > ./kube/deploy.${TAG}.yaml
  cat ./kube/deploy.${TAG}.yaml

.kubectl-apply: &kubectl-apply
- |
  kubectl apply -f ./kube/deploy.${TAG}.yaml -n ${NAMESPACE}

.kubectl-scale-by-3: &kubectl-scale-by-3
- |
  kubectl scale --replicas=3 deploy ${APPLICATION_NAME} -n ${NAMESPACE}

.display-information: &display-information
- |
  echo "🌍 http://${HOST}"
  kubectl get pods --namespace ${KUBE_NAMESPACE}


🚀production-deploy:
  stage: 🚢deploy
  image: ${REGISTRY}/k33g/k3g.utilities:1.0.0
  only:
    - master    
  environment:
    name: production/${CI_PROJECT_NAME}-${CI_COMMIT_REF_SLUG}
    url: http://${CI_PROJECT_NAME}.${CI_COMMIT_REF_SLUG}.${CLUSTER_IP}.nip.io
  script:
    - *environment-variables
    - *environment-variables-substitution
    - *kubectl-apply
    - *kubectl-scale-by-3
    - *display-information

🎉preview-deploy:
  stage: 🚢deploy
  image: ${REGISTRY}/k33g/k3g.utilities:1.0.0
  only:
    - branches
  except:
    - master   
  environment:
    name: preview/${CI_PROJECT_NAME}-${CI_COMMIT_REF_SLUG}
    url: http://${CI_PROJECT_NAME}.${CI_COMMIT_REF_SLUG}.${CLUSTER_IP}.nip.io
    on_stop: 😢stop-preview-deploy
  script:
    - *environment-variables
    - *environment-variables-substitution
    - *kubectl-apply
    - *display-information

😢stop-preview-deploy:
  stage: 🚢deploy
  image: ${REGISTRY}/k33g/k3g.utilities:1.0.0
  only:
    - branches
  except:
    - master     
  when: manual    
  environment:
    name: preview/${CI_PROJECT_NAME}-${CI_COMMIT_REF_SLUG}
    action: stop
  script:
    - kubectl delete namespace ${KUBE_NAMESPACE}

Commit and push an update of your application on master branch but on a feature branch too, and you'll get all your environments in the dashboard:

alt k9s

You will be able to reach your webapp with an URL like this one: http://hello-world.master.192.168.64.27.nip.io/ (<project-name>.<branch-name>.<cluster IP>.nip.io)

That's all for this part.

Last Articles

Last Updated: 3/19/2020, 7:37:47 AM