How to setup Gitlab CI/CD ?

We’ve been using Jenkins since 2014. Jenkins serves us well for the last couple of years. But with the rise of CI/CD, there are more tools providing rich full feature for CICD integration. Gitlab CICD is a new player from a well-known vendor.

This article won’t go into details about the pros and cons of Gitlab CICD or the feature it provides. This will be covered by other articles following this one.

In this article, we just want to detail the installation procedure we followed and the trouble encountered while installing Gitlab CICD on Docker Swarm. While experimenting to automate the installation, we struggled to use the default Gitlab runner image. The image provided by Gitlab doesn’t provide any way to automatically register and unregister the runner dynamically. That’s why we provide our own Docker image to do this job seamlessly.

Overview

The following block diagram presents an overview of the deployment. We believe it’s a fairly standard configuration that should be useful to most people getting started with Gitlab CI/CD.

Gitlab CICD block diagram

We have 3 physical servers named: master1, worker1 and worker2. - Docker swarm is deployed on all physical servers - Gitlab is pinned on the master1 - Gitlab Runner is deployed as global service on all worker nodes.

Take note, our solution is also working with standalone Docker installations. Docker Swarm is not required !

Installation

To deploy Gitlab CI/CD we planned to use a docker-compose.yml file, but that proved to be difficult. First, we need to configure Gitlab, then retrieve a Gitlab runner token and make use of this token to deploy the runners. Basically, we need to proceed in two steps.

  1. Deploy Gitlab (only). Configure it and get the token.

  2. Deploy Gitlab Runners with the token.

Deploy Gitlab

Here’s our docker-compose.yml file and like any other docker-compose you are invited to adapt it to your needs. To get more information about how to use it, take a look at our Github project: https://github.com/ikus060/docker-gitlab-runner

To follow Linux directory file structure, we decide to store our data into /srv directory on the master1 server. As a prerequisite step, we need to create the directories:

mkdir -p /srv/gitlab/config
mkdir -p /srv/gitlab/log
mkdir -p /srv/gitlab/data

Next, you can clone the git repository and make use of docker-compose.yml file to start Gitlab.

git clone https://github.com/ikus060/docker-gitlab-runner.git
cd docker-gitlab-runner 
GITLAB_URL= http://gitlab.example.com docker-compose up web

When starting Gitlab for the first time, be patient. It takes a while to get it started. Once started, you may use your web browser. If your DNS is appropriately configured, you may browse to http://gitlab.example.com. Otherwise, use the IP address.

The first time you connect to Gitlab, you will be prompted to define a password for root user. Afterward, you can login using this password.

Gitlab first login screen

Final step, you need to get the registration token from http://gitlab.example.com/admin/runners/

Gitlab Admin Runner screen

Deploy Gitlab Runners

Once you get your hand on the registration token, you are ready to deploy Gitlab runners. Using the previous git repository, you may start the runner as follows.

GITLAB_URL= http://gitlab.example.com GITLAB_RUNNER_TOKEN=Qz3sxs3x5ZP6xNMTw9bA docker-compose up runner

You should see your runner starting.

It should also appear in Gitlab Web Interface.

Gitlab Admin Runner screen with runner

Advance configuration

While the previous step is enough to get you started and have a “working” Gitlab with Runners, it’s not enough for real production environment. This section will provide references to continue tweaking your Gitlab configuration for your needs.

Gitlab Configuration

Most Gitlab configuration is done through the GITLAB_OMNIBUS_CONFIG environment variable. Full documentation of omnibus is available here. Basically, anything you would have added to /etc/gitlab/gitlab.rb, you must add it to GITLAB_OMNIBUS_CONFIG. Let have a look at this example:

[...]

services:
  web:
    image: 'gitlab/gitlab-ce:latest'
    restart: always
    hostname: 'git.patrikdufresne.com'
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        # If your gitlab should be accessible via a DNS. You must provide
        # it to gitlab. Also, if you want to enable Gitlab pages, your
        # should also define another DNS.
        external_url 'http://gitlab.example.com'
        pages_external_url 'http://pages.example.com'

        # We decide to configure LDAP authentication. Gitlab provide good
        # documentation and example how to configure this part.
        # https://docs.gitlab.com/omnibus/settings/ldap.html
        # This is our LDAP configuration for OpenLDAP.
        gitlab_rails['ldap_enabled'] = true
        gitlab_rails['ldap_host'] = 'ldap.example.com'
        gitlab_rails['ldap_port'] = 389
        gitlab_rails['ldap_uid'] = 'uid'
        gitlab_rails['ldap_method'] = 'plain'
        gitlab_rails['ldap_bind_dn'] = 'cn=admin,dc=example,dc=com'
        gitlab_rails['ldap_password'] = 'CHANGEME'
        gitlab_rails['ldap_allow_username_or_email_login'] = true
        gitlab_rails['ldap_base'] = 'dc=example,dc=com'

        # Configure timezone
        # Do I need to provide more info ?
        gitlab_rails['time_zone'] = 'America/Montreal'

        # Gmail config
        # Gitlab provide very good notification system through email.
        # This is our configuration using Gmail account. If you are
        # not using gmail, have a look here for more example.
        # https://docs.gitlab.com/omnibus/settings/smtp.html
        gitlab_rails['smtp_enable'] = true
        gitlab_rails['smtp_address'] = "smtp.gmail.com"
        gitlab_rails['smtp_port'] = 587
        gitlab_rails['smtp_user_name'] = "noreply@example.com"
        gitlab_rails['smtp_password'] = "CHANGEME"
        gitlab_rails['smtp_domain'] = "smtp.gmail.com"
        gitlab_rails['smtp_authentication'] = "login"
        gitlab_rails['smtp_enable_starttls_auto'] = true
        gitlab_rails['smtp_tls'] = false
        gitlab_rails['smtp_openssl_verify_mode'] = 'peer'

    ports:
      - '80:80'
      - '443:443'
    volumes: 
      - '/srv/gitlab/config:/etc/gitlab'
      - '/srv/gitlab/log:/var/log/gitlab'
      - '/srv/gitlab/data:/var/opt/gitlab'
    networks:
    - ci
[...]

Hopefully, Gitlab provides good documentation. Most step describes in this article is provided by Gitlab documentation, but the information may be spread.

References:

https://docs.gitlab.com/omnibus/docker/