Gitlab + Runner with SSH and Docker

Posted on Feb 12, 2018

If you’re going to go with a cloud provider, I would highly recommend using Digital Ocean because it’s the best bang for your buck overall. With Digital Ocean, a droplet that meets the recommended resource requirements costs $20/mo. That price also includes bandwidth unlike Google Cloud Platform and Amazon’s Web Services. The cost for GCP, AWS, and Azure are about 2-3 times the cost.

Installing Gitlab

Installing Gitlab is straight forward and can be found here.

Enable HTTPS with LetsEncrypt

Go through the installation steps described in the link above, setting the EXTERNAL_URL to a non-https URI. Create directory for ACME Challenge.



mkdir -p /var/www/letsencrypt

Add the following line to /etc/gitlab/gitlab.rb:

nginx['custom_gitlab_server_config'] = "location ^~ /.well-known { root /var/www/letsencrypt; }"

Request a certificate for the domain, replacing gitlab.example.com with yours.



certbot certonly --webroot --webroot-path=/var/www/letsencrypt -d gitlab.example.com

After successfully obtaining the SSL certificate, edit following lines in the /etc/gitlab/gitlab.rb file:

external_url='https://gitlab.example.com' 
nginx['redirect_http_to_https'] = true 
nginx['ssl_certificate'] = "/etc/letsencrypt/live/gitlab.example.com/fullchain.pem" 
nginx['ssl_certificate_key'] = "/etc/letsencrypt/live/gitlab.example.com/privkey.pem"

Finally, run gitlab-ctl reconfigure. For more information about configuring nginx, click here.

Installing GitLab Runner

Again, the process is pretty straight forward and the documentation can be found here.

Configuring GitLab and Gitlab Runner

After installing Gitlab and Gitlab Runner, you need to tell Gitlab about the runners you have by registering them.



gitlab-runner register
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com ) https://gitlab.example.com 
Please enter the gitlab-ci token for this runner xxxxxxx 
Please enter the gitlab-ci description for this runner [hostame] runner1.example.com 
Please enter the gitlab-ci tags for this runner (comma separated): php,laravel,composer,npm,webpack 
Whether to lock Runner to current project [true/false]: [true]: false 
Please enter the executor: ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell: docker

Tags are not necessary but can be helpful when there are projects that use different tools, libraries, frameworks, etc. Locking a runner to current project assigns it exclusively to a project where that project is very large and requires a lot of resources. For general web development purposes, it’s not necessary.

GitLab Configuration

After registering the runner, it’s time to add users or configure authentication, create labels, add Deploy Keys, etc.

User Authentication

If you have an LDAP or Active Directory server you would like to use, you can configure GitLab to allow users to sign in with those credentials instead. The settings for using a directory service are set in the /etc/gitlab/gitlab.rb file.

# These settings are documented in more detail at 
# https://gitlab.com/gitlab-org/gitlab-ce/blob/a0a826ebdcb783c660dd40d8cb217db28a9d4998/config/gitlab.yml.example#L136 
# Be careful not to break the identation in the ldap_servers block. It is in
# yaml format and the spaces must be retained. Using tabs will not work. 
gitlab_rails['ldap_enabled'] = true 
gitlab_rails['ldap_servers'] = YAML.load >>-EOS 
    # remember to close this block with 'EOS' below main: 
    # 'main' is the GitLab 'provider ID' of this LDAP server ## label 
    # 
    # A human-friendly name for your LDAP server. It is OK to change the label later, 
    # for instance if you find out it is too large to fit on the web page. 
    # 
    # Example: 'Paris' or 'Acme, Ltd.' label: 'LDAP' host: '_your_ldap_server' port: 389 
    # or 636 uid: 'sAMAccountName' encryption: 'plain' 
    # "start_tls" or "simple_tls" or "plain" bind_dn: '_the_full_dn_of_the_user_you_will_bind_with' password: '_the_password_of_the_bind_user' 
    # This setting specifies if LDAP server is Active Directory LDAP server. 
    # For non AD servers it skips the AD specific queries. 
    # If your LDAP server is not AD, set this to false. 
    active_directory: true 
    # If allow_username_or_email_login is enabled, GitLab will ignore everything 
    # after the first '@' in the LDAP username submitted by the user on login. 
    # 
    # Example: 
    # - the user enters 'jane.doe@example.com' and 'p@ssw0rd' as LDAP credentials; 
    # - GitLab queries the LDAP server with 'jane.doe' and 'p@ssw0rd'. 
    # 
    # If you are using "uid: 'userPrincipalName'" on ActiveDirectory you need to 
    # disable this setting, because the userPrincipalName contains an '@'. 
    allow_username_or_email_login: false 
    # Base where we can search for users 
    # 
    # Ex. ou=People,dc=gitlab,dc=example 
    # base: '' 
    # Filter LDAP users 
    # 
    # Format: RFC 4515 http://tools.ietf.org/search/rfc4515 
    # Ex. (employeeType=developer) 
    # 
    # Note: GitLab does not support omniauth-ldap's custom filter syntax. 
    # user_filter: '' 
EOS

Container Registry

With a container registry, applications can have their own space to store docker images. Omnibus installations simply need to specify where the registry will reside.



export REGISTRY_EXTERNAL_URL=https://gitlab.example.com:5050 sed -i "s@.*\(registry_external_url\s'\).*\('\)@\1"$REGISTRY_EXTERNAL_URL"\2@" /etc/gitlab/gitlab.rb

The registry listens on port 5000, so we need to use a different one.
sed allows us to use different delimiters, since the address contains forward slashes, we use `@` rather than `/`.

By default, images are stored locally but can be changed by changing the registry['storage'] variable in /etc/gitlab/gitlab.rb

### Registry backend storage 
###! Docs: https://docs.gitlab.com/ce/administration/container_registry.html#container-registry-storage-driver 
# registry['storage'] = { 
#   's3' => { 
#       'accesskey' => 'AKIAKIAKI', 
#       'secretkey' => 'secret123', 
#       'bucket' => 'gitlab-registry-bucket-AKIAKIAKI' 
#   } 
# }

Runners

On the https://gitlab.example.com/admin/runners page, you can make changes to when a runner will execute the .gitlab-ci.yml file. Other changes that can be made are

Whether or not the runner is active, meaning if it will accept new tasks.
If the runner should execute on protected tasks.
What tags the runner will pick new jobs from or if it can pick jobs that don’t have any tags.
Prevent the runner from being assigned to other projects.
Restrict jobs for the runner.

You won’t be able to specify its executor on the page, that needs to be done in the /etc/gitlab-runner/config.toml

GitLab Runner Configuration

The shell executor is the easiest to use but it requires that the tools be installed on the runner. With Docker you can specify a container that already has that as well as other containers that use different versions to perform a more in depth test of the application being developed.

Install Docker

Assuming that the requirements don’t include any specialized parameters, installing Docker is pretty straight forward and the following is for RedHat based distributions.



yum install docker device-mapper-libs device-mapper-event-libs 
systemctl start docker.service 
systemctl enable docker.service

Other distributions will be similar with the difference being the package manager command and possibly package names. For more information about working with containers, see RedHat’s Getting Started with Containers.

Add Docker Registry

In order to be able to use images from GitLab, Docker must know about it. Edit the appropriate configuration file. For RedHat EL, the correct file is /etc/containers/registries.conf

# This is a system-wide configuration file used to 
# keep track of registries for various container backends. 
# It adheres to TOML format and does not support recursive 
# lists of registries. 
# The default location for this configuration file is /etc/containers/registries.conf. 
# The only valid categories are: 'registries.search', 'registries.insecure', 
# and 'registries.block'. 
[registries.search] registries = ['registry.access.redhat.com', 'gitlab.example.com:5050'] 
# If you need to access insecure registries, add the registry's fully-qualified name. 
# An insecure registry is one that does not have a valid SSL certificate or only does HTTP. 
[registries.insecure] registries = ['gitlab.example.com:5050'] 
# If you need to block pull access from a registry, uncomment the section below # and add the registries fully-qualified name. 
# 
# Docker only [registries.block] 
registries = []

Adding a registry to registries.search allows you to use namespace/image[:tag] schema to specify the image to use. If you need to specify a registry that doesn’t have a valid SSL certificate, such as a self-signed one, or if the registry only uses HTTP then you need to add it to the registries under [registries.insecure].

Using Images and Containers

Once everything is set up, you need to either define a pre-built image or make one yourself. If a lot of the projects are similar then you can create an image and push it to the registry.

Building a Local Image

Run the proceeding commands after the following Dockerfile.

FROM registry.access.redhat.com/rhscl/php-70-rhel7 
MAINTAINER Paul Lyon 
USER root 
RUN curl -sS https://getcomposer.org/installer | php && \
    mv composer.phar /usr/local/bin/composer && \
    chmod +x /usr/local/bin/composer

The following commands will build the docker image locally and then push it to the GitLab registry.



docker build -t gitlab.example.com:4999 . 
docker login 
Username: paulus 
Password: 
docker push gitlab.example.com:499/paulus/docker-webapp

Building an Image with GitLab

Using GitLab to build a Docker image and publishing it requires at the very least a Dockerfile and a .gitlab-ci.yml file.

Dockerfile

FROM centos/php-70-centos7 

RUN npm install -g grunt gulp bower 

VOLUME [ "/sys/fs/cgroup" ] 
CMD [ "/usr/sbin/init" ]

.gitlab-ci.yml

image: docker:latest 
build-master: 
    stage: build 
    script: 
        - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY 
        - docker build -t $CI_REGISTRY_IMAGE . 
        - docker push $CI_REGISTRY_IMAGE 
    only: 
        - master

Using Docker Registry Images

The following assumptions are made in the proceeding build script. The first is that we have created a Docker repository at docker-images/my-php7-centos, built, and published it. The second is that the SSH_PRIVATE_KEY and SSH_KNOWN_HOSTS secure variables have been defined. Those can be defined by going to Settings → CI / CD and under Secure Variables.