# Gitlab CI/CD

# Terminology

  • Runner or Gitlab Runner is the server on which the CI commands are executed.
  • Runner executor is how the runner executes commands.
  • Pipeline is a set of stages through which a project is built or tested.
  • Job is some action in the Pipeline, e.g. the launch of unit tests.
  • Stage is a Pipeline stage that can simultaneously execute some jobs.
  • CI / CD is shortened from Continuous Integration and Continuous Deployment.

# Connecting the project to CI / CD

Read official documentation (opens new window).

# General recommendations

  1. We recommend keeping the files used for CI in the directory ./ci.
  2. If for the tests launch you need to deploy the database or any other services use [services] (https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#define-image-and-services-from-gitlab-ciyml (opens new window))
  3. Do not use Makefile to run commands in gitlab-ci. Makefile is used only as a developer tool.
  4. If you need to write conditions or a multi-line script, move the code into a separate bash file and put it inci / bin /.
  5. If you need to transfer artifacts from one stage to another, use artifacts anddependencies.
  6. Be sure to set a reasonable expires_in for artifacts. At least for a few hours. Documentation (opens new window).
  7. Write stage and job names in lowercase snake style (opens new window).
  • preparation - loading dependencies.
  • quality_checks - running tests, linters and static analyzers.
  • security_checks - checking dependencies for vulnerabilities, e.g.: npm audit.
  • build - artifact build, e.g. a docker image.
  • deploy - deploy to a cluster or ansible launch.
  • deploy_checks - check that the deployment was successful. Otherwise, roll back.

# CI/CD Checklist

# Tags

  • docker - container execution, documentation (opens new window).
  • shell - execution in a shell. You can use only those utilities that were preinstalled for the runner.
  • ansible - if you need to run Ansible.
  • ruby - if ruby is required. We recommend using docker with theimage: ruby declaration.

# Push images to private docker registry

  1. Authorization

    before_script:      docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY};

CI_JOB_TOKEN and CI_REGISTRY - are already available (opens new window) when running a pipeline in CI / CD.

  1. Publish with a tag containing the full address of the image, doc (opens new window):

    docker build -t GITLAB_REGISTRY:1443/group/subgroup/project . docker push GITLAB_REGISTRY:1443/group/subgroup/project

# How to push git tag on CI / CD launch

Gitlab has not yet implemented (opens new window) a mechanism for creating tags via CI, but you can use the API:

#!/usr/bin/env bash
set -e -x -o pipefail

curl -X POST --show-error --fail "https://${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/repository/tags?ref=${CI_COMMIT_SHA}&tag_name=${APP_VERSION}&private_token=${GITLAB_TOKEN}"
  • GITLAB_URL - GitLab URL.
  • CI_PROJECT_ID - project ID. Forwarded by Gitlab CI itself.
  • CI_COMMIT_SHA - commits SHA. Forwarded by Gitlab CI itself.
  • APP_VERSION - product or deployment version.
  • GITLAB_TOKEN - Personal Access Tokens. It is advisable not to use your own token, but create the token for the Bot user.

# DRY

# Improving execution speed

  1. Run independent jobs in parallel simultaneously:

    stages:

    • checks

    check_1: stage: checks script: ... check_2: stage: checks script: ...

  2. Use cache (opens new window) and artifacts/dependencies (opens new window). Cache to cache the result and reuse it in the same job. Artifacts - for using the result in other jobs.

  3. Use parallel (opens new window).

# FAQ

# How to debug failed jobs?

Read the logs

Go to a Project > CI / CD > Jobs

Try debug mode

  1. Add to script on failed job sleep 10000.
  2. Run pipeline.
  3. Press Debug button on the running job page.

Debug ENVs

Add to the .gitlab-ci.yml:

do_something:
  variables:
    CI_DEBUG_TRACE: "true"

Run gitlab-runner locally

You can run gitlab-runner locally, but it has a number of things to mention:

  1. The pod must be committed, because checkout is done during the execution.
  2. extends are not supported.
  3. Global (before|after)_script, image, services are not supported.
  4. Local cache is not supported. Connection to minio/s3 (opens new window) can be configured.

Installation https://docs.gitlab.com/runner/install (opens new window)

Then you can run jobs locally.

An example with docker-executor:

gitlab-runner exec docker do_something

An example with shell-executor:

gitlab-runner exec shell do_something