Gitlab is a great CI/CD tool along with all the benefits of being a SCM tool as well. Having said that, if you are self hosting Gitlab, and want to control the amount of storage, then a little bit of housekeeping is required to remove old pipelines and artefacts.
One way to do that is to run a scheduled job. Below is a gitlab-ci.yml file that runs a job to clean the pipelines and runs on a custom schedule schedule1.
### STAGES ###
stages:
- Delete Old Pipelines
.delete_old_pipelines: &delete_old_pipelines
artifacts:
paths:
- file.log
script:
- sudo apt-get update
- sudo apt-get install -y jq
- sudo chmod +x cleanup_old_pipelines.sh && ./cleanup_old_pipelines.sh
### JOBS ###
gitlab:delete_old_pipelines:
needs: []
stage: Delete Old Pipelines
<<: *delete_old_pipelines
tags:
- ubuntu
rules:
- if: '$CI_PIPELINE_SOURCE == "schedule" && $CI_COMMIT_REF_NAME == "schedule1"'
when: always
This gitlab-ci.yml file defines a CI/CD pipeline in GitLab with a stage, “Delete Old Pipelines”. This stage consists of a job that is executed when certain conditions are met. The job uses YAML anchors and aliases to reuse configurations, which helps in keeping the file DRY (Don’t Repeat Yourself).
Here’s an analysis of each section and their purpose:
STAGES
Defines the stages that will be used in the CI/CD pipeline, which are executed sequentially.
.delete_old_pipelines (YAML anchor):
Artifacts: Specifies file.log as an artifact that will be passed to later stages.
Script: Lists commands to update the package list, install jq (a lightweight and flexible command-line JSON processor), and execute the cleanup_old_pipelines.sh script after making it executable.
JOBS
Defines actual jobs that are executed during the pipeline run.
gitlab:delete_old_pipelines:
needs: []: Indicates this job does not have dependencies on other jobs.
stage: The stage in which this job is executed, which is “Delete Old Pipelines”.
<<: *delete_old_pipelines: YAML alias that inserts the configuration defined by the .delete_old_pipelines anchor.
tags: Specifies that the job should run on a runner tagged with ubuntu.
rules: Specifies that this job will run only if the pipeline was triggered by a schedule and if the branch name is schedule1.
The job is designed to always run when its conditions is met, regardless of whether previous stages have failed or not. This setup is useful for maintenance tasks that should be run independently of the pipeline’s success or failure, such as cleaning up resources or archiving artifacts. The conditional rules based on the pipeline source and commit reference name ensure that this job is executed only during scheduled runs, likely for routine maintenance tasks, and for specific schedules (schedule1).