Skip to content

GitLab

·

Collect GitLab operation data and report it to Guance in the form of metrics.

Configuration

Collector Configuration

First, you need to open the data collection function of GitLab service and set the white list. See the following sections for specific operations.

After the GitLab setup is complete, configure the DataKit. Note that the data collected may vary depending on the GitLab version and configuration.

Go to the conf.d/gitlab directory under the DataKit installation directory, copy gitlab.conf.sample and name it gitlab.conf. Examples are as follows:

[[inputs.gitlab]]
    ## set true if you need to collect metric from url below
    enable_collect = true

    ## param type: string - default: http://127.0.0.1:80/-/metrics
    prometheus_url = "http://127.0.0.1:80/-/metrics"

    ## param type: string - optional: time units are "ms", "s", "m", "h" - default: 10s
    interval = "10s"

    ## datakit can listen to gitlab ci data at /v1/gitlab when enabled
    enable_ci_visibility = true

    ## Set true to enable election
    election = true

    ## extra tags for gitlab-ci data.
    ## these tags will not overwrite existing tags.
    [inputs.gitlab.ci_extra_tags]
    # some_tag = "some_value"
    # more_tag = "some_other_value"

    ## extra tags for gitlab metrics
    [inputs.gitlab.tags]
    # some_tag = "some_value"
    # more_tag = "some_other_value"

Once configured, restart DataKit.

The collector can now be turned on by ConfigMap injection collector configuration.

GitLab Turns on Data Collection

GitLab needs to turn on the Prometheus data collection function as follows (taking English page as an example):

  • Log in to your GitLab page as an administrator account
  • Go to Admin Area > Settings > Metrics and profiling
  • Select Metrics - Prometheus, click Enable Prometheus Metrics and save change
  • Restart the GitLab service

See official configuration doc.

Configure Data Access Whitelist

It is not enough to turn on the data collection function. GitLab is very strict with data management, so it is necessary to configure the white list on the access side. The opening mode is as follows:

  • Modify the GitLab configuration file /etc/gitlab/gitlab.rb, find gitlab_rails['monitoring_whitelist'] = ['::1/128'] and add the access IP of the DataKit to the array (typically the IP of the host where the DataKit resides, if the GitLab is running in a container, depending on the actual situation)
  • Restart the GitLab service

See official configuration doc.

Turn on GitLab CI Visualization

Ensure that the DataFlux Func platform is available.

By configuring GitLab Webhook, GitLab CI visualization can be achieved. Data reporting needs to be done through DataFlux Func, and the steps to enable it are as follows:

  1. Install the GitLab CI integration (script ID: guance_gitlab_ci) on DataFlux Func. Follow the installation process as referenced in GitLab CI Integration Configuration;
  2. In GitLab go to Settings > Webhooks, configure the URL to the API address obtained from step one. Trigger configure Job events and Pipeline events, and click Add webhook to confirm the addition;

Triggering the GitLab CI process will allow you to log in to Guance to view the execution status of CI after completion.

Metric

For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.gitlab.tags] if needed:

 [inputs.gitlab.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

We can specify additional tags for Gitlab CI data in the configuration by [inputs.gitlab.ci_extra_tags]:

 [inputs.gitlab.ci_extra_tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

Note: To ensure that GitLab CI functions properly, the extra tags specified for GitLab CI data do not overwrite tags already in its data (see below for a list of GitLab CI tags).

gitlab

GitLab runtime metrics

  • Tags
Tag Description
action Action
controller Controller
feature_category Feature category
storage Storage
  • Metrics
Metric Description Type Unit
banzai_cacheless_render_real_duration_seconds_count The count of duration of rendering Markdown into HTML when cached output exists float s
banzai_cacheless_render_real_duration_seconds_sum The sum of duration of rendering Markdown into HTML when cached output exists float s
cache_misses_total The cache read miss count float -
cache_operation_duration_seconds_count The count of cache access time float s
cache_operation_duration_seconds_sum The count of cache access time float s
cache_operations_total The count of cache access time float -
rack_requests_total The rack request count float -
redis_client_requests_duration_seconds_count The count of redis request latency, excluding blocking commands float s
redis_client_requests_duration_seconds_sum The sum of redis request latency, excluding blocking commands float s
redis_client_requests_total Number of Redis client requests float -
sql_duration_seconds_count The total SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT float s
sql_duration_seconds_sum The sum of SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT float s
transaction_cache_read_hit_count_total The counter for cache hits for Rails cache calls float count
transaction_cache_read_miss_count_total The counter for cache misses for Rails cache calls float count
transaction_db_cached_count_total The counter for db cache float count
transaction_db_count_total The counter for db float count
transaction_duration_seconds_count The count of duration for all transactions (gitlab_transaction_* metrics) float s
transaction_duration_seconds_sum The sum of duration for all transactions (gitlab_transaction_* metrics) float s
transaction_new_redis_connections_total The counter for new Redis connections float -
transaction_view_duration_total The duration for views float -

gitlab_base

GitLab programming language level metrics

  • Tags

NA

  • Metrics
Metric Description Type Unit
rails_queue_duration_seconds_count The counter for latency between GitLab Workhorse forwarding a request to Rails float s
rails_queue_duration_seconds_sum The sum for latency between GitLab Workhorse forwarding a request to Rails float s
ruby_gc_duration_seconds_count The count of time spent by Ruby in GC float s
ruby_gc_duration_seconds_sum The sum of time spent by Ruby in GC float s
ruby_sampler_duration_seconds_total The time spent collecting stats float s

gitlab_http

GitLab HTTP metrics

  • Tags
Tag Description
method 方法
status 状态码
  • Metrics
Metric Description Type Unit
http_health_requests_total Number of health requests float -
http_request_duration_seconds_count The counter for request duration float s
http_request_duration_seconds_sum The sum for request duration float s

gitlab_pipeline

GitLab Pipeline event metrics

  • Tags
Tag Description
author_email Author email
ci_status CI type
commit_sha The commit SHA of the most recent commit of the code that triggered the Pipeline
object_kind Event type, in this case Pipeline
operation_name Operation name
pipeline_name Pipeline name
pipeline_source Sources of Pipeline triggers
pipeline_url Pipeline URL
ref Branches involved
repository_url Repository URL
resource Project name
  • Metrics
Metric Description Type Unit
commit_message The message attached to the most recent commit of the code that triggered the Pipeline. string -
created_at Millisecond timestamp of Pipeline creation int msec
duration Pipeline duration (microseconds) int μs
finished_at Millisecond timestamp of the end of the Pipeline int msec
message The message attached to the most recent commit of the code that triggered the Pipeline. Same as commit_message string -
pipeline_id Pipeline id string -

gitlab_job

GitLab Job Event metrics

  • Tags
Tag Description
build_commit_sha The commit SHA corresponding to build
build_failure_reason Build failure reason
build_name Build name
build_repo_name Repository name corresponding to build
build_stage Build stage
build_status Build status
object_kind Event type, in this case Job
project_name Project name
sha The commit SHA corresponding to build
user_email User email
  • Metrics
Metric Description Type Unit
build_commit_message The message attached to the most recent commit of the code that triggered the build string -
build_duration Build duration (microseconds) int μs
build_finished_at Millisecond timestamp of the end of build int msec
build_id build id string -
build_started_at Millisecond timestamp of the start of build int msec
message The message attached to the most recent commit of the code that triggered the build. Same as build_commit_message string -
pipeline_id Pipeline id for build string -
project_id Project id for build string -
runner_id Runner id for build string -

Feedback

Is this page helpful? ×