跳转至

GitLab


·


采集 GitLab 运行数据并以指标的方式上报到观测云。

配置

首先需要打开 GitLab 服务的数据采集功能和设置白名单,具体操作见后续分段。

GitLab 设置完成后,对 DataKit 进行配置。注意,根据 GitLab 版本和配置不同,采集到的数据可能存在差异。

进入 DataKit 安装目录下的 conf.d/gitlab 目录,复制 gitlab.conf.sample 并命名为 gitlab.conf。示例如下:

[[inputs.gitlab]]
    ## set true if you need to collect metric from url below
    enable_collect = true

    ## param type: string - default: http://127.0.0.1:80/-/metrics
    prometheus_url = "http://127.0.0.1:80/-/metrics"

    ## param type: string - optional: time units are "ms", "s", "m", "h" - default: 10s
    interval = "10s"

    ## datakit can listen to gitlab ci data at /v1/gitlab when enabled
    enable_ci_visibility = true

    ## Set true to enable election
    election = true

    ## extra tags for gitlab-ci data.
    ## these tags will not overwrite existing tags.
    [inputs.gitlab.ci_extra_tags]
    # some_tag = "some_value"
    # more_tag = "some_other_value"

    ## extra tags for gitlab metrics
    [inputs.gitlab.tags]
    # some_tag = "some_value"
    # more_tag = "some_other_value"

配置好后,重启 DataKit 即可。

目前可以通过 ConfigMap 方式注入采集器配置来开启采集器。

GitLab 开启数据采集功能

GitLab 需要开启 Prometheus 数据采集功能,开启方式如下(以英文页面为例):

  • 以管理员账号登陆己方 GitLab 页面
  • 转到 Admin Area > Settings > Metrics and profiling
  • 选择 Metrics - Prometheus,点击 Enable Prometheus Metrics 并且 save change
  • 重启 GitLab 服务

详情见官方配置文档

配置数据访问端白名单

只开启数据采集功能还不够,GitLab 对于数据管理十分严格,需要再配置访问端的白名单。开启方式如下:

  • 修改 GitLab 配置文件 /etc/gitlab/gitlab.rb,找到 gitlab_rails['monitoring_whitelist'] = ['::1/128'] 并在该数组中添加 DataKit 的访问 IP(通常情况为 DataKit 所在主机的 IP,如果 GitLab 运行在容器中需根据实际情况添加)
  • 重启 GitLab 服务

详情见官方配置文档

开启 GitLab CI 可视化

确保当前 Datakit 版本(1.2.13 及以后)支持 GitLab CI 可视化功能。

通过配置 GitLab Webhook,可以实现 GitLab CI 可视化。开启步骤如下:

  • 在 GitLab 转到 Settings -> Webhooks 中,将 URL 配置为 http://Datakit_IP:PORT/v1/gitlab,Trigger 配置 Job events 和 Pipeline events 两项,点击 Add webhook 确认添加;
  • 可点击 Test 按钮测试 Webhook 配置是否正确,Datakit 接收到 Webhook 后应返回状态码 200。正确配置后,Datakit 可以顺利采集到 GitLab 的 CI 信息。

Datakit 接收到 Webhook Event 后,是将数据作为 logging 打到数据中心的。

注意:如果将 GitLab 数据打到本地网络的 Datakit,需要对 GitLab 进行额外的配置,见 allow requests to the local network

另外:GitLab CI 功能不参与采集器选举,用户只需将 GitLab Webhook 的 URL 配置为其中一个 Datakit 的 URL 即可;若只需要 GitLab CI 可视化功能而不需要 GitLab 指标采集,可通过配置 enable_collect = false 关闭指标采集功能。

指标

以下所有数据采集,默认会追加名为 host 的全局 tag(tag 值为 DataKit 所在主机名)。

可以在配置中通过 [inputs.gitlab.tags]GitLab 指标数据指定其它标签:

 [inputs.gitlab.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

可以在配置中通过 [inputs.gitlab.ci_extra_tags]GitLab CI 数据指定其它标签:

 [inputs.gitlab.ci_extra_tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

注意:为了确保 GitLab CI 功能正常,为 GitLab CI 数据指定的 extra tags 不会覆盖其数据中已有的标签(GitLab CI 标签列表见下)。

gitlab

GitLab runtime metrics

  • 标签
Tag Description
action Action
controller Controller
feature_category Feature category
storage Storage
  • 指标列表
Metric Description Type Unit
banzai_cacheless_render_real_duration_seconds_count The count of duration of rendering Markdown into HTML when cached output exists float s
banzai_cacheless_render_real_duration_seconds_sum The sum of duration of rendering Markdown into HTML when cached output exists float s
cache_misses_total The cache read miss count float -
cache_operation_duration_seconds_count The count of cache access time float s
cache_operation_duration_seconds_sum The count of cache access time float s
cache_operations_total The count of cache access time float -
rack_requests_total The rack request count float -
redis_client_requests_duration_seconds_count The count of redis request latency, excluding blocking commands float s
redis_client_requests_duration_seconds_sum The sum of redis request latency, excluding blocking commands float s
redis_client_requests_total Number of Redis client requests float -
sql_duration_seconds_count The total SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT float s
sql_duration_seconds_sum The sum of SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT float s
transaction_cache_read_hit_count_total The counter for cache hits for Rails cache calls float count
transaction_cache_read_miss_count_total The counter for cache misses for Rails cache calls float count
transaction_db_cached_count_total The counter for db cache float count
transaction_db_count_total The counter for db float count
transaction_duration_seconds_count The count of duration for all transactions (gitlab_transaction_* metrics) float s
transaction_duration_seconds_sum The sum of duration for all transactions (gitlab_transaction_* metrics) float s
transaction_new_redis_connections_total The counter for new Redis connections float -
transaction_view_duration_total The duration for views float -

gitlab_base

GitLab programming language level metrics

  • 标签

NA

  • 指标列表
Metric Description Type Unit
rails_queue_duration_seconds_count The counter for latency between GitLab Workhorse forwarding a request to Rails float s
rails_queue_duration_seconds_sum The sum for latency between GitLab Workhorse forwarding a request to Rails float s
ruby_gc_duration_seconds_count The count of time spent by Ruby in GC float s
ruby_gc_duration_seconds_sum The sum of time spent by Ruby in GC float s
ruby_sampler_duration_seconds_total The time spent collecting stats float s

gitlab_http

GitLab HTTP metrics

  • 标签
Tag Description
method 方法
status 状态码
  • 指标列表
Metric Description Type Unit
http_health_requests_total Number of health requests float -
http_request_duration_seconds_count The counter for request duration float s
http_request_duration_seconds_sum The sum for request duration float s

gitlab_pipeline

GitLab Pipeline event metrics

  • 标签
Tag Description
author_email Author email
ci_status CI type
commit_sha The commit SHA of the most recent commit of the code that triggered the Pipeline
object_kind Event type, in this case Pipeline
operation_name Operation name
pipeline_name Pipeline name
pipeline_source Sources of Pipeline triggers
pipeline_url Pipeline URL
ref Branches involved
repository_url Repository URL
resource Project name
  • 指标列表
Metric Description Type Unit
commit_message The message attached to the most recent commit of the code that triggered the Pipeline. string -
created_at Millisecond timestamp of Pipeline creation int msec
duration Pipeline duration (microseconds) int μs
finished_at Millisecond timestamp of the end of the Pipeline int msec
message The message attached to the most recent commit of the code that triggered the Pipeline. Same as commit_message string -
pipeline_id Pipeline id string -

gitlab_job

GitLab Job Event metrics

  • 标签
Tag Description
build_commit_sha The commit SHA corresponding to build
build_failure_reason Build failure reason
build_name Build name
build_repo_name Repository name corresponding to build
build_stage Build stage
build_status Build status
object_kind Event type, in this case Job
project_name Project name
sha The commit SHA corresponding to build
user_email User email
  • 指标列表
Metric Description Type Unit
build_commit_message The message attached to the most recent commit of the code that triggered the build string -
build_duration Build duration (microseconds) int μs
build_finished_at Millisecond timestamp of the end of build int msec
build_id build id string -
build_started_at Millisecond timestamp of the start of build int msec
message The message attached to the most recent commit of the code that triggered the build. Same as build_commit_message string -
pipeline_id Pipeline id for build string -
project_id Project id for build string -
runner_id Runner id for build string -

文档评价

文档内容是否对您有帮助? ×