콘텐츠로 이동

GitLab

·


GitLab 운영 데이터를 수집하여 메트릭으로 Guance에 보고합니다.

설정

먼저 GitLab 서비스의 데이터 수집 기능을 활성화하고 허용 목록을 설정합니다. 자세한 절차는 다음 섹션을 참조하세요.

GitLab 설정을 완료한 후 DataKit을 구성합니다. GitLab 버전과 구성에 따라 수집되는 데이터가 다를 수 있습니다.

DataKit 설치 디렉터리의 conf.d/samples 디렉터리로 이동하여 gitlab.conf.sample을 복사한 다음 gitlab.conf로 이름을 변경합니다. 예:

[[inputs.gitlab]]
    ## set true if you need to collect metric from url below
    enable_collect = true

    ## param type: string - default: http://127.0.0.1:80/-/metrics
    prometheus_url = "http://127.0.0.1:80/-/metrics"

    ## param type: string - optional: time units are "ms", "s", "m", "h" - default: 10s
    interval = "10s"

    ## datakit can listen to gitlab ci data at /v1/gitlab when enabled
    enable_ci_visibility = true

    ## Set true to enable election
    election = true

    ## Bearer token file for authentication
    bearer_token_file = ""

    ## HTTP headers
    [inputs.gitlab.http_headers]
    # X-Custom-Header = "custom-value"

    ## TLS configuration
    # tls_ca = "/path/to/ca.pem"
    # tls_cert = "/path/to/cert.pem"
    # tls_key = "/path/to/key.pem"
    # insecure_skip_verify = false

    ## extra tags for gitlab-ci data.
    ## these tags will not overwrite existing tags.
    [inputs.gitlab.ci_extra_tags]
    # some_tag = "some_value"
    # more_tag = "some_other_value"

    ## extra tags for gitlab metrics
    [inputs.gitlab.tags]
    # some_tag = "some_value"
    # more_tag = "some_other_value"

구성을 완료한 후 DataKit을 다시 시작합니다.

ConfigMap을 통해 수집기 구성을 주입하여 수집기를 활성화할 수 있습니다.

GitLab 데이터 수집 기능 활성화

다음과 같이 GitLab에서 Prometheus 데이터 수집 기능을 활성화합니다(영어 UI 기준).

  • 관리자 계정으로 GitLab 인스턴스에 로그인합니다
  • Admin Area > Settings > Metrics and profiling으로 이동합니다
  • Metrics - Prometheus를 선택하고 Enable Prometheus Metrics, save change 순서로 클릭합니다
  • GitLab 서비스를 다시 시작합니다

자세한 내용은 공식 구성 문서를 참조하세요.

데이터 액세스 허용 목록 구성

데이터 수집 기능만 활성화해서는 충분하지 않습니다. GitLab은 데이터 액세스를 엄격하게 관리하므로 액세스 클라이언트의 허용 목록도 구성해야 합니다.

  • GitLab 구성 파일 /etc/gitlab/gitlab.rb를 편집하고 gitlab_rails['monitoring_whitelist'] = ['::1/128']를 찾아 배열에 DataKit 액세스 IP를 추가합니다(일반적으로 DataKit이 실행되는 호스트의 IP이며, GitLab이 컨테이너에서 실행되는 경우 환경에 맞는 IP를 추가하세요)
  • GitLab 서비스를 다시 시작합니다

자세한 내용은 공식 구성 문서를 참조하세요.

GitLab CI 시각화 활성화

GitLab Webhook을 구성하면 GitLab CI를 시각화할 수 있습니다. DataKit에서 GitLab Webhook을 수신하는 기본 URL은 http://DataKit-IP:9529/v1/gitlab입니다.

GitLab 왼쪽 사이드바에서 SettingsWebhooks로 이동합니다. URL을 위 주소로 설정하고 Trigger에서 Job eventsPipeline events를 선택한 다음 Add webhook을 클릭합니다.

메트릭

아래에서 수집되는 모든 데이터에는 기본적으로 전역 선출 tag가 추가됩니다. 구성에서 추가 태그를 지정할 수도 있습니다.

  • 구성의 [inputs.gitlab.tags]를 사용하여 GitLab 메트릭 데이터에 추가 태그를 지정할 수 있습니다.
[inputs.gitlab.tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
# ...
  • 구성의 [inputs.gitlab.ci_extra_tags]를 사용하여 GitLab CI 데이터에 추가 태그를 지정할 수 있습니다.
[inputs.gitlab.ci_extra_tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
# ...

참고: GitLab CI가 정상적으로 작동하도록 GitLab CI 데이터에 지정한 extra tags는 기존 태그를 덮어쓰지 않습니다(아래 GitLab CI 태그 목록 참조).

gitlab

GitLab runtime metrics

Tags & Fields Description
action
(tag)
Action
class
(tag)
Database connection class
controller
(tag)
Controller
feature_category
(tag)
Feature category
host
(tag)
Database host
operation
(tag)
Cache operation type (read/write)
pid
(tag)
Process ID
port
(tag)
Database port
storage
(tag)
Storage
banzai_cacheless_render_real_duration_seconds_count The count of duration of rendering Markdown into HTML when cached output exists
Type: float | (histogram)
Unit: count
banzai_cacheless_render_real_duration_seconds_sum The sum of duration of rendering Markdown into HTML when cached output exists
Type: float | (histogram)
Unit: time,s
cache_misses_total The cache read miss count
Type: float | (count)
Unit: count
cache_operation_duration_seconds_count The count of cache access time
Type: float | (histogram)
Unit: count
cache_operation_duration_seconds_sum The sum of cache access time
Type: float | (histogram)
Unit: time,s
cache_operations_total The count of cache access time
Type: float | (count)
Unit: count
database_connection_pool_busy Number of busy database connections
Type: float | (gauge)
Unit: count
database_connection_pool_connections Total number of database connections
Type: float | (gauge)
Unit: count
database_connection_pool_dead Number of dead database connections
Type: float | (gauge)
Unit: count
database_connection_pool_idle Number of idle database connections
Type: float | (gauge)
Unit: count
database_connection_pool_size Size of database connection pool
Type: float | (gauge)
Unit: count
database_connection_pool_waiting Number of waiting database connections
Type: float | (gauge)
Unit: count
rack_requests_total The rack request count
Type: float | (count)
Unit: count
redis_client_requests_duration_seconds_count The count of redis request latency, excluding blocking commands
Type: float | (histogram)
Unit: count
redis_client_requests_duration_seconds_sum The sum of redis request latency, excluding blocking commands
Type: float | (histogram)
Unit: time,s
redis_client_requests_total Number of Redis client requests
Type: float | (count)
Unit: count
sql_duration_seconds_count The total SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT
Type: float | (histogram)
Unit: count
sql_duration_seconds_sum The sum of SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT
Type: float | (histogram)
Unit: time,s
transaction_cache_read_hit_count_total The counter for cache hits for Rails cache calls
Type: float | (count)
Unit: count
transaction_cache_read_miss_count_total The counter for cache misses for Rails cache calls
Type: float | (count)
Unit: count
transaction_db_cached_count_total The counter for db cache
Type: float | (count)
Unit: count
transaction_db_count_total The counter for db
Type: float | (count)
Unit: count
transaction_duration_seconds_count The count of duration for all transactions (gitlab_transaction_* metrics)
Type: float | (histogram)
Unit: count
transaction_duration_seconds_sum The sum of duration for all transactions (gitlab_transaction_* metrics)
Type: float | (histogram)
Unit: time,s
transaction_new_redis_connections_total The counter for new Redis connections
Type: float | (count)
Unit: count
transaction_rails_queue_duration_total Total queue latency in seconds between GitLab Workhorse forwarding requests and Rails receiving them.
Type: float | (count)
Unit: time,s
transaction_view_duration_total The duration for views
Type: float | (count)
Unit: time,s

gitlab_base

GitLab programming language level metrics

Tags & Fields Description
pid
(tag)
Process ID
thread_name
(tag)
Thread name
uses_db_connection
(tag)
Whether thread uses DB connection (yes/no)
version
(tag)
GitLab version
deployments Number of deployments
Type: float | (gauge)
Unit: count
rails_queue_duration_seconds_count The counter for latency between GitLab Workhorse forwarding a request to Rails
Type: float | (histogram)
Unit: count
rails_queue_duration_seconds_sum The sum for latency between GitLab Workhorse forwarding a request to Rails
Type: float | (histogram)
Unit: time,s
ruby_file_descriptors Number of file descriptors
Type: float | (gauge)
Unit: count
ruby_gc_duration_seconds_count The count of time spent by Ruby in GC
Type: float | (histogram)
Unit: count
ruby_gc_duration_seconds_sum The sum of time spent by Ruby in GC
Type: float | (histogram)
Unit: time,s
ruby_process_cpu_seconds_total Total CPU time used by Ruby process
Type: float | (gauge)
Unit: time,s
ruby_process_max_fds Maximum file descriptors
Type: float | (gauge)
Unit: count
ruby_process_proportional_memory_bytes Proportional memory usage in bytes
Type: float | (gauge)
Unit: digital,B
ruby_process_resident_memory_bytes Resident memory usage in bytes
Type: float | (gauge)
Unit: digital,B
ruby_process_start_time_seconds Process start time in seconds
Type: float | (gauge)
Unit: timeStamp,sec
ruby_process_unique_memory_bytes Unique memory usage in bytes
Type: float | (gauge)
Unit: digital,B
ruby_sampler_duration_seconds_total The time spent collecting stats
Type: float | (count)
Unit: time,s
ruby_threads_max_expected_threads Maximum expected threads per Puma process
Type: float | (gauge)
Unit: count
ruby_threads_running_threads Number of running threads
Type: float | (gauge)
Unit: count

gitlab_http

GitLab HTTP metrics

Tags & Fields Description
feature_category
(tag)
Feature category
method
(tag)
HTTP method
status
(tag)
HTTP status code
http_health_requests_total Number of health requests
Type: float | (count)
Unit: count
http_request_duration_seconds_count The counter for request duration
Type: float | (histogram)
Unit: count
http_request_duration_seconds_sum The sum for request duration
Type: float | (histogram)
Unit: time,s
http_requests_total Total HTTP requests
Type: float | (count)
Unit: count

로그

gitlab_pipeline

GitLab Pipeline event metrics

Tags & Fields Description
author_email
(tag)
Author email
ci_status
(tag)
CI type
commit_sha
(tag)
The commit SHA of the most recent commit of the code that triggered the Pipeline
object_kind
(tag)
Event type, in this case Pipeline
operation_name
(tag)
Operation name
pipeline_name
(tag)
Pipeline name
pipeline_source
(tag)
Sources of Pipeline triggers
pipeline_url
(tag)
Pipeline URL
ref
(tag)
Branches involved
repository_url
(tag)
Repository URL
resource
(tag)
Project name
commit_message Commit message for the most recent commit that triggered the pipeline.
Type: string
Unit: N/A
created_at Millisecond timestamp of Pipeline creation
Type: int
Unit: timeStamp,msec
duration Pipeline duration (microseconds)
Type: int
Unit: time,μs
event_raw The raw JSON body of the webhook event
Type: string
Unit: N/A
finished_at Millisecond timestamp of the end of the Pipeline
Type: int
Unit: timeStamp,msec
message The message attached to the most recent commit of the code that triggered the Pipeline. Same as commit_message
Type: string
Unit: N/A
pipeline_id GitLab pipeline identifier from the webhook event.
Type: string
Unit: N/A
queued_duration Pipeline queued duration
Type: int
Unit: time,ms

gitlab_job

GitLab Job Event metrics

Tags & Fields Description
build_commit_sha
(tag)
The commit SHA corresponding to build
build_failure_reason
(tag)
Build failure reason
build_name
(tag)
Build name
build_repo_name
(tag)
Repository name corresponding to build
build_stage
(tag)
Build stage
build_status
(tag)
Build status
object_kind
(tag)
Event type, in this case Job
project_name
(tag)
Project name
sha
(tag)
The commit SHA corresponding to build
user_email
(tag)
User email
build_commit_message Commit message for the most recent commit that triggered the job build.
Type: string
Unit: N/A
build_duration Build duration (microseconds)
Type: int
Unit: time,μs
build_finished_at Millisecond timestamp of the end of build
Type: int
Unit: timeStamp,msec
build_id GitLab job build identifier from the webhook event.
Type: string
Unit: N/A
build_started_at Millisecond timestamp of the start of build
Type: int
Unit: timeStamp,msec
event_raw The raw JSON body of the webhook event
Type: string
Unit: N/A
message Commit message for the most recent commit that triggered the job build. Same value as build_commit_message.
Type: string
Unit: N/A
pipeline_id GitLab pipeline identifier associated with this job build.
Type: string
Unit: N/A
project_id GitLab project identifier associated with this job build.
Type: string
Unit: N/A
queued_duration Job queued duration
Type: int
Unit: time,ms
runner_id GitLab runner identifier that executed this job build.
Type: string
Unit: N/A

문서 평가

이 페이지가 도움이 되었나요?