GitLab
GitLab 운영 데이터를 수집하여 메트릭으로 Guance에 보고합니다.
설정¶
먼저 GitLab 서비스의 데이터 수집 기능을 활성화하고 허용 목록을 설정합니다. 자세한 절차는 다음 섹션을 참조하세요.
GitLab 설정을 완료한 후 DataKit을 구성합니다. GitLab 버전과 구성에 따라 수집되는 데이터가 다를 수 있습니다.
DataKit 설치 디렉터리의 conf.d/samples 디렉터리로 이동하여 gitlab.conf.sample을 복사한 다음 gitlab.conf로 이름을 변경합니다. 예:
[[inputs.gitlab]]
## set true if you need to collect metric from url below
enable_collect = true
## param type: string - default: http://127.0.0.1:80/-/metrics
prometheus_url = "http://127.0.0.1:80/-/metrics"
## param type: string - optional: time units are "ms", "s", "m", "h" - default: 10s
interval = "10s"
## datakit can listen to gitlab ci data at /v1/gitlab when enabled
enable_ci_visibility = true
## Set true to enable election
election = true
## Bearer token file for authentication
bearer_token_file = ""
## HTTP headers
[inputs.gitlab.http_headers]
# X-Custom-Header = "custom-value"
## TLS configuration
# tls_ca = "/path/to/ca.pem"
# tls_cert = "/path/to/cert.pem"
# tls_key = "/path/to/key.pem"
# insecure_skip_verify = false
## extra tags for gitlab-ci data.
## these tags will not overwrite existing tags.
[inputs.gitlab.ci_extra_tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
## extra tags for gitlab metrics
[inputs.gitlab.tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
구성을 완료한 후 DataKit을 다시 시작합니다.
ConfigMap을 통해 수집기 구성을 주입하여 수집기를 활성화할 수 있습니다.
GitLab 데이터 수집 기능 활성화¶
다음과 같이 GitLab에서 Prometheus 데이터 수집 기능을 활성화합니다(영어 UI 기준).
- 관리자 계정으로 GitLab 인스턴스에 로그인합니다
Admin Area>Settings>Metrics and profiling으로 이동합니다Metrics - Prometheus를 선택하고Enable Prometheus Metrics,save change순서로 클릭합니다- GitLab 서비스를 다시 시작합니다
자세한 내용은 공식 구성 문서를 참조하세요.
데이터 액세스 허용 목록 구성¶
데이터 수집 기능만 활성화해서는 충분하지 않습니다. GitLab은 데이터 액세스를 엄격하게 관리하므로 액세스 클라이언트의 허용 목록도 구성해야 합니다.
- GitLab 구성 파일
/etc/gitlab/gitlab.rb를 편집하고gitlab_rails['monitoring_whitelist'] = ['::1/128']를 찾아 배열에 DataKit 액세스 IP를 추가합니다(일반적으로 DataKit이 실행되는 호스트의 IP이며, GitLab이 컨테이너에서 실행되는 경우 환경에 맞는 IP를 추가하세요) - GitLab 서비스를 다시 시작합니다
자세한 내용은 공식 구성 문서를 참조하세요.
GitLab CI 시각화 활성화¶
GitLab Webhook을 구성하면 GitLab CI를 시각화할 수 있습니다. DataKit에서 GitLab Webhook을 수신하는 기본 URL은 http://DataKit-IP:9529/v1/gitlab입니다.
GitLab 왼쪽 사이드바에서 Settings ➔ Webhooks로 이동합니다. URL을 위 주소로 설정하고 Trigger에서 Job events와 Pipeline events를 선택한 다음 Add webhook을 클릭합니다.
메트릭¶
아래에서 수집되는 모든 데이터에는 기본적으로 전역 선출 tag가 추가됩니다. 구성에서 추가 태그를 지정할 수도 있습니다.
- 구성의
[inputs.gitlab.tags]를 사용하여 GitLab 메트릭 데이터에 추가 태그를 지정할 수 있습니다.
- 구성의
[inputs.gitlab.ci_extra_tags]를 사용하여 GitLab CI 데이터에 추가 태그를 지정할 수 있습니다.
참고: GitLab CI가 정상적으로 작동하도록 GitLab CI 데이터에 지정한 extra tags는 기존 태그를 덮어쓰지 않습니다(아래 GitLab CI 태그 목록 참조).
gitlab¶
GitLab runtime metrics
| Tags & Fields | Description |
|---|---|
| action ( tag) |
Action |
| class ( tag) |
Database connection class |
| controller ( tag) |
Controller |
| feature_category ( tag) |
Feature category |
| host ( tag) |
Database host |
| operation ( tag) |
Cache operation type (read/write) |
| pid ( tag) |
Process ID |
| port ( tag) |
Database port |
| storage ( tag) |
Storage |
| banzai_cacheless_render_real_duration_seconds_count | The count of duration of rendering Markdown into HTML when cached output exists Type: float | (histogram) Unit: count |
| banzai_cacheless_render_real_duration_seconds_sum | The sum of duration of rendering Markdown into HTML when cached output exists Type: float | (histogram) Unit: time,s |
| cache_misses_total | The cache read miss count Type: float | (count) Unit: count |
| cache_operation_duration_seconds_count | The count of cache access time Type: float | (histogram) Unit: count |
| cache_operation_duration_seconds_sum | The sum of cache access time Type: float | (histogram) Unit: time,s |
| cache_operations_total | The count of cache access time Type: float | (count) Unit: count |
| database_connection_pool_busy | Number of busy database connections Type: float | (gauge) Unit: count |
| database_connection_pool_connections | Total number of database connections Type: float | (gauge) Unit: count |
| database_connection_pool_dead | Number of dead database connections Type: float | (gauge) Unit: count |
| database_connection_pool_idle | Number of idle database connections Type: float | (gauge) Unit: count |
| database_connection_pool_size | Size of database connection pool Type: float | (gauge) Unit: count |
| database_connection_pool_waiting | Number of waiting database connections Type: float | (gauge) Unit: count |
| rack_requests_total | The rack request count Type: float | (count) Unit: count |
| redis_client_requests_duration_seconds_count | The count of redis request latency, excluding blocking commands Type: float | (histogram) Unit: count |
| redis_client_requests_duration_seconds_sum | The sum of redis request latency, excluding blocking commands Type: float | (histogram) Unit: time,s |
| redis_client_requests_total | Number of Redis client requests Type: float | (count) Unit: count |
| sql_duration_seconds_count | The total SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT Type: float | (histogram) Unit: count |
| sql_duration_seconds_sum | The sum of SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT Type: float | (histogram) Unit: time,s |
| transaction_cache_read_hit_count_total | The counter for cache hits for Rails cache calls Type: float | (count) Unit: count |
| transaction_cache_read_miss_count_total | The counter for cache misses for Rails cache calls Type: float | (count) Unit: count |
| transaction_db_cached_count_total | The counter for db cache Type: float | (count) Unit: count |
| transaction_db_count_total | The counter for db Type: float | (count) Unit: count |
| transaction_duration_seconds_count | The count of duration for all transactions (gitlab_transaction_* metrics) Type: float | (histogram) Unit: count |
| transaction_duration_seconds_sum | The sum of duration for all transactions (gitlab_transaction_* metrics) Type: float | (histogram) Unit: time,s |
| transaction_new_redis_connections_total | The counter for new Redis connections Type: float | (count) Unit: count |
| transaction_rails_queue_duration_total | Total queue latency in seconds between GitLab Workhorse forwarding requests and Rails receiving them. Type: float | (count) Unit: time,s |
| transaction_view_duration_total | The duration for views Type: float | (count) Unit: time,s |
gitlab_base¶
GitLab programming language level metrics
| Tags & Fields | Description |
|---|---|
| pid ( tag) |
Process ID |
| thread_name ( tag) |
Thread name |
| uses_db_connection ( tag) |
Whether thread uses DB connection (yes/no) |
| version ( tag) |
GitLab version |
| deployments | Number of deployments Type: float | (gauge) Unit: count |
| rails_queue_duration_seconds_count | The counter for latency between GitLab Workhorse forwarding a request to Rails Type: float | (histogram) Unit: count |
| rails_queue_duration_seconds_sum | The sum for latency between GitLab Workhorse forwarding a request to Rails Type: float | (histogram) Unit: time,s |
| ruby_file_descriptors | Number of file descriptors Type: float | (gauge) Unit: count |
| ruby_gc_duration_seconds_count | The count of time spent by Ruby in GC Type: float | (histogram) Unit: count |
| ruby_gc_duration_seconds_sum | The sum of time spent by Ruby in GC Type: float | (histogram) Unit: time,s |
| ruby_process_cpu_seconds_total | Total CPU time used by Ruby process Type: float | (gauge) Unit: time,s |
| ruby_process_max_fds | Maximum file descriptors Type: float | (gauge) Unit: count |
| ruby_process_proportional_memory_bytes | Proportional memory usage in bytes Type: float | (gauge) Unit: digital,B |
| ruby_process_resident_memory_bytes | Resident memory usage in bytes Type: float | (gauge) Unit: digital,B |
| ruby_process_start_time_seconds | Process start time in seconds Type: float | (gauge) Unit: timeStamp,sec |
| ruby_process_unique_memory_bytes | Unique memory usage in bytes Type: float | (gauge) Unit: digital,B |
| ruby_sampler_duration_seconds_total | The time spent collecting stats Type: float | (count) Unit: time,s |
| ruby_threads_max_expected_threads | Maximum expected threads per Puma process Type: float | (gauge) Unit: count |
| ruby_threads_running_threads | Number of running threads Type: float | (gauge) Unit: count |
gitlab_http¶
GitLab HTTP metrics
| Tags & Fields | Description |
|---|---|
| feature_category ( tag) |
Feature category |
| method ( tag) |
HTTP method |
| status ( tag) |
HTTP status code |
| http_health_requests_total | Number of health requests Type: float | (count) Unit: count |
| http_request_duration_seconds_count | The counter for request duration Type: float | (histogram) Unit: count |
| http_request_duration_seconds_sum | The sum for request duration Type: float | (histogram) Unit: time,s |
| http_requests_total | Total HTTP requests Type: float | (count) Unit: count |
로그¶
gitlab_pipeline¶
GitLab Pipeline event metrics
| Tags & Fields | Description |
|---|---|
| author_email ( tag) |
Author email |
| ci_status ( tag) |
CI type |
| commit_sha ( tag) |
The commit SHA of the most recent commit of the code that triggered the Pipeline |
| object_kind ( tag) |
Event type, in this case Pipeline |
| operation_name ( tag) |
Operation name |
| pipeline_name ( tag) |
Pipeline name |
| pipeline_source ( tag) |
Sources of Pipeline triggers |
| pipeline_url ( tag) |
Pipeline URL |
| ref ( tag) |
Branches involved |
| repository_url ( tag) |
Repository URL |
| resource ( tag) |
Project name |
| commit_message | Commit message for the most recent commit that triggered the pipeline. Type: string Unit: N/A |
| created_at | Millisecond timestamp of Pipeline creation Type: int Unit: timeStamp,msec |
| duration | Pipeline duration (microseconds) Type: int Unit: time,μs |
| event_raw | The raw JSON body of the webhook event Type: string Unit: N/A |
| finished_at | Millisecond timestamp of the end of the Pipeline Type: int Unit: timeStamp,msec |
| message | The message attached to the most recent commit of the code that triggered the Pipeline. Same as commit_message Type: string Unit: N/A |
| pipeline_id | GitLab pipeline identifier from the webhook event. Type: string Unit: N/A |
| queued_duration | Pipeline queued duration Type: int Unit: time,ms |
gitlab_job¶
GitLab Job Event metrics
| Tags & Fields | Description |
|---|---|
| build_commit_sha ( tag) |
The commit SHA corresponding to build |
| build_failure_reason ( tag) |
Build failure reason |
| build_name ( tag) |
Build name |
| build_repo_name ( tag) |
Repository name corresponding to build |
| build_stage ( tag) |
Build stage |
| build_status ( tag) |
Build status |
| object_kind ( tag) |
Event type, in this case Job |
| project_name ( tag) |
Project name |
| sha ( tag) |
The commit SHA corresponding to build |
| user_email ( tag) |
User email |
| build_commit_message | Commit message for the most recent commit that triggered the job build. Type: string Unit: N/A |
| build_duration | Build duration (microseconds) Type: int Unit: time,μs |
| build_finished_at | Millisecond timestamp of the end of build Type: int Unit: timeStamp,msec |
| build_id | GitLab job build identifier from the webhook event. Type: string Unit: N/A |
| build_started_at | Millisecond timestamp of the start of build Type: int Unit: timeStamp,msec |
| event_raw | The raw JSON body of the webhook event Type: string Unit: N/A |
| message | Commit message for the most recent commit that triggered the job build. Same value as build_commit_message. Type: string Unit: N/A |
| pipeline_id | GitLab pipeline identifier associated with this job build. Type: string Unit: N/A |
| project_id | GitLab project identifier associated with this job build. Type: string Unit: N/A |
| queued_duration | Job queued duration Type: int Unit: time,ms |
| runner_id | GitLab runner identifier that executed this job build. Type: string Unit: N/A |