GitLab
GitLab の稼働データを収集し、メトリクスとしてGuanceに送信します。
設定¶
まず、GitLab サービスのデータ収集機能を有効にし、許可リストを設定します。詳しい手順は以降のセクションを参照してください。
GitLab の設定が完了したら、DataKit を設定します。GitLab のバージョンや設定によって、収集されるデータが異なる場合があります。
DataKit のインストールディレクトリにある conf.d/samples ディレクトリへ移動し、gitlab.conf.sample をコピーして gitlab.conf に名前を変更します。例:
[[inputs.gitlab]]
## set true if you need to collect metric from url below
enable_collect = true
## param type: string - default: http://127.0.0.1:80/-/metrics
prometheus_url = "http://127.0.0.1:80/-/metrics"
## param type: string - optional: time units are "ms", "s", "m", "h" - default: 10s
interval = "10s"
## datakit can listen to gitlab ci data at /v1/gitlab when enabled
enable_ci_visibility = true
## Set true to enable election
election = true
## Bearer token file for authentication
bearer_token_file = ""
## HTTP headers
[inputs.gitlab.http_headers]
# X-Custom-Header = "custom-value"
## TLS configuration
# tls_ca = "/path/to/ca.pem"
# tls_cert = "/path/to/cert.pem"
# tls_key = "/path/to/key.pem"
# insecure_skip_verify = false
## extra tags for gitlab-ci data.
## these tags will not overwrite existing tags.
[inputs.gitlab.ci_extra_tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
## extra tags for gitlab metrics
[inputs.gitlab.tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
設定が完了したら、DataKit を再起動します。
ConfigMap を使用してコレクター設定を注入することで、コレクターを有効にできます。
GitLab のデータ収集機能を有効にする¶
GitLab で Prometheus データ収集機能を有効にします(英語 UI の例):
- 管理者アカウントで GitLab インスタンスにログインします
Admin Area>Settings>Metrics and profilingに移動しますMetrics - Prometheusを選択し、Enable Prometheus Metrics、save changeの順にクリックします- GitLab サービスを再起動します
詳細については、公式設定ドキュメントを参照してください。
データアクセス許可リストを設定する¶
データ収集機能を有効にするだけでは不十分です。GitLab ではデータアクセスが厳密に管理されるため、アクセス元の許可リストも設定します。
- GitLab 設定ファイル
/etc/gitlab/gitlab.rbを編集し、gitlab_rails['monitoring_whitelist'] = ['::1/128']を見つけて、配列に DataKit のアクセス IP を追加します(通常は DataKit を実行するホストの IP。GitLab がコンテナで稼働している場合は、環境に応じた IP を追加してください) - GitLab サービスを再起動します
詳細については、公式設定ドキュメントを参照してください。
GitLab CI の可視化を有効にする¶
GitLab Webhook を設定すると、GitLab CI を可視化できます。DataKit が GitLab Webhook を受信するデフォルト URL は http://DataKit-IP:9529/v1/gitlab です。
GitLab の左サイドバーで Settings ➔ Webhooks に移動します。URL に上記のアドレスを設定し、Trigger で Job events と Pipeline events を選択して、Add webhook をクリックします。
メトリクス¶
以下で収集されるすべてのデータには、デフォルトでグローバル選出 tag が追加されます。設定で追加のタグを指定することもできます。
- 設定の
[inputs.gitlab.tags]で GitLab メトリクスデータの追加タグを指定できます。
- 設定の
[inputs.gitlab.ci_extra_tags]で GitLab CI データの追加タグを指定できます。
注:GitLab CI を正常に動作させるため、GitLab CI データに指定した extra tags は既存のタグを上書きしません(GitLab CI のタグ一覧は以下を参照)。
gitlab¶
GitLab runtime metrics
| Tags & Fields | Description |
|---|---|
| action ( tag) |
Action |
| class ( tag) |
Database connection class |
| controller ( tag) |
Controller |
| feature_category ( tag) |
Feature category |
| host ( tag) |
Database host |
| operation ( tag) |
Cache operation type (read/write) |
| pid ( tag) |
Process ID |
| port ( tag) |
Database port |
| storage ( tag) |
Storage |
| banzai_cacheless_render_real_duration_seconds_count | The count of duration of rendering Markdown into HTML when cached output exists Type: float | (histogram) Unit: count |
| banzai_cacheless_render_real_duration_seconds_sum | The sum of duration of rendering Markdown into HTML when cached output exists Type: float | (histogram) Unit: time,s |
| cache_misses_total | The cache read miss count Type: float | (count) Unit: count |
| cache_operation_duration_seconds_count | The count of cache access time Type: float | (histogram) Unit: count |
| cache_operation_duration_seconds_sum | The sum of cache access time Type: float | (histogram) Unit: time,s |
| cache_operations_total | The count of cache access time Type: float | (count) Unit: count |
| database_connection_pool_busy | Number of busy database connections Type: float | (gauge) Unit: count |
| database_connection_pool_connections | Total number of database connections Type: float | (gauge) Unit: count |
| database_connection_pool_dead | Number of dead database connections Type: float | (gauge) Unit: count |
| database_connection_pool_idle | Number of idle database connections Type: float | (gauge) Unit: count |
| database_connection_pool_size | Size of database connection pool Type: float | (gauge) Unit: count |
| database_connection_pool_waiting | Number of waiting database connections Type: float | (gauge) Unit: count |
| rack_requests_total | The rack request count Type: float | (count) Unit: count |
| redis_client_requests_duration_seconds_count | The count of redis request latency, excluding blocking commands Type: float | (histogram) Unit: count |
| redis_client_requests_duration_seconds_sum | The sum of redis request latency, excluding blocking commands Type: float | (histogram) Unit: time,s |
| redis_client_requests_total | Number of Redis client requests Type: float | (count) Unit: count |
| sql_duration_seconds_count | The total SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT Type: float | (histogram) Unit: count |
| sql_duration_seconds_sum | The sum of SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT Type: float | (histogram) Unit: time,s |
| transaction_cache_read_hit_count_total | The counter for cache hits for Rails cache calls Type: float | (count) Unit: count |
| transaction_cache_read_miss_count_total | The counter for cache misses for Rails cache calls Type: float | (count) Unit: count |
| transaction_db_cached_count_total | The counter for db cache Type: float | (count) Unit: count |
| transaction_db_count_total | The counter for db Type: float | (count) Unit: count |
| transaction_duration_seconds_count | The count of duration for all transactions (gitlab_transaction_* metrics) Type: float | (histogram) Unit: count |
| transaction_duration_seconds_sum | The sum of duration for all transactions (gitlab_transaction_* metrics) Type: float | (histogram) Unit: time,s |
| transaction_new_redis_connections_total | The counter for new Redis connections Type: float | (count) Unit: count |
| transaction_rails_queue_duration_total | Total queue latency in seconds between GitLab Workhorse forwarding requests and Rails receiving them. Type: float | (count) Unit: time,s |
| transaction_view_duration_total | The duration for views Type: float | (count) Unit: time,s |
gitlab_base¶
GitLab programming language level metrics
| Tags & Fields | Description |
|---|---|
| pid ( tag) |
Process ID |
| thread_name ( tag) |
Thread name |
| uses_db_connection ( tag) |
Whether thread uses DB connection (yes/no) |
| version ( tag) |
GitLab version |
| deployments | Number of deployments Type: float | (gauge) Unit: count |
| rails_queue_duration_seconds_count | The counter for latency between GitLab Workhorse forwarding a request to Rails Type: float | (histogram) Unit: count |
| rails_queue_duration_seconds_sum | The sum for latency between GitLab Workhorse forwarding a request to Rails Type: float | (histogram) Unit: time,s |
| ruby_file_descriptors | Number of file descriptors Type: float | (gauge) Unit: count |
| ruby_gc_duration_seconds_count | The count of time spent by Ruby in GC Type: float | (histogram) Unit: count |
| ruby_gc_duration_seconds_sum | The sum of time spent by Ruby in GC Type: float | (histogram) Unit: time,s |
| ruby_process_cpu_seconds_total | Total CPU time used by Ruby process Type: float | (gauge) Unit: time,s |
| ruby_process_max_fds | Maximum file descriptors Type: float | (gauge) Unit: count |
| ruby_process_proportional_memory_bytes | Proportional memory usage in bytes Type: float | (gauge) Unit: digital,B |
| ruby_process_resident_memory_bytes | Resident memory usage in bytes Type: float | (gauge) Unit: digital,B |
| ruby_process_start_time_seconds | Process start time in seconds Type: float | (gauge) Unit: timeStamp,sec |
| ruby_process_unique_memory_bytes | Unique memory usage in bytes Type: float | (gauge) Unit: digital,B |
| ruby_sampler_duration_seconds_total | The time spent collecting stats Type: float | (count) Unit: time,s |
| ruby_threads_max_expected_threads | Maximum expected threads per Puma process Type: float | (gauge) Unit: count |
| ruby_threads_running_threads | Number of running threads Type: float | (gauge) Unit: count |
gitlab_http¶
GitLab HTTP metrics
| Tags & Fields | Description |
|---|---|
| feature_category ( tag) |
Feature category |
| method ( tag) |
HTTP method |
| status ( tag) |
HTTP status code |
| http_health_requests_total | Number of health requests Type: float | (count) Unit: count |
| http_request_duration_seconds_count | The counter for request duration Type: float | (histogram) Unit: count |
| http_request_duration_seconds_sum | The sum for request duration Type: float | (histogram) Unit: time,s |
| http_requests_total | Total HTTP requests Type: float | (count) Unit: count |
ログ¶
gitlab_pipeline¶
GitLab Pipeline event metrics
| Tags & Fields | Description |
|---|---|
| author_email ( tag) |
Author email |
| ci_status ( tag) |
CI type |
| commit_sha ( tag) |
The commit SHA of the most recent commit of the code that triggered the Pipeline |
| object_kind ( tag) |
Event type, in this case Pipeline |
| operation_name ( tag) |
Operation name |
| pipeline_name ( tag) |
Pipeline name |
| pipeline_source ( tag) |
Sources of Pipeline triggers |
| pipeline_url ( tag) |
Pipeline URL |
| ref ( tag) |
Branches involved |
| repository_url ( tag) |
Repository URL |
| resource ( tag) |
Project name |
| commit_message | Commit message for the most recent commit that triggered the pipeline. Type: string Unit: N/A |
| created_at | Millisecond timestamp of Pipeline creation Type: int Unit: timeStamp,msec |
| duration | Pipeline duration (microseconds) Type: int Unit: time,μs |
| event_raw | The raw JSON body of the webhook event Type: string Unit: N/A |
| finished_at | Millisecond timestamp of the end of the Pipeline Type: int Unit: timeStamp,msec |
| message | The message attached to the most recent commit of the code that triggered the Pipeline. Same as commit_message Type: string Unit: N/A |
| pipeline_id | GitLab pipeline identifier from the webhook event. Type: string Unit: N/A |
| queued_duration | Pipeline queued duration Type: int Unit: time,ms |
gitlab_job¶
GitLab Job Event metrics
| Tags & Fields | Description |
|---|---|
| build_commit_sha ( tag) |
The commit SHA corresponding to build |
| build_failure_reason ( tag) |
Build failure reason |
| build_name ( tag) |
Build name |
| build_repo_name ( tag) |
Repository name corresponding to build |
| build_stage ( tag) |
Build stage |
| build_status ( tag) |
Build status |
| object_kind ( tag) |
Event type, in this case Job |
| project_name ( tag) |
Project name |
| sha ( tag) |
The commit SHA corresponding to build |
| user_email ( tag) |
User email |
| build_commit_message | Commit message for the most recent commit that triggered the job build. Type: string Unit: N/A |
| build_duration | Build duration (microseconds) Type: int Unit: time,μs |
| build_finished_at | Millisecond timestamp of the end of build Type: int Unit: timeStamp,msec |
| build_id | GitLab job build identifier from the webhook event. Type: string Unit: N/A |
| build_started_at | Millisecond timestamp of the start of build Type: int Unit: timeStamp,msec |
| event_raw | The raw JSON body of the webhook event Type: string Unit: N/A |
| message | Commit message for the most recent commit that triggered the job build. Same value as build_commit_message. Type: string Unit: N/A |
| pipeline_id | GitLab pipeline identifier associated with this job build. Type: string Unit: N/A |
| project_id | GitLab project identifier associated with this job build. Type: string Unit: N/A |
| queued_duration | Job queued duration Type: int Unit: time,ms |
| runner_id | GitLab runner identifier that executed this job build. Type: string Unit: N/A |