Solr
Solr collector, which collects statistics of Solr Cache, Request Times, and so on.
Configuration¶
Preconditions¶
DataKit uses the Solr Metrics API to collect metrics data and supports Solr 7.0 and above. Available for Solr 6.6, but the indicator data is incomplete.
Already tested version:
- 8.11.2
- 7.0.0
Collector Configuration¶
Go to the conf.d/db directory under the DataKit installation directory, copy solr.conf.sample and name it solr.conf. Examples are as follows:
[[inputs.solr]]
##(optional) collect interval, default is 10 seconds
interval = '10s'
## specify a list of one or more Solr servers
servers = ["http://localhost:8983"]
## Optional HTTP Basic Auth Credentials
# username = "username"
# password = "pa$$word"
## Set true to enable election
election = true
# [inputs.solr.log]
# files = []
# #grok pipeline script path
# pipeline = "solr.p"
[inputs.solr.tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
After configuration, restart DataKit.
The collector can now be turned on by ConfigMap Injection Collector Configuration.
To collect Solr's log, open files in Solr.conf and write to the absolute path of the Solr log file. For example:
Metric¶
For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.solr.tags] if needed:
solr_cache¶
| Tags & Fields | Description |
|---|---|
| category ( tag) |
Solr metric category that owns the metric. |
| core ( tag) |
Solr core that emitted the metric. |
| group ( tag) |
Solr metric group that owns the metric. |
| host ( tag) |
Hostname of the Solr node. |
| instance ( tag) |
Instance name, generated based on server address. |
| name ( tag) |
Name of the Solr cache. |
| cumulative_evictions | Number of cache evictions across all caches since this node has been running. Type: int | (count) Unit: count Tagged by: category, core, group, host, instance, name |
| cumulative_hitratio | Ratio of cache hits to lookups across all the caches since this node has been running. Type: float | (gauge) Unit: percent,percent Tagged by: category, core, group, host, instance, name |
| cumulative_hits | Number of cache hits across all the caches since this node has been running. Type: int | (count) Unit: count Tagged by: category, core, group, host, instance, name |
| cumulative_inserts | Number of cache insertions across all the caches since this node has been running. Type: int | (count) Unit: count Tagged by: category, core, group, host, instance, name |
| cumulative_lookups | Number of cache lookups across all the caches since this node has been running. Type: int | (count) Unit: count Tagged by: category, core, group, host, instance, name |
| evictions | Number of cache evictions for the current index searcher. Type: int | (count) Unit: count Tagged by: category, core, group, host, instance, name |
| hitratio | Ratio of cache hits to lookups for the current index searcher. Type: float | (gauge) Unit: percent,percent Tagged by: category, core, group, host, instance, name |
| hits | Number of hits for the current index searcher. Type: int | (count) Unit: count Tagged by: category, core, group, host, instance, name |
| inserts | Number of inserts into the cache. Type: int | (count) Unit: count Tagged by: category, core, group, host, instance, name |
| lookups | Number of lookups against the cache. Type: int | (count) Unit: count Tagged by: category, core, group, host, instance, name |
| max_ram | Maximum heap that should be used by the cache beyond which keys will be evicted. Type: int | (gauge) Unit: digital,MB Tagged by: category, core, group, host, instance, name |
| ram_bytes_used | Actual heap usage of the cache at that particular instance. Type: int | (gauge) Unit: digital,B Tagged by: category, core, group, host, instance, name |
| size | Number of entries in the cache at that particular instance. Type: int | (gauge) Unit: count Tagged by: category, core, group, host, instance, name |
| warmup | Warm-up time for the registered index searcher. This time is taken in account for the "auto-warming" of caches. Type: int | (gauge) Unit: time,ms Tagged by: category, core, group, host, instance, name |
solr_request_times¶
| Tags & Fields | Description |
|---|---|
| category ( tag) |
Solr metric category that owns the metric. |
| core ( tag) |
Solr core that emitted the metric. |
| group ( tag) |
Solr metric group that owns the metric. |
| handler ( tag) |
Solr request handler path or name. |
| host ( tag) |
Hostname of the Solr node. |
| instance ( tag) |
Instance name, generated based on server address. |
| count | Total number of requests made since the Solr process was started. Type: int | (count) Unit: count Tagged by: category, core, group, handler, host, instance |
| max | Maximum request processing time. Type: float | (gauge) Unit: time,ms Tagged by: category, core, group, handler, host, instance |
| mean | Mean request processing time. Type: float | (gauge) Unit: time,ms Tagged by: category, core, group, handler, host, instance |
| median | Median request processing time. Type: float | (gauge) Unit: time,ms Tagged by: category, core, group, handler, host, instance |
| min | Minimum request processing time. Type: float | (gauge) Unit: time,ms Tagged by: category, core, group, handler, host, instance |
| p75 | 75th percentile request processing time. Type: float | (gauge) Unit: time,ms Tagged by: category, core, group, handler, host, instance |
| p95 | 95th percentile request processing time. Type: float | (gauge) Unit: time,ms Tagged by: category, core, group, handler, host, instance |
| p99 | 99th percentile request processing time. Type: float | (gauge) Unit: time,ms Tagged by: category, core, group, handler, host, instance |
| p999 | 99.9th percentile request processing time. Type: float | (gauge) Unit: time,ms Tagged by: category, core, group, handler, host, instance |
| rate_15min | Requests per second received over the past 15 minutes. Type: float | (gauge) Unit: throughput,reqps Tagged by: category, core, group, handler, host, instance |
| rate_1min | Requests per second received over the past 1 minute. Type: float | (gauge) Unit: throughput,reqps Tagged by: category, core, group, handler, host, instance |
| rate_5min | Requests per second received over the past 5 minutes. Type: float | (gauge) Unit: throughput,reqps Tagged by: category, core, group, handler, host, instance |
| rate_mean | Average number of requests per second received. Type: float | (gauge) Unit: throughput,reqps Tagged by: category, core, group, handler, host, instance |
| stddev | Standard deviation of request processing time. Type: float | (gauge) Unit: time,ms Tagged by: category, core, group, handler, host, instance |
solr_searcher¶
| Tags & Fields | Description |
|---|---|
| category ( tag) |
Solr metric category that owns the metric. |
| core ( tag) |
Solr core that emitted the metric. |
| group ( tag) |
Solr metric group that owns the metric. |
| host ( tag) |
Hostname of the Solr node. |
| instance ( tag) |
Instance name, generated based on server address. |
| deleted_docs | The number of deleted documents. Type: int | (gauge) Unit: count Tagged by: category, core, group, host, instance |
| max_docs | The largest possible document number. Type: int | (gauge) Unit: count Tagged by: category, core, group, host, instance |
| num_docs | The total number of indexed documents. Type: int | (gauge) Unit: count Tagged by: category, core, group, host, instance |
| warmup | The time spent warming up. Type: int | (gauge) Unit: time,ms Tagged by: category, core, group, host, instance |
Log Collection¶
Example of cutting logs:
Cut fields:
| Field Name | Field Value |
|---|---|
| Reporter | webapp.reporter |
| status | INFO |
| thread | org.apache.solr.core.SolrCore |
| time | 1380630788319000000 |