Skip to content

Solr

·


Solr collector, which collects statistics of Solr Cache, Request Times, and so on.

Configuration

Preconditions

DataKit uses the Solr Metrics API to collect metrics data and supports Solr 7.0 and above. Available for Solr 6.6, but the indicator data is incomplete.

Already tested version:

  • 8.11.2
  • 7.0.0

Collector Configuration

Go to the conf.d/db directory under the DataKit installation directory, copy solr.conf.sample and name it solr.conf. Examples are as follows:

[[inputs.solr]]
  ##(optional) collect interval, default is 10 seconds
  interval = '10s'

  ## specify a list of one or more Solr servers
  servers = ["http://localhost:8983"]

  ## Optional HTTP Basic Auth Credentials
  # username = "username"
  # password = "pa$$word"

  ## Set true to enable election
  election = true

  # [inputs.solr.log]
  # files = []
  # #grok pipeline script path
  # pipeline = "solr.p"

  [inputs.solr.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"

After configuration, restart DataKit.

The collector can now be turned on by ConfigMap Injection Collector Configuration.


To collect Solr's log, open files in Solr.conf and write to the absolute path of the Solr log file. For example:

[inputs.solr.log]
    # fill in the absolute path
    files = ["/path/to/demo.log"]

Metric

For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.solr.tags] if needed:

 [inputs.solr.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

solr_cache

Tags & Fields Description
category
(tag)
Category name.
core
(tag)
Solr core.
group
(tag)
Metric group.
host
(tag)
System hostname.
instance
(tag)
Instance name, generated based on server address.
name
(tag)
Cache name.
cumulative_evictions Number of cache evictions across all caches since this node has been running.
Type: int | (gauge)
Unit: count
cumulative_hitratio Ratio of cache hits to lookups across all the caches since this node has been running.
Type: float | (gauge)
Unit: percent,percent
cumulative_hits Number of cache hits across all the caches since this node has been running.
Type: int | (gauge)
Unit: count
cumulative_inserts Number of cache insertions across all the caches since this node has been running.
Type: int | (gauge)
Unit: count
cumulative_lookups Number of cache lookups across all the caches since this node has been running.
Type: int | (gauge)
Unit: count
evictions Number of cache evictions for the current index searcher.
Type: int | (gauge)
Unit: count
hitratio Ratio of cache hits to lookups for the current index searcher.
Type: float | (gauge)
Unit: percent,percent
hits Number of hits for the current index searcher.
Type: int | (gauge)
Unit: count
inserts Number of inserts into the cache.
Type: int | (gauge)
Unit: count
lookups Number of lookups against the cache.
Type: int | (gauge)
Unit: count
max_ram Maximum heap that should be used by the cache beyond which keys will be evicted.
Type: int | (gauge)
Unit: digital,MB
ram_bytes_used Actual heap usage of the cache at that particular instance.
Type: int | (gauge)
Unit: digital,B
size Number of entries in the cache at that particular instance.
Type: int | (gauge)
Unit: count
warmup Warm-up time for the registered index searcher. This time is taken in account for the "auto-warming" of caches.
Type: int | (gauge)
Unit: time,ms

solr_request_times

Tags & Fields Description
category
(tag)
Category name.
core
(tag)
Solr core.
group
(tag)
Metric group.
handler
(tag)
Request handler.
host
(tag)
System hostname.
instance
(tag)
Instance name, generated based on server address.
count Total number of requests made since the Solr process was started.
Type: int | (gauge)
Unit: count
max Max of all the request processing time.
Type: float | (gauge)
Unit: time,ms
mean Mean of all the request processing time.
Type: float | (gauge)
Unit: time,ms
median Median of all the request processing time.
Type: float | (gauge)
Unit: time,ms
min Min of all the request processing time.
Type: float | (gauge)
Unit: time,ms
p75 Request processing time for the request which belongs to the 75th Percentile.
Type: float | (gauge)
Unit: time,ms
p95 Request processing time in milliseconds for the request which belongs to the 95th Percentile.
Type: float | (gauge)
Unit: time,ms
p99 Request processing time in milliseconds for the request which belongs to the 99th Percentile.
Type: float | (gauge)
Unit: time,ms
p999 Request processing time in milliseconds for the request which belongs to the 99.9th Percentile.
Type: float | (gauge)
Unit: time,ms
rate_15min Requests per second received over the past 15 minutes.
Type: float | (gauge)
Unit: throughput,reqps
rate_1min Requests per second received over the past 1 minutes.
Type: float | (gauge)
Unit: throughput,reqps
rate_5min Requests per second received over the past 5 minutes.
Type: float | (gauge)
Unit: throughput,reqps
rate_mean Average number of requests per second received
Type: float | (gauge)
Unit: throughput,reqps
stddev Stddev of all the request processing time.
Type: float | (gauge)
Unit: time,ms

solr_searcher

Tags & Fields Description
category
(tag)
Category name.
core
(tag)
Solr core.
group
(tag)
Metric group.
host
(tag)
System hostname.
instance
(tag)
Instance name, generated based on server address.
deleted_docs The number of deleted documents.
Type: int | (gauge)
Unit: count
max_docs The largest possible document number.
Type: int | (gauge)
Unit: count
num_docs The total number of indexed documents.
Type: int | (gauge)
Unit: count
warmup The time spent warming up.
Type: int | (gauge)
Unit: time,ms

Log Collection

Example of cutting logs:

2013-10-01 12:33:08.319 INFO (org.apache.solr.core.SolrCore) [collection1] webapp.reporter

Cut fields:

Field Name Field Value
Reporter webapp.reporter
status INFO
thread org.apache.solr.core.SolrCore
time 1380630788319000000

Feedback

Is this page helpful? ×