{product-title} uses Elasticsearch (ES) to organize the log data from Fluentd into datastores, or indices.
Elasticsearch subdivides each index into multiple pieces called shards, which it spreads across a set of Elasticsearch nodes in an Elasticsearch cluster. You can configure Elasticsearch to make copies of the shards, called replicas. Elasticsearch also spreads these replicas across the Elasticsearch nodes. The ClusterLogging Custom Resource allows you to specify the replication policy in the Custom Resource Definition (CRD) to provide data redundancy and resilience to failure.
|
Note
|
The number of primary shards for the index templates is equal to the number of Elasticsearch data nodes. |
The Cluster Logging Operator and companion Elasticsearch Operator ensure that each Elasticsearch node is deployed using a unique Deployment that includes its own storage volume. You can use a Cluster Logging Custom Resource (CR) to increase the number of Elasticsearch nodes. Refer to Elastic’s documentation for considerations involved in choosing storage and network location as directed below.
|
Note
|
A highly-available Elasticsearch environment requires at least three Elasticsearch nodes, each on a different host. |
Role-based access control (RBAC) applied on the Elasticsearch indices enables the controlled access of the logs to the developers. Access to the indexes with the project.{project_name}.{project_uuid}.* format is restricted based on the permissions of the user in the specific project.
For more information, see Elasticsearch (ES).