You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Storing long-term metrics with Prometheus in CrateDB
3
3
4
-
In this tutorial, I show how to
4
+
This tutorial shows how to:
5
5
6
6
* Set up Docker Compose to run CrateDB, Prometheus, and the CrateDB Prometheus Adapter
7
7
* Run the applications with Docker Compose
8
8
9
-
*Note: this blog post uses CrateDB 4.7.0, Prometheus 2.33.3 and CrateDB Prometheus Adapter 0.4.0*
9
+
:::{note}
10
+
These examples use CrateDB 4.7.0, Prometheus 2.33.3, and the CrateDB Prometheus Adapter 0.5.8.
11
+
:::
10
12
11
13
## Motivation
12
14
13
-
[Prometheus](https://prometheus.io/docs/introduction/overview/) is a monitoring software for collecting metrics data from applications and infrastructures. Its focus lies on collecting big amounts of concise event data from the monitored system, roughly timestamp points with key-value pairs.
15
+
[Prometheus] is a monitoring system that collects metrics from applications and
16
+
infrastructure. It focuses on ingesting large volumes of concise time‑series
17
+
events—timestamped points with key‑value labels.
14
18
15
-
Such data is very useful to track the state and trajectory of a system, so storing this data for the long term is a common need for Prometheus users.
19
+
This data helps you track a system’s state and trajectory. Many Prometheus users
20
+
want to retain it long term.
16
21
17
-
This is where [CrateDB](https://cratedb.com/database) comes into place. With the [CrateDB Prometheus Adapter](https://github.com/crate/cratedb-prometheus-adapter), one can easily store the collected metrics data in CrateDB and take advantage of its high ingestion and query speed and friendly UI to massively scale-out Prometheus.
22
+
[CrateDB] helps here. With the [CrateDB Prometheus Adapter], you can store metrics
23
+
in CrateDB and use its fast ingestion and query performance to scale Prometheus.
18
24
19
25
## Set up Docker Compose
20
26
21
-
Both CrateDB, Prometheus, and the CrateDB Prometheus Adapter applications can be run as [Docker containers](https://www.docker.com/resources/what-container). To then centralize the container management I use [Docker Compose](https://docs.docker.com/compose/), this way I can build and run all the containers with a single command and set up the connections between them in a YAML file.
27
+
Run CrateDB, Prometheus, and the CrateDB Prometheus Adapter as [Docker containers].
28
+
Use [Docker Compose] to centralize container management so you can build and run
29
+
all containers with one command and define their connections in a YAML file.
22
30
23
-
Before anything else, I follow the [Docker Installation Tutorial](https://docs.docker.com/get-docker/) to get Docker in my local machine.
31
+
Install Docker by following the [Docker installation guide].
24
32
25
-
Then, I create a directory in my local machine to host the necessary configuration files.
26
-
I’ll have a total of three of them, all following the YAML format. They can be easily created using any [text editor](https://www.computerhope.com/jargon/e/editor.htm), like TextEdit on a Mac, and then saved with the `.yml`format.
33
+
Create a directory on your machine to host the configuration files.
34
+
Create three YAML files with your preferred editorand save them with the `.yml`extension.
27
35
28
36
### Create `docker-compose.yml`
29
37
30
-
The first YAML file I create is `docker-compose.yml`, which wraps up the configurations for the three containers.
31
-
32
-
I specify CrateDB, Prometheus, and Adapter as services. Then, I add `config.yml` and `prometheus.yml` files as volumes to the Adapter and Prometheus containers, respectively. These files will be created in the following steps.
38
+
Create `docker-compose.yml` to configure the three containers.
39
+
Define services for CrateDB, Prometheus, and the adapter. Mount `config.yml`
40
+
and `prometheus.yml` into the adapter and Prometheus containers, respectively.
41
+
You will create these files in the following steps.
Next, following the[ Prometheus Documentation](https://prometheus.io/docs/prometheus/latest/getting_started/), I create a `prometheus.yml` file, which holds the scraping configuration for whichever service Prometheus collects metrics from.
75
+
Next, create `prometheus.yml` following the [Prometheus documentation].
76
+
This file defines the services that Prometheus scrapes.
57
77
58
-
To keep it simple, I follow the example in the Prometheus documentation and set it to monitor itself.
78
+
To keep it simple, monitor Prometheus itself.
59
79
60
-
One last bit of configuration necessary to forward requests from Prometheus to the CrateDB Adapter is to set `remote_write` and `remote_read` to the Adapter URL, as stated in [CrateDB Prometheus Adapter Setup](https://github.com/crate/cratedb-prometheus-adapter).
80
+
To forward samples to CrateDB, set `remote_write` and `remote_read` to the adapter URL (see the [CrateDB Prometheus Adapter setup](https://github.com/crate/cratedb-prometheus-adapter)).
61
81
62
-
As I’m running the Adapter on Docker instead of locally, the host in its URL will not be `localhost`, but rather however I called the Adapter service previously in my `docker-compose.yml` file, in this case, `cratedb-prometheus-adapter`.
82
+
Because the adapter runs in Docker, use the adapter service name from `docker-compose.yml` (`cratedb-prometheus-adapter`) instead of `localhost` in the URLs.
63
83
64
-
The resulting prometheus.yml looks then like this:
84
+
Use the following `prometheus.yml`:
65
85
```yaml
66
86
global:
67
87
scrape_interval: 15s # By default, scrape targets every 15 seconds.
Finally, following the [CrateDB Prometheus Adapter setup instructions](https://github.com/crate/cratedb-prometheus-adapter), I create the `config.yml` file, which defines the CrateDB endpoints the Adapter writes to.
113
+
Finally, create `config.yml` following the [CrateDB Prometheus Adapter].
114
+
This file defines the CrateDB endpoints that the adapter writes to.
94
115
95
-
As I did previously in the `prometheus.yml` file, the host is set to `cratedb`, which is how I declared the CrateDB service on the `docker-compose.yml` file, instead of the default `localhost`. The remaining variables are set with their default values.
116
+
Set the host to `cratedb` (the service name in `docker-compose.yml`) instead of `localhost`. Keep the remaining variables at their defaults.
96
117
```yaml
97
118
cratedb_endpoints:
98
119
- host: "cratedb" # Host to connect to (default: "localhost")
@@ -105,18 +126,18 @@ cratedb_endpoints:
105
126
enable_tls: false # Whether to connect using TLS (default: false).
I make sure both `docker-compose.yml`, `config.yml`, and `prometheus.yml` are in the same directory in my local machine.
129
+
Place `docker-compose.yml`, `config.yml`, and `prometheus.yml` in the same directory on your machine.
109
130
110
131
## Start the services
111
132
112
-
Finally, I navigate to my CrateDB-Prometheus directory in my terminal and start Docker Compose with the `docker-compose up` command
133
+
Finally, navigate to your CrateDB–Prometheus directory and start Docker Compose:
113
134
```shell
114
-
$ cd /Users/Path/To/Directory/CrateDB-Prometheus
115
-
$ docker-compose up
135
+
cd /Users/Path/To/Directory/CrateDB-Prometheus
136
+
docker-compose up
116
137
```
117
138
118
-
With Docker Compose up and running, I follow the [CrateDB Prometheus Adapter setup instructions](https://github.com/crate/cratedb-prometheus-adapter), navigate to the [CrateDB Admin UI](https://www.google.com/search?client=safari&rls=en&q=cratedb+admin+ui&ie=UTF-8&oe=UTF-8) at [http://localhost:4200](http://localhost:4200/) and create a `metrics` table in CrateDB, which will store the metrics gathered by Prometheus.
119
-
139
+
After Docker Compose starts, open the CrateDB Admin UI at `http://localhost:4200`
140
+
and create a `metrics` table to store metrics gathered by Prometheus.
Navigate to `http://localhost:9090` to open the Prometheus UI. Go to **Status** → **Targets**.
154
+
155
+
156
+
157
+
Confirm that Prometheus monitors itself.
158
+
159
+
160
+
161
+
Return to the CrateDB Admin UI and select the `metrics` table you created.
134
162
135
-
And confirm that Prometheus is successfully monitoring itself.
163
+
After a few minutes, Prometheus will have gathered hundreds of thousands of data points.
0 commit comments