Add current docs (#311)

Signed-off-by: Anders Swanson <[email protected]>

Author: anders-swanson

site/README.md

@@ -2,6 +2,8 @@
 
 This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
 
+The `content` directory contains the built, static site.
+
 Requirements:
 - latest version of NodeJS
 
@@ -22,7 +24,9 @@ This command starts a local development server and opens up a browser window. Mo
 ## Build
 
 ```bash
+rm -rf content
 npm run build
+cp -r build content
 ```
 
 ## Versioning 

site/docs/advanced/_category_.json

@@ -0,0 +1,8 @@
+{
+  "label": "Advanced",
+  "position": 4,
+  "link": {
+    "type": "generated-index",
+    "description": "Advanced configuration."
+  }
+}

site/docs/advanced/development.md

@@ -0,0 +1,54 @@
+---
+title: Development
+sidebar_position: 3
+---
+
+# Development
+
+The exporter is a Go program using the Prometheus SDK. 
+
+External contributions are welcome, see [CONTRIBUTING](https://github.com/oracle/oracle-db-appdev-monitoring/blob/main/CONTRIBUTING.md) for details.
+
+The exporter initialization is as follows:
+
+- Parse flags options and configuration properties
+- Load the default toml file (`default-metrics.toml`) and store each metric in a `Metric` struct
+- Load the custom toml file (if a custom toml file is given)
+- Create an `Exporter` object
+- Register exporter in prometheus library
+- Launching a web server to handle incoming requests
+- Attempt connection to any configured Oracle Database servers
+
+These operations are mainly done in the `main` function.
+
+After this initialization phase, the exporter will wait for the arrival of a request.
+
+Each time, it will iterate over the content of the `metricsToScrape` structure (in the function scrape `func (e * Export) scrape (ch chan <- prometheus.Metric)`).
+
+For each element (of `Metric` type), a call to the `ScrapeMetric` function will be made which will itself make a call to the `ScrapeGenericValues` function.
+
+The `ScrapeGenericValues` function will read the information from the `Metric` structure and, depending on the parameters, will generate the metrics to return. In particular, it will use the `GeneratePrometheusMetrics` function which will make SQL calls to the database.
+
+### Docker/container build
+
+To build a container image, run the following command:
+
+```bash
+make docker
+```
+
+For ARM:
+
+```bash
+make docker-arm
+```
+
+### Building Binaries
+
+Run build:
+
+```bash
+make go-build
+```
+
+This will create binaries and archives inside the `dist` folder for the building operating system.

site/docs/advanced/go-runtime.md

@@ -0,0 +1,14 @@
+---
+title: Configuring the Go Runtime
+sidebar_position: 1
+---
+
+# Exporter Go Runtime
+
+If you are running in an environment with limited memory, or you are running a large number of exporters, you may want to control the exporter's usage of memory.
+
+Under normal circumstances, the exporter process will retain OS memory that was used by the Go garbage collector but is no longer needed, in case it may be needed again in the future, unless the host OS is under memory pressure.  The result of this behavior (which is the normal behavior of the Go runtime) is that the resident set size will not decrease until the host OS memory is almost all used.  Under most circumstances, this will not cause any issues, but if you are in an environment where you need to conserve memory, the following options are provided:
+
+- You may set the `FREE_INTERVAL` environment variable to a Go [duration string](https://pkg.go.dev/maze.io/x/duration), e.g., `60s` and run the exporter in debug mode by setting the `GODEBUG` environment variable to a value including `madvdontneed=1`, e.g., `GODEBUG=gctrace=1,madvdontneed=1`.  The exporter will call the [FreeOSMemory()](https://pkg.go.dev/runtime/debug#FreeOSMemory) at the specified interval.  This tells the Go runtime to attempt to release memory which is no longer needed.  Please note that this does not guarantee that the memory will be released to the OS, but over time you should see the RSS shrink sooner than without these settings.
+- You may set the `RESTART_INTERVAL` environment variable to a Go [duration string](https://pkg.go.dev/maze.io/x/duration), e.g., `10m`.  The exporter will restart its own process at the specified iterval (by calling the OS `exec` syscall).  As no new process is created, the process identifier (PID) does not change, but the machine code, data, heap, and stack of the process are replaced by those of the new program (source: [Wikipedia](https://en.wikipedia.org/wiki/Exec_(system_call))).  This has the side effect of freeing the resident set, so that it will return to its original size.
+- In addition to these, you may also set `GOMAXPROCS`, `GOGC`, and `GOMEMLIMIT` (see [documentation](https://pkg.go.dev/runtime#hdr-Environment_Variables)) to further limit the amount of resources that the Go runtime may use.

site/docs/advanced/txeventq.md

@@ -0,0 +1,117 @@
+---
+title: Monitoring TxEventQ
+sidebar_position: 2
+---
+
+# Monitoring Transactional Event Queues
+
+[Oracle Transactional Event Queues](https://docs.oracle.com/en/database/oracle/oracle-database/23/adque/index.html) ("TxEventQ") is a fault-tolerant, scalable, real-time messaging backbone offered by converged Oracle Database that allows you to build an enterprise-class event-driven architectures.
+
+Access to the real-time broker, producer, and consumer metrics in a single dashboard and receiving alerts for issues allows teams to understand the state of their system.
+
+The exporter includes a set of metrics for monitoring TxEventQ and a pre-built Grafana dashboard.
+
+> Note: The metrics are written for Oracle Database 21c or later.
+
+### How to create some traffic with PL/SQL
+
+If you need to create a topic to monitor, you can use these statements to create and start a topic, and create a subscriber:
+
+```sql
+declare
+    subscriber sys.aq$_agent;
+begin
+  -- create the topic
+  dbms_aqadm.create_transactional_event_queue(
+    queue_name => 'my_topic',
+    multiple_consumers => true  -- true makes a pub/sub topic
+  );
+
+  -- start the topic
+  dbms_aqadm.start_queue(
+    queue_name => 'my_topic'
+  );
+
+  -- create a subscriber
+  dbms_aqadm.add_subscriber(
+    queue_name => 'my_teq',
+    subscriber => sys.aq$_agent(
+      'my_subscriber',    -- the subscriber name
+      null,               -- address, only used for notifications
+      0                   -- protocol
+    ),
+    rule => 'correlation = ''my_subscriber'''
+  );
+end;
+```
+
+You can produce a message with these commands:
+
+```sql
+declare
+    enqueue_options    dbms_aq.enqueue_options_t;
+    message_properties dbms_aq.message_properties_t;
+    message_handle     raw(16);
+    message            SYS.AQ$_JMS_TEXT_MESSAGE;
+begin
+    -- create the message payload
+    message := SYS.AQ$_JMS_TEXT_MESSAGE.construct;
+    message.set_text('{"orderid": 12345, "username": "Jessica Smith"}');
+
+    -- set the consumer name
+    message_properties.correlation := 'my_subscriber';
+
+    -- enqueue the message
+    dbms_aq.enqueue(
+        queue_name           => 'my_topic',
+        enqueue_options      => enqueue_options,
+        message_properties   => message_properties,
+        payload              => message,
+        msgid                => message_handle);
+
+    -- commit the transaction
+    commit;
+end;
+```
+
+### How to create some traffic with Java (Spring Boot)
+
+A simple load generator is provided in [this directory](https://github.com/oracle/oracle-db-appdev-monitoring/tree/main/docker-compose/txeventq-load) which you can use to create some traffic so you can experiment with the sample dashboard.
+
+To run the sample, first update [application.yaml](https://github.com/oracle/oracle-db-appdev-monitoring/blob/main/docker-compose/txeventq-load/src/main/resources/application.yaml) with the correct IP address for your database, then start the application as follows:
+
+```bash
+mvn spring-boot:run
+```
+
+The application will create ten queues names TOPIC_0 through TOPIC_9 and randomly produce and consume messages on those queues.  The example dashboard shown below was monitoring traffic produced using this application.
+
+### Metrics definitions
+
+The metrics definitions are provided in [this file](https://github.com/oracle/oracle-db-appdev-monitoring/blob/main/custom-metrics-example/txeventq-metrics.toml).  You need to provide this file to the exporter, e.g., by adding it to your container image, or creating a Kubernetes config map containing the file and mounting that config map as a volume in your deployment.  You also need to set the `CUSTOM_METRICS` environment variable to the location of this file.
+
+### Additional database permissions
+
+The database user that the exporter uses to connect to the database will also need additional permissions, which can be granted with these statements.  This example assumes the exporter connects with the username "exporter":
+
+```sql
+grant execute on dbms_aq to exporter;
+grant execute on dbms_aqadm to exporter;
+grant execute on dbms_aqin to exporter;
+grant execute on dbms_aqjms_internal to exporter;
+grant execute on dbms_teqk to exporter;
+grant execute on DBMS_RESOURCE_MANAGER to exporter;
+grant select_catalog_role to exporter;
+grant select on sys.aq$_queue_shards to exporter;
+grant select on user_queue_partition_assignment_table to exporter;
+```
+
+### Grafana dashboard
+
+A Grafana dashboard for Transactional Event Queues is provided [in this file](https://github.com/oracle/oracle-db-appdev-monitoring/blob/main/docker-compose/grafana/dashboards/txeventq.json).  This can be imported into your Grafana environment.  Choose the Prometheus datasource that is collecting metrics from the exporter.
+
+> Note:  You may not see any activity on the dashboard unless there are clients producing and consuming messages from topics.
+
+The dashboard will look like this:
+
+![Oracle Database Dashboard](/img/txeventq-dashboard-v2.png)

site/docs/configuration/alert-logs.md

@@ -0,0 +1,30 @@
+---
+title: Alert Logs
+sidebar_position: 5
+---
+
+# Alert logs
+
+Collect export alert logs with a log ingestion tool.
+
+The exporter exports alert log records as a JSON file suitable for collection by a log ingestion tool like Promtail or FluentBit.
+
+Alert logging is configured with the following parameters in the exporter config file:
+
+| Parameter       | Description                   | Default          |
+|-----------------|-------------------------------|------------------|
+| log.destination | Log file path                 | `/log/alert.log` |
+| log.interval    | Interval to log records       | `15s`            |
+| log.disable     | Disable logging if set to `1` | `0`              |
+
+Example alert log YAML configuration:
+
+```yaml
+log:
+  # Path of log file
+  destination: /opt/exporter/alert.log
+  # Interval of log updates
+  interval: 15s
+  ## Set disable to 1 to disable logging
+  # disable: 0
+```

site/docs/configuration/azure-vault.md

@@ -0,0 +1,40 @@
+---
+title: Azure Vault
+sidebar_position: 7
+---
+
+# Azure Vault
+
+Securely load database credentials from Azure Vault.
+
+Each database in the config file may be configured to use Azure Vault. To load the database username and/or password from Azure Vault, set the `vault.azure` property to contain the Azure Vault ID, and secret names for the database username/password:
+
+```yaml
+databases:
+  mydb:
+    vault:
+      azure:
+        id: <VAULT ID>
+        usernameSecret: <Secret containing DB username>
+        passwordSecret: <Secret containing DB password>
+```
+
+### Authentication
+
+If you are running the exporter outside Azure, we recommend using [application service principal](https://learn.microsoft.com/en-us/azure/developer/go/sdk/authentication/authentication-on-premises-apps).
+
+If you are running the exporter inside Azure, we recommend using a [managed identity](https://learn.microsoft.com/en-us/azure/developer/go/sdk/authentication/authentication-azure-hosted-apps).
+
+You should set the following additional environment variables to allow the exporter to authenticate to Azure:
+
+- `AZURE_TENANT_ID` should be set to your tenant ID
+- `AZURE_CLIENT_ID` should be set to the client ID to authenticate to Azure
+- `AZURE_CLIENT_SECRET` should be set to the client secret to authenticate to Azure
+
+### Azure Vault CLI Configuration (without exporter config file)
+
+If using the default database with CLI parameters, the exporter will read the database username and password from secrets stored in Azure Key Vault if you set these environment variables:
+
+- `AZ_VAULT_ID` should be set to the ID of the Azure Key Vault that you wish to use
+- `AZ_VAULT_USERNAME_SECRET` should be set to the name of the secret in the Azure Key Vault which contains the database username
+- `AZ_VAULT_PASSWORD_SECRET` should be set to the name of the secret in the Azure Key Vault which contains the database password

site/docs/configuration/config-file.md

@@ -0,0 +1,103 @@
+---
+title: Exporter Config File
+sidebar_position: 1
+---
+
+# Exporter Config File
+
+The recommended way to configure the exporter is with the `--config.file` argument, specifying a YAML configuration file.
+
+The configuration file contains the following options:
+
+```yaml
+# Example Oracle Database Metrics Exporter Configuration file.
+# Environment variables of the form ${VAR_NAME} will be expanded.
+# If you include a config value that contains a '$' character, escape that '$' with another '$', e.g.,
+# "$test$pwd" => "$$test$$pwd"
+# Otherwise, the value will be expanded as an environment variable.
+
+# Example Oracle Database Metrics Exporter Configuration file.
+# Environment variables of the form ${VAR_NAME} will be expanded.
+
+databases:
+  ## Path on which metrics will be served
+  # metricsPath: /metrics
+  ## Database connection information for the "default" database.
+  default:
+    ## Database username
+    username: ${DB_USERNAME}
+    ## Database password
+    password: ${DB_PASSWORD}
+    ## Database password file
+    ## If specified, will load the database password from a file.
+    # passwordFile: ${DB_PASSWORD_FILE}
+    ## Database connection url
+    url: localhost:1521/freepdb1
+
+    ## Metrics query timeout for this database, in seconds
+    queryTimeout: 5
+
+    ## Rely on Oracle Database External Authentication by network or OS
+    # externalAuth: false
+    ## Database role
+    # role: SYSDBA
+    ## Path to Oracle Database wallet, if using wallet
+    # tnsAdmin: /path/to/database/wallet
+
+    ### Connection settings:
+    ### Either the go-sql or Oracle Database connection pool may be used.
+    ### To use the Oracle Database connection pool over the go-sql connection pool,
+    ### set maxIdleConns to zero and configure the pool* settings.
+
+    ### Connection pooling settings for the go-sql connection pool
+    ## Max open connections for this database using go-sql connection pool
+    maxOpenConns: 10
+    ## Max idle connections for this database using go-sql connection pool
+    maxIdleConns: 10
+
+    ### Connection pooling settings for the Oracle Database connection pool
+    ## Oracle Database connection pool increment.
+    # poolIncrement: 1
+    ## Oracle Database Connection pool maximum size
+    # poolMaxConnections: 15
+    ## Oracle Database Connection pool minimum size
+    # poolMinConnections: 15
+
+    ## Arbitrary labels to add to each metric scraped from this database
+    # labels:
+    #   label_name1: label_value1
+    #   label_name2: label_value2
+
+metrics:
+  ## How often to scrape metrics. If not provided, metrics will be scraped on request.
+  # scrapeInterval: 15s
+  ## Path to default metrics file.
+  default: default-metrics.toml
+  ## Paths to any custom metrics files
+  custom:
+    - custom-metrics-example/custom-metrics.toml
+
+log:
+  # Path of log file
+  destination: /opt/alert.log
+  # Interval of log updates
+  interval: 15s
+  ## Set disable to 1 to disable logging
+  # disable: 0
+
+# Optionally configure prometheus webserver
+#web:
+#  listenAddresses: [':9161']
+#  systemdSocket: true|false
+#  configFile: /path/to/webconfigfile
+```
+
+### Config file in a container image
+
+To add your custom config file to a container image, you can layer the base exporter image and include that config:
+
+```Dockerfile
+FROM container-registry.oracle.com/database/observability-exporter:2.0.2
+COPY my-exporter-config.yaml /
+ENTRYPOINT ["/oracledb_exporter", "--config.file", "/my-exporter-config.yaml"]
+```