Skip to content

refactor: Molecule testing, handlers, Tinkey version guard, README #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Azerkyz wants to merge 4 commits into
base: main
Choose a base branch
from
Open

refactor: Molecule testing, handlers, Tinkey version guard, README #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8109bed
refactor: Add Molecule testing for Stream deployment
Nov 27, 2025
1bbe3a7
refactor: add Molecule testing, handlers fixes, Tinkey version guard,…
Mar 20, 2026
31bdf24
fix: apply PR review feedback on secret keys, SELinux and uninstall s…
Apr 1, 2026
381c21e
change to stream node instead of stream cluster
Apr 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Licence files - never commit these!
*.lic
.DS_Store
# Environment files with credentials - never commit these!
*.env
355 changes: 353 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,353 @@
-------
This role deploys Stream in High availabilty mode.
Evertrust Stream
================

Requirements
------------
To be able to use this role you have to previously provision VMs running on **CentOS/RHEL** and meeting the requirements from the official Evertrust documentation.
A root access to these VMs is mandatory, as per a normal RPM install of Stream. You will have to configure your Ansible playbook to use these accounts while playing the role.

**IMPORTANT:** A running instance of MongoDB is also necessary. MongoDB should be accessible from all Stream nodes.

### Ansible Prerequisites

It is necessary to:

1. **Install the ansible.posix collection:**
```bash
ansible-galaxy collection install ansible.posix
```

2. **Configure SSH access:** Set up SSH key-based authentication from your control machine to all target nodes:
```bash
# Generate SSH key if needed
ssh-keygen -t rsa -b 4096

# Copy to each target node
ssh-copy-id root@stream-node1-ip
ssh-copy-id root@stream-node2-ip
```

3. **Dynamic Inventory:** The role includes a dynamic inventory script at `tests/inventory.py` that automatically generates the Ansible inventory from your `mandatory_vars.yml` configuration. The script sets appropriate connection parameters and maps hostnames to IP addresses, eliminating manual inventory management.

4. **Licence File Setup**
Before deployment, you must provide a valid Stream licence file. Create the required directories and place your licence file:
```bash
# For Molecule testing
mkdir -p molecule/default/files
cp /path/to/your/stream.lic molecule/default/files/stream.lic

# For VM deployment
mkdir -p tests/files
cp /path/to/your/stream.lic tests/files/stream.lic
```

Note: The files/ directories are not included in the repository and must be created before running the role.

### Firewall Configuration

**IMPORTANT:** This role automatically configures firewalld with the necessary ports. The following ports are opened:

| Port/Service | Purpose | When Opened |
|--------------|---------|-------------|
| **22 (SSH)** | Remote administration | Always |
| **443** | HTTPS web access | Always |
| **7626** | Akka Management (cluster discovery) | Only for HA (2+ nodes) |
| **17355** | Akka Artery (cluster communication) | Only for HA (2+ nodes) |

**Note:** Port 9000 (Stream application) is bound to `127.0.0.1` only and is NOT exposed externally. Nginx acts as a reverse proxy on port 443.

**Note on Akka/Pekko:** Stream internally uses Pekko (the successor to Akka), but configuration variables maintain the `AKKA_*` naming convention for backwards compatibility with older Stream versions. The functionality and port numbers remain the same.

**For MongoDB:** If managing the MongoDB VM separately, ensure port 27017 is open only to Stream node IPs for security.

## Role Variables

The following table regroups the data that you have to provide the Ansible role with for the deployment and configuration to work properly.

**IMPORTANT:** All these variables must be configured in `defaults/main/mandatory_vars.yml` with real values before deployment.

| Key | Value Type |
|-----|------------|
| `stream_play_http_secret_key` | Random string, 128 characters, may include `@`, `!`, `#` |
| `stream_default_ssv_key` | Random string, 128 characters, may include `@`, `!`, `#` |
| `stream_event_seal_secret` | Random string, 128 characters, may include `@`, `!`, `#` |
| `stream_version` | Stream version (e.g., `2.1.7`) |
| `stream_pkg_uri` | URL where you store the Stream RPM (check architecture: x86_64 or aarch64) |
| `stream_repository_username` | Username to authenticate to the Evertrust repository |
| `stream_repository_password` | Password to authenticate to the Evertrust repository |
| `stream_tinkey_version` | Tinkey version (e.g., `1.12.0.1`) |
| `stream_tinkey_pkg_uri` | URL where you store the Tinkey RPM (check architecture: x86_64 or aarch64) |
| `stream_licence_src_path` | Path to your Stream license on your control machine |
| `stream_mongodb_username` | MongoDB user pre-created on your MongoDB instance |
| `stream_mongodb_password` | Password of the MongoDB user |
| `stream_mongodb_uri` | MongoDB connection string (auto-built from username/password/hostname) |
| `stream_mongodb_shell_package_uri` | URL where you store the Mongosh RPM (check architecture: x86_64 or aarch64) |
| `stream_mongodb_hostname` | Hostname of your MongoDB instance |
| `stream_mongodb_ip` | IP address of your MongoDB instance |
| `stream_nodes` | List of Stream nodes with hostname and IP (see below) |
| `stream_akka_discovery_port` | Port for Akka cluster discovery (default: 7626) |
| `stream_akka_artery_port` | Port for Akka cluster communication (default: 17355) |
| `stream_internal_api_port` | Port for internal API (default: 9000) |
| `stream_configure_firewall` | Set to `false` to skip firewall configuration (default: `true`) |
| `stream_extra_allowed_hosts` | Additional hosts allowed in Play filter, e.g. load balancer hostname (default: `[]`) |

### Tinkey / KMS Variables

| Key | Value Type |
|-----|------------|
| `stream_tink_master_key_uri` | KMS key URI (e.g. `gcp-kms://...` or `aws-kms://...`). Leave empty for soft mode (plaintext keyset). |
| `stream_tink_credentials_src_path` | Path to the KMS credentials file on the Ansible controller. Leave empty for soft mode or HSM. |
| `stream_tink_credentials_path` | Destination path of the KMS credentials file on the VM (default: `/opt/stream/etc/tink-credentials.json`) |

**Stream Nodes Configuration:**

For **standalone deployment** (single node):
```yaml
stream_nodes:
- hostname: stream-node1-vm
ip: 10.10.22.1
```

For **High Availability** (2+ nodes):
```yaml
stream_nodes:
- hostname: stream-node1-vm
ip: 10.10.22.1
- hostname: stream-node2-vm
ip: 10.10.22.2
- hostname: stream-node3-vm
ip: 10.10.22.3
```

**IMPORTANT - Architecture Note:** Ensure the `stream_mongodb_shell_package_uri` matches your system architecture:
- For **x86_64** (most common): Use the x86_64 RPM URL
- For **aarch64** (ARM): Use the aarch64 RPM URL

The default configuration uses the architecture detected at runtime (`ansible_facts['architecture']`). The Stream and Tinkey RPMs default to `noarch` and work on both architectures.

## Overview

The previous table is not exhaustive but the excluded variables can be left as they are for a deployment with default values (valid in most cases). Note that the more variables you customize for your environment and needs, the less configuration will be necessary afterwards.

| Initial variables localisation |
|--------------------------------|
| `defaults/main/default_values.yml` : variables that can optionally be changed |
| `defaults/main/mandatory_vars.yml` : list of variables that must be set before using the role |

This role is composed of the following steps played in order:

### 1. Package Installation

The download and install of the necessary packages to run Stream, including:
- System dependencies (wget, nginx, postfix, firewalld, epel-release)
- Stream RPM package
- Tinkey RPM package (used for AES256 keyset encryption)
- MongoDB shell (mongosh)
- SELinux configuration if enforcing mode is detected

### 2. Certificate Generation

The generation of certificates for Nginx if you didn't configure the variables `stream_csr`, `stream_pem`, `stream_key`, and `stream_chain_pem`. If you did, this step copies the content of these variables to provide your web server.

By default (`stream_generate_default_cert: true`), the role automatically generates self-signed SSL/TLS certificates for Nginx with the following specifications:

- **Key type**: RSA 4096-bit
- **Hashing algorithm**: SHA-256
- **Certificate location**: `/etc/nginx/ssl/`
- **Validity**: 365 days
- **Subject fields**: Configurable via variables in `default_values.yml` (country, organization, OU, CN)

The generated files include:
- `stream.key` - Private key
- `stream.csr` - Certificate Signing Request
- `stream.pem` - Self-signed certificate
- `stream-chain.pem` - Certificate chain

All certificate files are automatically set with proper permissions (root:nginx, 0640).

**If you want to add your own certificate for Stream after deployment:**

1. Copy your certificates to the Stream nodes:
- Certificate: `/etc/nginx/ssl/stream.pem`
- Private key: `/etc/nginx/ssl/stream.key`
- Certificate chain: `/etc/nginx/ssl/stream-chain.pem`
- CSR (optional): `/etc/nginx/ssl/stream.csr`

2. Set proper permissions:
```bash
sudo chown root:nginx /etc/nginx/ssl/stream.*
sudo chmod 640 /etc/nginx/ssl/stream.*
```
3. Test and Reload Nginx:
```bash
sudo nginx -t
sudo systemctl reload nginx
```

### 3. Tinkey Keyset Management

Generation and distribution of the AES256-GCM encryption keyset used by Stream:
- On HA deployments: generated on node 1 and distributed to all other nodes
- On standalone deployments: generated directly on the single node
- Keyset stored at `/opt/stream/etc/stream.keyset` (permissions: `stream:stream 0640`)

### 4. Configuration Provisioning

The provisioning of your Stream licence and the different configuration files needed for Stream to run properly, based on their respective templates:
- Deployment of license file to `/opt/stream/etc/stream.lic`
- Updates to `/etc/hosts` with cluster node entries
- Generation of `/etc/default/stream` with JVM, Play, MongoDB, and Akka cluster settings
- Configuration of hosts.allowed whitelist
- Nginx symlink creation

### 5. Firewall Configuration

Automatic configuration of firewalld to open necessary ports:
- SSH (22) to prevent lockout
- HTTPS (443) for web access
- Akka cluster ports (7626, 17355) for HA deployments only
- Can be disabled by setting `stream_configure_firewall: false`

### 6. Service Management

The start of adequate services:
- Postfix service
- Stream service
- Nginx service (with configuration test)

### Akka Cluster Split-Brain Resolver

**IMPORTANT:** For High Availability deployments (2+ nodes), the role automatically configures Akka's split-brain resolver using a MongoDB-based lease-majority strategy:

```hocon
akka.cluster.split-brain-resolver {
active-strategy = "lease-majority"
lease-majority {
lease-implementation = "lease.mongo"
}
}
```

This configuration ensures proper cluster behavior during network partitions and prevents data inconsistency. It is automatically included in the `hosts_allowed.j2` template.

### Hosts Allowed Configuration

Additionally, a touchy and key element of the Stream configuration is the Play variable "hosts allowed" in the stream-extra.conf file. It configures the whitelist allowed to access Stream.

The role automatically configures this whitelist to include:
- localhost
- All Stream cluster node hostnames
- Any additional hosts defined in `stream_extra_allowed_hosts` (e.g. load balancer hostname, `127.0.0.1` for local testing)

This prevents "Host not allowed" errors when accessing Stream through different hostnames.

## Deployment Types

### Standalone Deployment

For a single Stream instance (no High Availability), configure only one node in `stream_nodes`. The role will:
- Only open HTTPS and SSH ports
- Not open Akka cluster ports
- Configure Stream without cluster formation
- Generate keyset locally

### High Availability Deployment

For a clustered Stream deployment (2+ nodes), configure multiple nodes in `stream_nodes`. The role will:
- Open all required ports including Akka cluster ports
- Configure nodes to automatically discover each other
- Enable split-brain resolver for cluster stability
- Generate and distribute shared keyset from first node
- Provide redundancy and load distribution

## Example Playbook

Here is a basic way of using this role:

```yaml
- name: Deploy Stream
hosts: stream_nodes
become: true
gather_facts: true
roles:
- role: stream
```

### Using the Provided Test Playbook

```bash
# 1. Configure your variables in defaults/main/mandatory_vars.yml

# 2. Syntax check
ansible-playbook -i tests/inventory.py tests/deploy.yml --syntax-check

# 3. Test connectivity
ansible -i tests/inventory.py all -m ping

# 4. Dry run (check mode)
ansible-playbook -i tests/inventory.py tests/deploy.yml --check

# 5. Deploy
ansible-playbook -i tests/inventory.py tests/deploy.yml

# 6. With verbose output
ansible-playbook -i tests/inventory.py tests/deploy.yml -vvv
```

## Check the Installation

If your Ansible role didn't fail during the play, your Stream instance should be deployed.

To test it, run:

```bash
# Check Stream service status
sudo systemctl status stream

# Verify Tinkey installation
ls -la /opt/evertrust/tinkey/bin/tinkey

# Check keyset exists
ls -la /opt/stream/etc/stream.keyset

# Verify firewall configuration
sudo firewall-cmd --list-all

# For HA: Check Akka cluster configuration
grep "AKKA_DISCOVERY_ENDPOINTS" /etc/default/stream

# Check Stream logs
sudo journalctl -u stream -f
```

### Additional Verification Steps

```bash
# Verify nginx configuration
sudo nginx -t

# Check nginx service status
sudo systemctl status nginx

# Check postfix service status
sudo systemctl status postfix
```

## Pre-Deployment Checklist

Before running the playbook, verify:

- [ ] All VMs are provisioned and accessible via SSH
- [ ] SSH keys are copied to all target nodes
- [ ] MongoDB is running and accessible
- [ ] MongoDB user is created with appropriate roles on the `stream` database
- [ ] `mandatory_vars.yml` is configured with real values (no placeholders)
- [ ] MongoDB shell package URI matches your architecture (x86_64 vs aarch64)
- [ ] Stream license file path is correct
- [ ] Ansible and ansible.posix collection are installed
- [ ] For RHEL 9: System is registered with subscription manager
- [ ] Repository credentials are valid for both Stream and Tinkey RPMs

## Author Information

If you have difficulties to use this role or modifications recommendations, please contact Evertrust.
Loading