Skip to content

Scale setup script to bootstrap N simulated Edge instances #64

New issue
New issue
Open
Open
Scale setup script to bootstrap N simulated Edge instances#64
@amalbet

Description

@amalbet
amalbet
opened on Mar 10, 2026

User Story

As a platform developer, I want to launch N simulated Edge instances from the setup script, so that I can test the billing engine and UI at higher scale with multiple edges and meters.

Context

The current setup bootstraps exactly one Edge (edge0) with 4 simulated meters (1 grid, 2 production, 1 consumption). To test multi-tenant billing at scale, we need N edges each running independent simulations. Currently scaling requires manually duplicating config files, editing UUIDs, registering devices in Postgres, and adding docker-compose services — all error-prone and tedious.

The setup script (setup.sh on feature/setup-script) already handles single-edge bootstrap: builds images, initializes Odoo, registers edge0, validates configs, starts the stack, and verifies connectivity. This ticket extends that to N edges.

What defines one Edge instance

Each simulated Edge requires (from the git-tracked template in openems-edge/config.d/):

Category Count Files Key IDs
Backend API 1 Controller/Api/Backend/<uuid>.config ctrlBackend0
Other Controllers 10 Controller/Api/{ModbusTcp,Rest,Websocket}/<uuid>.config, Controller/{CHP,Debug,Ess,IO,Symmetric}/<uuid>.config ctrlApiModbusTcp0, ctrlApiRest0, ctrlApiWebsocket0, ctrlChpSoc0, ctrlDebugLog0, ctrlLimitTotalDischarge0, ctrlChannelSingleThreshold0, ctrlIoFixDigitalOutput0, ctrlIoHeatingElement0, ctrlBalancing0
Datasources 3 Simulator/Datasource/Single/Direct/<uuid>.config datasource0 (consumption, 500W), datasource1 (PV West, 3000W), datasource2 (PV East, 2000W)
Grid Meter 1 Simulator/GridMeter/Reacting/<uuid>.config meter0
Production Meters 2 Simulator/ProductionMeter/Acting/<uuid>.config meter1 (PV West), meter2 (PV East)
Consumption Meter 1 Simulator/NRCMeter/Acting/<uuid>.config meter3
ESS (battery) 1 Simulator/EssSymmetric/Reacting/<uuid>.config ess0
IO (relays) 1 Simulator/IO/DigitalInputOutput/<uuid>.config io0
Scheduler 1 Scheduler/AllAlphabetically/<uuid>.config scheduler0
Timedata 1 Timedata/Rrd4j/rrd4j0.config rrd4j0
Logging/Felix 3 org/ops4j/pax/logging.config, org_apache_felix_cm_impl_DynamicBindings.config (2 files + 1 dir-level) —

Total: 24 git-tracked config files. An additional ~11 singleton configs (Core/*, Ess/Power, Evcs/SlowPowerIncreaseFilter) are auto-generated by Felix at runtime and should NOT be included in the template — they appear in the working directory as untracked files after the first run.

Per-edge requirements:

  1. Unique config directory — openems-edge/config.d/ is the template; each edge needs its own copy with unique service.pid values and matching filenames
  2. Unique apikey — must match between the Backend config file (Controller/Api/Backend/<uuid>.config:4) and the Odoo openems_device table
  3. Unique device name — edge0, edge1, ..., edgeN-1 registered in openems_device
  4. Unique Docker service — separate container, unique port mappings, own volume mount
  5. Backend connection — all edges connect to the same ws://openems-backend:8081 (Backend Edge/Websocket.config listens on port 8081 and multiplexes edges by apikey lookup against Odoo)

Acceptance Criteria

  • setup.sh accepts --edges N flag (default: 1) to configure the number of Edge instances
  • For each edge i (0 to N-1): config directory generated at openems-edge/config-edge{i}/ from the template in openems-edge/config.d/
  • Each generated config has unique service.pid values (filename must match last PID segment per Felix ConfigAdmin rules)
  • Each edge registered in Odoo database as edge{i} with a unique apikey
  • docker-compose.override.yml (or equivalent) generated with N edge services, each with unique ports and volume mount
  • All N edges connect to the backend and show as online
  • ./setup.sh --edges 3 bootstraps a full stack with 3 independent edges, each running 4 simulated meters
  • ./setup.sh (no flag) behaves identically to current behavior (1 edge)

Out of Scope

  • Varying simulation profiles per edge (deferred — all edges run identical meter profiles for now)
  • Custom meter counts per edge (deferred — each edge gets the same 4 meters)
  • Horizontal backend scaling (single backend handles all edges)
  • Kubernetes/ECS deployment of multi-edge (local Docker only)

Dev Notes

Current state of the template

The git-tracked template at openems-edge/config.d/ contains 24 config files (on feature/rrd4j-fix). Of these:

  • 21 use factory pattern with UUID filenames (e.g., Controller/Api/Backend/28628d01-a978-4328-9a8f-b9c7751ed2d2.config)
  • 1 uses component-id filename (Timedata/Rrd4j/rrd4j0.config — see PR Fix RRD4j config naming for Felix ConfigAdmin #63)
  • 2 are logging/Felix internals (org/ops4j/pax/logging.config, org_apache_felix_cm_impl_DynamicBindings.config)

The ~11 Core/*, Ess/Power.config, and Evcs/SlowPowerIncreaseFilter.config files visible in the working directory are runtime-generated Felix singletons (untracked in git). They must NOT be included in the template — Felix auto-creates them on startup.

Recommended approach: template + generation script

Treat openems-edge/config.d/ as a template and generate per-edge configs at setup time.

Config generation logic (per edge i):

  1. Copy entire template dir to openems-edge/config-edge{i}/
  2. For each .config file with a UUID filename:
    • Generate a new UUID
    • Rename the file to <new-uuid>.config
    • Inside the file, update service.pid to use the new UUID
  3. Critical: update LDAP target filters. Four configs embed their own service.pid UUID inside datasource.target or Component.target LDAP filter strings. When the UUID changes, these filters must be updated too:
    • Controller/Api/ModbusTcp/ReadOnly/<uuid>.config:2 — Component.target references own PID
    • Simulator/NRCMeter/Acting/<uuid>.config:4 — datasource.target references own PID
    • Simulator/ProductionMeter/Acting/<uuid-1>.config:4 — datasource.target references own PID
    • Simulator/ProductionMeter/Acting/<uuid-2>.config:4 — datasource.target references own PID
  4. Update Controller/Api/Backend/<uuid>.config:
    • apikey → unique per edge (generate random 20-char key, matching addons/openems/models/device.py:181-192 format)
    • uri stays ws://openems-backend:8081 (all edges share one backend)
  5. Component id values (meter0, ess0, etc.) stay the same within each edge — uniqueness is per-JVM container, not global

docker-compose.override.yml generation

Docker Compose automatically merges docker-compose.override.yml with docker-compose.yml when running docker compose up. For multi-edge, generate an override that:

  1. Disables the default openems-edge service — set profiles: ["disabled"] or deploy: { replicas: 0 } to prevent the original from starting alongside the generated per-edge services
  2. Adds N edge services with unique names, ports, and volume mounts
# Generated by setup.sh --edges 3
services:
  # Disable the default single-edge service from docker-compose.yml
  openems-edge:
    profiles: ["disabled"]

  openems-edge-0:
    image: openems-edge:latest
    ports:
      - "8080:8080"   # Felix console
      - "8085:8085"   # Edge websocket
    volumes:
      - ./openems-edge/config-edge0:/etc/openems.d

  openems-edge-1:
    image: openems-edge:latest
    ports:
      - "8180:8080"
      - "8185:8085"
    volumes:
      - ./openems-edge/config-edge1:/etc/openems.d

  openems-edge-2:
    image: openems-edge:latest
    ports:
      - "8280:8080"
      - "8285:8085"
    volumes:
      - ./openems-edge/config-edge2:/etc/openems.d

Port scheme: edge i gets Felix console on host port 8{i}80 → container 8080, websocket on host port 8{i}85 → container 8085. No conflicts with existing services (Backend uses 8079/8081/8082; UI uses 4200; InfluxDB uses 8086; Odoo uses 10016/20016). Scheme supports up to 10 edges (i=0..9) before port collisions at 8{10}80.

Note: each edge also exposes REST API on container port 8084 internally (Controller/Api/Rest/ReadOnly config), but this is not mapped to host in the current docker-compose.

Database registration (extend setup.sh Step 4, lines 97-116)

The current Step 4 (setup.sh:97-116) handles single-edge registration. Extend to loop over N edges:

for i in $(seq 0 $((EDGE_COUNT - 1))); do
  EDGE_NAME="edge${i}"
  EDGE_KEY=$(generate_apikey)  # random 20-char alphanumeric, matching Odoo model format
  # Check/insert/update in openems_device table
  # Then write the apikey into config-edge${i}/Controller/Api/Backend/<uuid>.config
done

The openems_device table requires: name (unique), apikey (unique, required), create_uid, create_date, write_uid, write_date. See addons/openems/models/device.py:13-16 for uniqueness constraints and device.py:83 for apikey requirement.

Current apikey mismatch to resolve

The template config at openems-edge/config.d/Controller/Api/Backend/28628d01-a978-4328-9a8f-b9c7751ed2d2.config:4 has apikey="DEMO_API_KEY". The setup script at setup.sh:32 hardcodes EDGE_APIKEY="E3SZ5xdJ2FJ0jPMtajdm" and writes that to the DB, but never updates the config file — so there is a mismatch on fresh setups. For multi-edge, each edge gets its own generated apikey written to both the config file and the DB, which also fixes this existing bug.

Verification updates (extend setup.sh Step 7, lines 135-183)

The current verification checks (setup.sh:138-183) use hardcoded service name openems-edge. For N edges, loop:

for i in $(seq 0 $((EDGE_COUNT - 1))); do
  check_logs "openems-edge-${i}" "Scheduler"
  check_logs "openems-edge-${i}" "Rrd4j"
done

Also restart all edge containers (setup.sh:133 currently restarts only openems-edge).

Backend handles multiple edges without changes

The Backend listens on a single websocket port (Edge/Websocket.config, port 8081). Each edge connects and authenticates with its apikey. The Backend looks up the apikey in openems_device via Odoo metadata (Metadata/Odoo.config) to resolve the edge name. No backend config changes are needed for N edges.

Key constraint: Felix ConfigAdmin filename rule

The config filename MUST be the last segment of service.pid. Felix FilePersistenceManager converts path/to/file.config → PID path.to.file. If mismatch, config is silently ignored. See #61 for the full investigation.

When generating configs, if you change service.pid, you MUST also rename the file. For factory configs (21 of 24): generate new UUID, rename file, update service.pid and any LDAP filter references. For the 3 non-factory configs: copy as-is (no UUID to change).

Key files to modify

setup.sh:24-28        ← Add --edges flag parsing (extend arg loop)
setup.sh:32           ← Remove hardcoded EDGE_APIKEY, generate per-edge
setup.sh:97-116       ← Extend Step 4 to loop over N edges
setup.sh:125-133      ← Extend Step 6 to start/restart all edge services
setup.sh:135-183      ← Extend Step 7 verification for N edge services
setup.sh:186-195      ← Update summary output with per-edge URLs

Key files to read (template)

openems-edge/config.d/                      ← 24 git-tracked config files (template for generation)
openems-edge/Dockerfile:12                  ← JVM cmd: java -Dfelix.cm.dir=/etc/openems.d/ -jar openems.jar
openems-backend/config.d/Edge/Websocket.config  ← Backend edge listener (port 8081, no per-edge config needed)
addons/openems/models/device.py:13-16       ← Uniqueness constraints (name, serial number)
addons/openems/models/device.py:83          ← apikey field (required, unique)
addons/openems/models/device.py:181-192     ← API key generation format (20-char alphanumeric)

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions