Skip to content

WARP.md

Latest commit

 

History

History
139 lines (106 loc) · 5.19 KB

WARP.md

File metadata and controls

139 lines (106 loc) · 5.19 KB

WARP.md

This file provides guidance to WARP (warp.dev) when working with code in this repository.

Project Overview

This is the Transpara OPC UA Extractor, a Python-based service that connects to OPC UA servers, monitors data nodes, and forwards data to TStore. It supports three operation modes: ACTIVE (full operation), LOG_ONLY (monitoring without data forwarding), and CONFIG (configuration validation only).

Core Architecture

Main Components

  • app/main.py: Entry point with operation mode management, resource lifecycle, and FastAPI server
  • app/opcua_client.py: Core OPC UA client logic with connection management, subscription handling, and enumeration detection
  • app/core/config.py: Configuration management using Pydantic settings with YAML config loading
  • app/api/: FastAPI REST API for testing connections and validating configurations
  • app/state.py: Application state management singleton

State Machine Flow

The application follows a complex state machine documented in the README.md mermaid diagram:

  1. Start → Operation Mode Validation → Resource Initialization
  2. OPC UA Connection → Node Monitoring → Data Handling
  3. Error Handling with automatic reconnection and retry logic

Key Design Patterns

  • Async Context Manager: manage_resources() handles complete resource lifecycle
  • State Management: Connection state tracking with counters (connection_success_count, consecutive_errors)
  • Enumeration Caching: OpcUaEnumerationsManager detects and caches OPC UA enumerations
  • Retry Logic: Exponential backoff for connections and subscriptions

Development Commands

Local Development

# Install dependencies
pip install -r requirements.txt

# Run the application directly
python app/main.py

# Run FastAPI development server only
python start.py

Docker Development

# Build Docker image
docker build -t transpara/extractor-opcua:dev .

# Build and push (production)
./build.sh

# Run with docker-compose
docker-compose up

Testing

# The application includes built-in REST API endpoints for testing:
# - POST /api/v1/test-connection - Test OPC UA server connectivity
# - POST /api/v1/validate-items - Validate monitored items
# - GET /api/v1/available-items - Get available OPC UA items
# - GET /api/v1/health - Health check endpoint

Configuration

Primary Configuration Files

  • app/opcua_config.yaml: OPC UA server settings, monitored items, security configuration
  • Environment Variables: Runtime settings via .env or container environment

Key Settings

  • OPERATION_MODE: Controls application behavior (ACTIVE/LOG_ONLY/CONFIG)
  • OPCUA_ENDPOINT_URL: Target OPC UA server URL
  • TSTORE_API_URL: Destination for data forwarding
  • Security settings: security_mode, security_policy, user identity configuration

Configuration Updates

The application supports live configuration updates through settings.wait_for_config_update() mechanism that triggers resource restarts when configuration changes.

Deployment

Kubernetes (Helm)

# Deploy using Helm chart
helm install extractor-opcua ./helm/

# The helm chart includes:
# - Deployment with configurable replicas
# - Service for API access
# - ConfigMap for configuration management

Environment Variables for Deployment

Key environment variables for container deployment:

  • OPCUA_ENDPOINT_URL, TSTORE_API_URL, OPERATION_MODE
  • EXTRACTOR_PREFIX, SOURCE_TZ, LOG_LEVEL
  • TStore configuration: TSTORE_FLUSH_SIZE, TSTORE_FLUSH_INTERVAL_SECONDS

Error Handling & Monitoring

Built-in Resilience

  • Connection Management: Automatic reconnection with exponential backoff
  • Subscription Monitoring: Error threshold tracking with forced reconnection
  • Health Checks: Docker health check endpoint at /health

Logging

  • Uses Transpara's tlogging framework
  • Configurable log levels via LOG_LEVEL environment variable
  • Structured error tracking with counters and state information

Monitoring Endpoints

  • Health Check: GET /health - Docker health check endpoint
  • Connection Test: POST /test-connection - Validate OPC UA connectivity
  • Item Validation: POST /validate-items - Test monitored item configuration

Development Notes

Dependencies

  • Core: fastapi, asyncua, pydantic-settings, transpara-sdk
  • Runtime: uvicorn for ASGI server

Code Organization

  • app/core/: Configuration and operation mode management
  • app/api/: FastAPI routes and request/response schemas
  • app/schemas.py: Pydantic models for API and configuration
  • app/opcua_client.py: Main OPC UA client implementation with enumeration handling

Testing Strategy

  • Integration tests via REST API endpoints
  • Configuration validation through /validate-items endpoint
  • Connection testing via /test-connection endpoint
  • Health monitoring through Docker healthcheck

Release Process

Uses semantic-release for automated versioning:

  • Conventional Commits: feat: (minor), fix: (patch), breaking changes (major)
  • Automated: GitHub Actions workflow on main branch pushes
  • Docker Images: Tagged and pushed to transpara/extractor-opcua