Merge remote-tracking branch 'origin/claude/expand-variable-bounds-PkM2W' into test/variadic-max-min-operators-test

Author: GuillaumeMaistre

.github/copilot-instructions.md

@@ -0,0 +1 @@
+See [AGENTS.md](../AGENTS.md) for project overview, architecture, commands, conventions, and agent guardrails.

AGENTS.md

@@ -0,0 +1,93 @@
+# AGENTS.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+**GemsPy** is a Python interpreter for the GEMS (Generic Energy Modeling System) framework — a high-level modeling language for simulating energy systems under uncertainty. It allows users to define energy system models via YAML without writing solver code directly.
+
+## Commands
+
+**Install:**
+```bash
+pip install -r requirements.txt -r requirements-dev.txt
+```
+
+**Test:**
+```bash
+pytest                                          # run all tests
+pytest tests/path/to/test_file.py::test_name   # run a single test
+pytest --cov gems --cov-report xml         # with coverage
+```
+
+**Lint & Format:**
+```bash
+black --config pyproject.toml src/ tests/
+isort --profile black --filter-files src/ tests/
+mypy
+```
+
+**Running:**
+```bash
+# CLI entry point
+gemspy \
+  --model-libs  path/to/lib1.yml path/to/lib2.yml \
+  --components   path/to/components.yml \
+  --timeseries   path/to/timeseries/ \
+  --duration     8760 \
+  --scenarios    1
+
+# Python API (minimal example)
+from gems.main.main import build_problem
+```
+
+## Architecture
+
+The pipeline flows: **YAML input → parsing → model resolution → network building → optimization problem → OR-Tools solver → results**
+
+### Core Modules (`src/gems/`)
+
+**`model/`** — Immutable model templates.
+- `Model`: defines component behavior (parameters, variables, constraints, ports)
+- `Library`: a collection of models, loaded from YAML
+- Models are never instantiated directly — they are referenced by components
+
+**`expression/`** — Mathematical expression language and AST.
+- `ExpressionNode`: base frozen dataclass for all expression tree nodes
+- Node types cover: arithmetic (`+`, `-`, `*`, `/`), comparisons (`<=`, `>=`, `==`), time/scenario operators (`time_sum()`), and functions (`max()`, `min()`, `ceil()`, `floor()`)
+- Grammar is defined in `/grammar/` and parsed via ANTLR4 (generated files live in `expression/parsing/antlr/`)
+- `ExpressionVisitor` is the dominant pattern for traversing and transforming expression trees (evaluation, linearization, printing, degree analysis)
+- Expressions support operator overloading: `var('x') + 5 * param('p')`
+
+**`study/`** — Study definition and network instantiation.
+- `System`: top-level structure parsed from YAML (before instantiation)
+- `Network`: instantiated graph of `Node`s, `Component`s, and connections
+- `Component`: an instance of a `Model` with concrete parameter values
+- `DataBase`: manages time series and scenario data
+
+**`simulation/`** — Optimization problem construction and solving.
+- `OptimizationProblem`: main interface; translates network + database into OR-Tools constraints
+- `LinearExpression`: the linearized form of model constraints used by the solver
+- `BendersDecomposedProblem`: temporal decomposition strategy for large problems
+- `TimeBlock`: structure for defining temporal decomposition
+- `OutputValues`: result extraction and formatting
+
+### Key Design Patterns
+
+- **Frozen dataclasses** throughout for immutability (models, expressions, constraints)
+- **Visitor pattern** for all expression tree operations (`ExpressionVisitor` subclasses)
+- **Indexing dimensions**: parameters and variables carry time and scenario indices explicitly; expressions track these automatically
+- **`ValueType`** enum (`INTEGER`, `CONTINUOUS`, `BOOLEAN`) for variable typing
+
+### Type Checking
+
+Strict mypy is enforced (`disallow_untyped_defs`, `disallow_untyped_calls`). All new code must be fully typed. Configuration is in `mypy.ini`.
+
+## Further Reading
+
+- [Python Convention](docs/agents/python-convention.md) — Code style, conventions, and agent guardrails
+- [Testing](docs/agents/testing.md) — Testing strategy and layer overview
+- [docs/getting-started.md](docs/getting-started.md) — Installation and first study walkthrough
+- [docs/user-guide.md](docs/user-guide.md) — Full user documentation
+- [docs/developer-guide.md](docs/developer-guide.md) — Contributor guide
+- [grammar/](grammar/) — ANTLR4 grammar source (`Expr.g4`)

docs/agents/python-convention.md

@@ -0,0 +1,37 @@
+# Python conventions
+
+## Code Style & Conventions
+
+- **Formatter**: Black, line-length 88. Never adjust line breaks manually—let Black decide.
+- **Import order**: isort with `profile = "black"`. One `import` block, no manual blank lines between
+  groups.
+- **Type annotations**: All functions and methods **must** have full type annotations. mypy is run in
+  strict mode (`disallow_untyped_defs`, `disallow_untyped_calls`).
+- **Dataclasses**: Prefer `@dataclass(frozen=True)` for value objects (expression nodes, model
+  definitions). Mutability must be justified explicitly.
+- **Pydantic**: Use `ConfigDict(alias_generator=to_camel)` or kebab-case alias generation for YAML
+  round-tripping; use Pydantic v2 APIs only.
+- **Naming**:
+  - Classes: `PascalCase`
+  - Functions / variables: `snake_case`
+  - Constants / type-level aliases: `UPPER_SNAKE_CASE`
+  - YAML keys: `kebab-case`
+- **Commit messages**: Follow Conventional Commits — `<type>(<scope>): <summary>`, e.g.
+  `feat(expression): add floor operator`. Types used in this repo: `feat`, `fix`, `docs`, `refactor`,
+  `test`, `chore`, `release`.
+
+---
+
+## Agent Guardrails
+
+Rules for automated agents (CI bots, AI coding assistants, Dependabot, etc.):
+
+1. **Never auto-merge** to `main`; all changes require at least one human review.
+2. **Do not edit generated files** under `src/gems/expression/parsing/antlr/`; regenerate them
+   from `grammar/Expr.g4` instead.
+3. **Do not modify `pyproject.toml` version** manually; version bumps are handled via the
+   `feat(release):` commit workflow.
+4. **Keep pre-commit hooks passing**: any commit that breaks `black`, `isort`, or `mypy` must not
+   be auto-pushed.
+5. **Test coverage must not decrease** on `main`; PRs that drop coverage without justification
+   should be flagged.

docs/agents/testing.md

@@ -0,0 +1,15 @@
+# Testing Strategy
+
+## Philosophy
+
+Tests live alongside the module they exercise. Fixtures (YAML snippets, small networks) are kept
+in `tests/` sub-directories. No mocking of the solver—tests use real OR-Tools calls.
+
+## Layers
+
+| Layer | Location | Description |
+|---|---|---|
+| Unit | `tests/unittests/` | One file per module area; covers AST visitors, parsing, model loading, linearisation |
+| Integration | `tests/unittests/simulation/` | Full problem build + solve on tiny networks |
+| End-to-end | `tests/e2e/` | CLI-level tests reading real YAML fixtures |
+| Converter | `tests/input_converter/`, `tests/pypsa_converter/`, `tests/antares_historic/` | Format-specific conversion tests |

src/gems/expression/copy.py

@@ -14,6 +14,7 @@
 from typing import List, cast
 
 from .expression import (
+    AdditionNode,
     AllTimeSumNode,
     CeilNode,
     ComparisonNode,
@@ -44,6 +45,9 @@ class CopyVisitor(ExpressionVisitorOperations[ExpressionNode]):
     Simply copies the whole AST.
     """
 
+    def addition(self, node: AdditionNode) -> ExpressionNode:
+        return AdditionNode([visit(o, self) for o in node.operands])
+
     def literal(self, node: LiteralNode) -> ExpressionNode:
         return LiteralNode(node.value)
 

src/gems/simulation/optimization.py

@@ -680,10 +680,16 @@ def _create_variables(self) -> None:
                     instantiated_lb_expr = _instantiate_model_expression(
                         model_var.lower_bound, component.id, self.context
                     )
+                    instantiated_lb_expr = self.context.expand_operators(
+                        instantiated_lb_expr
+                    )
                 if model_var.upper_bound:
                     instantiated_ub_expr = _instantiate_model_expression(
                         model_var.upper_bound, component.id, self.context
                     )
+                    instantiated_ub_expr = self.context.expand_operators(
+                        instantiated_ub_expr
+                    )
 
                 time_indices: Iterable[Optional[int]] = [None]
                 if var_indexing.is_time_varying():
@@ -697,13 +703,23 @@ def _create_variables(self) -> None:
                     lower_bound = -self.solver.infinity()
                     upper_bound = self.solver.infinity()
                     if instantiated_lb_expr:
-                        lower_bound = _compute_expression_value(
-                            instantiated_lb_expr, self.context, t, s
+                        lb_linear = self.context.linearize_expression(
+                            instantiated_lb_expr, t, s
                         )
+                        if lb_linear.terms:
+                            raise ValueError(
+                                f"Lower bound of variable '{model_var.name}' must be a constant expression (no decision variables)."
+                            )
+                        lower_bound = lb_linear.constant
                     if instantiated_ub_expr:
-                        upper_bound = _compute_expression_value(
-                            instantiated_ub_expr, self.context, t, s
+                        ub_linear = self.context.linearize_expression(
+                            instantiated_ub_expr, t, s
                         )
+                        if ub_linear.terms:
+                            raise ValueError(
+                                f"Upper bound of variable '{model_var.name}' must be a constant expression (no decision variables)."
+                            )
+                        upper_bound = ub_linear.constant
 
                     solver_var_name = self._solver_variable_name(
                         component.id, model_var.name, t, s

tests/unittests/expressions/visitor/test_copy.py

@@ -11,6 +11,8 @@
 # This file is part of the Antares project.
 
 
+import time
+
 from gems.expression import (
     AdditionNode,
     DivisionNode,
@@ -29,6 +31,44 @@
 )
 
 
+def test_copy_large_addition_is_linear() -> None:
+    """
+    Copying an AdditionNode with T operands must be O(T), not O(T²).
+
+    Before the fix, CopyVisitor inherited the default addition() from
+    ExpressionVisitorOperations which accumulated results with `res = res + o`.
+    Each call to __add__ flattens the AdditionNode by copying the accumulated
+    operand list, giving 1+2+...+(T-1) = O(T²) list copies in total.
+
+    The fix overrides addition() in CopyVisitor to build AdditionNode directly
+    from a list comprehension, reducing the cost to O(T).
+
+    We verify linearity by checking that the time ratio between T=10_000 and
+    T=1_000 stays below 20 (linear ≈ 10, quadratic ≈ 100).
+    """
+    small_n = 1_000
+    large_n = 10_000
+
+    small_node = AdditionNode([VariableNode(f"x{i}") for i in range(small_n)])
+    large_node = AdditionNode([VariableNode(f"x{i}") for i in range(large_n)])
+
+    t0 = time.perf_counter()
+    copy_expression(small_node)
+    small_time = time.perf_counter() - t0
+
+    t0 = time.perf_counter()
+    copy_expression(large_node)
+    large_time = time.perf_counter() - t0
+
+    ratio = large_time / small_time
+    assert ratio < 20, (
+        f"copy_expression scaling looks super-linear: "
+        f"T={small_n} took {small_time:.4f}s, "
+        f"T={large_n} took {large_time:.4f}s, "
+        f"ratio={ratio:.1f} (expected <20 for O(T))"
+    )
+
+
 def test_copy_ast() -> None:
     ast = AllTimeSumNode(
         DivisionNode(