Skip to content

Replace print statements with structured logging in execute_graph_generation #5

@chigwell

Description

@chigwell

User Story
As a software maintainer,
I want structured logging implemented in the execute_graph_generation function
so that log data integrates with monitoring systems and output verbosity can be programmatically controlled.

Background
The current implementation uses print statements (e.g., print(f"Splitting text into...") and print(f"Processing chunk...") in eknowledge/main.py) for progress reporting. This creates three key issues:

  1. Monitoring limitations: Print outputs can't be easily parsed by tools like Prometheus/Grafana
  2. Verbosity control: No programmatic way to filter debug vs. operational logs
  3. Production unsuitability: Print statements lack metadata (timestamps, severity levels) needed for diagnostics

The primary focus is modifying execute_graph_generation in eknowledge/main.py, though other modules using similar patterns may need future updates.

Acceptance Criteria

  • Replace all print statements in execute_graph_generation with structured logging using Python's logging or structlog
  • Implement log severity levels (DEBUG for chunk processing details, INFO for major milestones, ERROR for retries)
  • Ensure logs output machine-readable format (JSON) when not in interactive mode
  • Preserve existing verbose flag behavior by:
    • Setting log level to DEBUG when verbose=True
    • Setting log level to WARNING when verbose=False
  • Add log context:
    • Chunk number/total in DEBUG messages
    • LLM retry attempts in WARNING messages
    • Node count per chunk in INFO messages

Validation

  1. Run execute_graph_generation with verbose=True and verify DEBUG logs appear in structured format
  2. Test with verbose=False and confirm only WARNING/INFO logs appear
  3. Pipe output to jq (or similar) to validate JSON structure
  4. Verify no print statements remain in main.py via code review

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions