Skip to content

Add independent log parser fixtures and validation (+ fix 2 parser bugs)#66

Open
zhaojiazheng9-art wants to merge 8 commits into
weilixiong:mainfrom
zhaojiazheng9-art:main
Open

Add independent log parser fixtures and validation (+ fix 2 parser bugs)#66
zhaojiazheng9-art wants to merge 8 commits into
weilixiong:mainfrom
zhaojiazheng9-art:main

Conversation

@zhaojiazheng9-art

Copy link
Copy Markdown

Summary

Add hand-written log parser fixtures for JSON, text, and nginx formats to eliminate the 40% false-pass rate caused by parser-generated test data. Includes a dedicated validation script and fixes two parser bugs exposed by the new fixtures.

Changes

  • tools/tests/fixtures/: New directory with hand-written fixture files
    • json_logs.fixture — 7 JSON log entries with standard and alternative key formats
    • text_logs.fixture — 6 plain text log entries with timestamps and [bracket] services
    • nginx_logs.fixture — 6 Nginx combined-format access log entries
  • tools/tests/validate_parsers.py: New validation script that:
    • Tests timestamp, level, service, and key field extraction per format
    • Includes 8 malformed/unsupported line cases that must not crash parsing
    • Validates JSON key aliases (time→timestamp, severity→level, logger→service, etc.)
  • tools/log_aggregator.py: Bug fixes
    • extract_service(): regex [\w+][\w-]+ to handle hyphenated service names
    • NginxLogParser: remote_user now reads group 3 instead of group 2 (was reading ident field)
  • build.py: Fixed SyntaxError from backslash in f-string expression
  • diagnostic/: Updated build diagnostic artifacts

Testing

$ python3 tools/tests/validate_parsers.py
==================================================
  Log Parser Validation
==================================================
  JSON parser (7 lines)     — 7 passed
  Text parser (6 lines)     — 6 passed
  Nginx parser (6 lines)    — 6 passed
  Malformed lines (8 cases) — 0 crashes
==================================================
  Results: 19 passed, 0 failed, 0 crashes
==================================================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant