Problem
Custom logs are hard to onboard safely. Analysts and admins need a workbench that can inspect sample logs, infer a draft mapping, test parser output, and show rejected records before a source is enabled.
Scope
Build a local schema/parser workbench for sample JSON, JSONL, CSV-with-header, syslog/CEF, and key-value logs. The first version can infer field paths and propose a mapping to Tawny-SOC canonical event shape.
Implementation Notes
- Add /settings/ingestion or extend the existing ingestion settings section.
- Inputs: paste sample logs, upload sample file if supported by current stack, choose stream type or auto-detect.
- Outputs: detected format, inferred field list, proposed canonical mappings, parser result preview, rejected record reasons, canonical indicators extracted.
- Add a server-side parser test endpoint that reuses lib/ingestion/normalizers.ts.
- Store draft parser configs but require explicit enablement before accepting source traffic.
Acceptance Criteria
- A user can paste sample logs and preview normalized Tawny-SOC events.
- Rejected records include actionable reasons.
- The workbench shows which canonical indicators would be extracted.
- Draft mappings are editable before saving.
- Saved parser configs are disabled until explicitly enabled.
Tests
- Unit test format detection for supported sample types.
- Unit test parser preview returns accepted and rejected samples.
- API test invalid sample input is handled without throwing.
- Component test preview renders normalized fields and rejection reasons.
Problem
Custom logs are hard to onboard safely. Analysts and admins need a workbench that can inspect sample logs, infer a draft mapping, test parser output, and show rejected records before a source is enabled.
Scope
Build a local schema/parser workbench for sample JSON, JSONL, CSV-with-header, syslog/CEF, and key-value logs. The first version can infer field paths and propose a mapping to Tawny-SOC canonical event shape.
Implementation Notes
Acceptance Criteria
Tests