Skip to content

Add structured logging and CloudWatch EMF metrics#20

Open
nandanadileep wants to merge 1 commit into
mainfrom
feat/structured-logging
Open

Add structured logging and CloudWatch EMF metrics#20
nandanadileep wants to merge 1 commit into
mainfrom
feat/structured-logging

Conversation

@nandanadileep

Copy link
Copy Markdown
Owner

Closes #6

Summary

  • Every Lambda invocation now emits two log lines: a plain structured JSON entry (queryable in Logs Insights) and a CloudWatch Embedded Metrics Format line (auto-extracted as custom metrics — no API call, no added latency)
  • Metrics published under the AlexaLLM namespace with Intent as the dimension: Invocations, Errors, Latency
  • cloudwatch_dashboard.json provides a ready-to-import dashboard with 4 widgets

Files changed

File Change
logger.py log_invocation() + emit_metrics() using EMF
lambda_function.py Handler timed; both logger functions called in finally block
cloudwatch_dashboard.json 4-widget dashboard: invocations by intent, avg latency, error rate %, P99 latency
test_lambda.py TestLogger (13 cases) covering JSON structure, EMF format, error flag, and handler integration

One-time setup

Import the dashboard:

aws cloudwatch put-dashboard \
  --dashboard-name AlexaLLM \
  --dashboard-body file://cloudwatch_dashboard.json

Logs Insights query to see all invocations:

fields intent, userId, latency_ms, error
| filter event = "invocation"
| sort @timestamp desc

Test plan

  • Invoke the skill — verify invocation JSON log appears in CloudWatch Logs
  • Verify AlexaLLM namespace appears in CloudWatch Metrics within ~5 min
  • Import dashboard JSON and confirm all 4 widgets populate
  • Force an LLM error — verify Errors metric increments
  • 192 unit tests pass

logger.py (new):
- log_invocation() emits a structured JSON line per request with intent,
  userId, latency_ms, and error for CloudWatch Logs Insights queries
- emit_metrics() emits a CloudWatch Embedded Metrics Format line that
  CloudWatch extracts as custom metrics (Invocations, Errors, Latency)
  under the AlexaLLM namespace with Intent as dimension — no boto3 API
  call, no added latency

lambda_function.py:
- lambda_handler timed with time.time(); log_invocation and emit_metrics
  called in a finally block so logs fire even on unhandled exceptions
- intent_name resolved before routing so all request types are captured

cloudwatch_dashboard.json (new):
- Four widgets: invocations by intent, average latency by intent,
  error rate (%), P99 latency for AskClaudeIntent
- Import via: aws cloudwatch put-dashboard --dashboard-name AlexaLLM
    --dashboard-body file://cloudwatch_dashboard.json

test_lambda.py:
- TestLogger: 10 cases covering log_invocation field correctness,
  emit_metrics EMF structure, error flag, latency, and namespace
- 3 handler-level tests asserting log_invocation and emit_metrics are
  called with the correct intent name

Closes #6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add structured logging and usage tracking

1 participant