Skip to content

feat: Kafka Consumer Lag Monitoring and Auto-Scaling Consumer Groups (#109)#138

Open
Mona-i wants to merge 1 commit into
VeriNode-Labs:mainfrom
Mona-i:feat/kafka-consumer-lag-monitoring-issue-109
Open

feat: Kafka Consumer Lag Monitoring and Auto-Scaling Consumer Groups (#109)#138
Mona-i wants to merge 1 commit into
VeriNode-Labs:mainfrom
Mona-i:feat/kafka-consumer-lag-monitoring-issue-109

Conversation

@Mona-i

@Mona-i Mona-i commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #109

Implements Kafka consumer lag monitoring and auto-scaling consumer groups as requested in issue #109.

Changes

New Files (9 files, 1364 insertions)

File Purpose
\src/types/kafka.ts\ TypeScript types: \PartitionLag, \ConsumerGroupLag, \ScalingEvent, \ConsumerGroupScalingConfig, \ScalingStatus, \KafkaMonitoringState\
\src/store/kafkaSlice.ts\ Zustand store slice — upsert/set groups, scaling status, event ring-buffer (max 200), markRefreshed, setError, reset
\src/services/kafkaMonitoringService.ts\ Core logic: lag threshold derivation, auto-scaling decision engine (\computeTargetInstances), \�uildScalingEvent, demo data generator, dual demo/production \KafkaMonitoringProvider\ factory
\src/hooks/useKafkaMonitoring.ts\ Polling hook (default 15 s) — parallel fetch of lag + scaling statuses, client-side auto-scaling evaluation, \ riggerManualScale, \
efresh\
\src/components/kafka/LagStatusBadge.tsx\ Accessible colour-coded status pill (healthy / warning / critical) with animated pulse
\src/components/kafka/ConsumerGroupLagCard.tsx\ Per-group card — expandable partition table, summary stats, manual scale input, last-event footer
\src/components/kafka/ScalingHistoryTable.tsx\ Compact table of recent auto-scaling events
\src/components/kafka/KafkaMonitoringDashboard.tsx\ Top-level dashboard composing all Kafka components
\src/tests/kafkaMonitoring.test.ts\ 32 unit tests covering all core logic, store mutations, demo service, and a performance guard

Testing

  • 32/32 unit tests pass (
    px vitest run src/tests/kafkaMonitoring.test.ts)
  • Build passes — \✓ Compiled successfully, \✓ Finished TypeScript, \✓ Generating static pages (11/11)\
  • Zero type errors across all 9 new files (verified via diagnostics)
  • Performance guard — 10 000 auto-scaling evaluations complete in < 100 ms (P99 target met)

Architecture

  • Demo-first pattern (matches existing codebase) — zero network calls in demo mode, real HTTP calls in production
  • Zustand store with 200-event ring-buffer for scaling history
  • Polling interval configurable per hook instance (default 15 s)
  • All components are accessible (role, aria-label, aria-expanded on interactive elements)

…eriNode-Labs#109)

Implements Kafka consumer lag monitoring and consumer group auto-scaling per issue VeriNode-Labs#109. closes VeriNode-Labs#109
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Kafka Consumer Lag Monitoring and Auto-Scaling Consumer Groups

1 participant