Skip to content

Commit ff49767

Browse files
evansenterclaude
andauthored
Refactor codebase per RFC #18 audit findings (#19)
* Add Phase 1: Project setup and FastMCP server skeleton - pyproject.toml with FastMCP, uvicorn, and dev dependencies - Makefile with check, fmt, lint, test, install, uninstall targets - LaunchAgent plist and install/uninstall scripts for auto-start - dev.sh script for development mode with auto-reload - Basic FastMCP server with placeholder tools: - get_status: Returns server status - ingest_logs: Placeholder for log ingestion - query_tool_frequency: Placeholder for frequency queries - Usage guide as MCP resource at session-analytics://guide - Tests for the placeholder tools - README with installation and usage instructions Server runs on port 8081 (to not conflict with event-bus on 8080). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Phase 2: SQLite storage layer - storage.py with SQLiteStorage class: - Events table with denormalized fields for fast queries - Sessions table for session metadata - Ingestion state tracking for incremental updates - Patterns table for pre-computed insights - Indexes on timestamp, session_id, tool_name, project_path - Data classes: Event, Session, IngestionState, Pattern - CRUD operations for all entities with batch insert support - get_db_stats() for monitoring database health - Updated server.py to use storage for get_status() - Comprehensive test suite (16 tests) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Phase 3: JSONL ingestion module Implements log file discovery and parsing: - find_log_files(): Discovers JSONL files within date range - parse_tool_use(): Extracts tool info (command, file_path, skill_name) - parse_entry(): Parses entries into Event objects - ingest_file(): Incremental ingestion with mtime/size tracking - ingest_logs(): Full ingestion orchestration - update_session_stats(): Aggregates session statistics Integrates with server.py to provide real data for ingest_logs tool. Closes #3 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Phase 4: Query tools implementation Implements all query MCP tools: - query_tool_frequency: Tool usage counts with project filter - query_timeline: Events in time window with filtering - query_commands: Bash command breakdown with prefix filter - query_sessions: Session metadata and token totals - query_tokens: Token usage grouped by day/session/model Also adds: - ensure_fresh_data(): Auto-refresh mechanism (5 min staleness) - Comprehensive tests for all queries (18 new tests) Closes #4 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Phase 5: Pattern detection and insights Implements pattern detection for /improve-workflow integration: - compute_tool_frequency_patterns(): Tool usage frequency - compute_command_patterns(): Bash command frequency - compute_sequence_patterns(): Tool n-gram detection - compute_permission_gaps(): Commands not in settings.json - get_insights(): Unified insights API for /improve-workflow New MCP tools: - query_sequences: Common tool patterns - query_permission_gaps: Commands needing settings.json - get_insights: Pre-computed patterns Adds 16 new tests (69 total). Closes #5 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Phase 6: CLI and documentation Adds command-line interface for shell access: - session-analytics-cli status: Database stats - session-analytics-cli ingest: Trigger log ingestion - session-analytics-cli frequency: Tool usage counts - session-analytics-cli commands: Bash command breakdown - session-analytics-cli sessions: Session metadata - session-analytics-cli tokens: Token usage by day/session/model - session-analytics-cli sequences: Tool patterns - session-analytics-cli permissions: Commands needing settings.json - session-analytics-cli insights: Pre-computed patterns All commands support --json for machine-readable output. Also updates README with CLI usage documentation. Closes #6 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Add Phase 7: Polish and CLI tests - Add comprehensive CLI test coverage (15 tests) - Fix format_output ordering for insights command - Test all CLI commands: status, ingest, frequency, commands, sessions, tokens, sequences, permissions, insights - Test both human-readable and JSON output modes Closes #7 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Update CLAUDE.md and README.md with comprehensive docs - Document all 10 MCP tools with descriptions - Add CLI usage examples for all 9 commands - Include example JSON output for key queries - Document architecture, data model, and integration points - Add development and installation instructions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Refactor codebase per RFC #18 audit findings Implements all 12 findings from the codebase audit: **P1 - Encapsulation:** - Add execute_query() and execute_write() public methods to SQLiteStorage - Migrate 8 locations (4 in queries.py, 4 in patterns.py) to use public API **P2 - Code Quality:** - Add build_where_clause() helper to reduce query duplication - Refactor format_output() to use formatter registry pattern - Change to module-qualified imports in server.py - Read version from importlib.metadata instead of hardcoding **P3 - Future Extensibility:** - Add schema migration framework with @migration decorator - Define __all__ exports in __init__.py **P4 - Polish:** - Remove empty pass branch in ingest.py - Document timestamp handling with clear comments - Add CLI epilog with examples - Update CLAUDE.md and README.md with architecture patterns Closes #18 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Ignore .claude/ directory with local settings --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 337c1e5 commit ff49767

25 files changed

Lines changed: 4697 additions & 58 deletions

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,3 +38,4 @@ htmlcov/
3838

3939
# Project-specific
4040
*.db
41+
.claude/

‎CLAUDE.md‎

Lines changed: 82 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,59 +1,114 @@
11
# CLAUDE.md
22

3-
Queryable analytics for Claude Code session logs, exposed as an MCP server.
3+
Queryable analytics for Claude Code session logs, exposed as an MCP server and CLI.
44

55
## Project Overview
66

77
This MCP server replaces the bash script `~/.claude/contrib/parse-session-logs.sh` with a persistent, queryable analytics layer. It parses JSONL session logs from `~/.claude/projects/` and provides:
88

9-
- **User-centric timeline**: Events across conversations, organized by timestamp
10-
- **Rich querying**: Tool frequency, command breakdown, sequences, permission gaps
11-
- **Persistent storage**: SQLite at `~/.claude/contrib/analytics/data.db`
12-
- **Auto-refresh**: Queries automatically refresh stale data (>5 min old)
13-
- **CLI access**: Full CLI for shell scripts and hooks
9+
- **Tool frequency analysis**: Which tools you use most (Read, Edit, Bash, etc.)
10+
- **Command breakdown**: Bash command patterns (git, make, cargo, etc.)
11+
- **Workflow sequences**: Common tool chains like Read → Edit → Bash
12+
- **Permission gap detection**: Commands that should be added to settings.json
13+
- **Token usage tracking**: Usage by day, session, or model
14+
- **Session timeline**: Events across conversations, organized by timestamp
1415

1516
## Architecture
1617

17-
Follows the `claude-event-bus` pattern:
18-
- FastMCP for MCP server implementation
19-
- SQLite for persistence
20-
- LaunchAgent for always-on availability
21-
- CLI wrapper for shell access
18+
```
19+
~/.claude/projects/**/*.jsonl → SQLite DB → MCP Server / CLI
20+
↓
21+
~/.claude/contrib/analytics/data.db
22+
```
23+
24+
Key components:
25+
- **FastMCP** for MCP server implementation
26+
- **SQLite** for persistent storage with incremental ingestion
27+
- **Auto-refresh** queries automatically refresh stale data (>5 min old)
28+
- **LaunchAgent** for always-on availability (macOS)
2229

2330
## Commands
2431

2532
```bash
26-
make check # Run fmt, lint, test
33+
make check # Run fmt, lint, test (84 tests)
2734
make install # Install LaunchAgent + CLI
2835
make uninstall # Remove LaunchAgent + CLI
2936
make dev # Run in dev mode with auto-reload
3037
```
3138

3239
## Key Files
3340

34-
- `src/session_analytics/server.py` - MCP tools + entry point
35-
- `src/session_analytics/storage.py` - SQLite backend
36-
- `src/session_analytics/ingest.py` - JSONL parsing
37-
- `src/session_analytics/queries.py` - Query implementations
38-
- `src/session_analytics/patterns.py` - Pattern detection
41+
| File | Purpose |
42+
|------|---------|
43+
| `src/session_analytics/server.py` | MCP tools + HTTP server entry point |
44+
| `src/session_analytics/cli.py` | CLI with formatter registry for output |
45+
| `src/session_analytics/storage.py` | SQLite backend with migration support |
46+
| `src/session_analytics/ingest.py` | JSONL parsing with incremental updates |
47+
| `src/session_analytics/queries.py` | Query implementations with `build_where_clause()` helper |
48+
| `src/session_analytics/patterns.py` | Pattern detection (sequences, permission gaps) |
49+
50+
## Architecture Patterns
51+
52+
- **Public API**: Use `storage.execute_query()` / `execute_write()` for raw SQL; avoid `_connect()`
53+
- **Formatter Registry**: CLI uses `@_register_formatter(predicate)` decorator pattern
54+
- **Schema Migrations**: Use `@migration(version, name)` decorator in storage.py for DB changes
55+
- **Module Imports**: server.py uses `from session_analytics import queries, patterns, ingest`
3956

4057
## MCP Tools
4158

4259
| Tool | Purpose |
4360
|------|---------|
61+
| `get_status` | Database stats and last ingestion time |
4462
| `ingest_logs` | Refresh data from JSONL files |
45-
| `query_timeline` | Events in time window |
46-
| `query_tool_frequency` | Tool usage counts |
47-
| `query_commands` | Bash command breakdown |
48-
| `query_sequences` | Common tool patterns |
49-
| `query_permission_gaps` | Commands needing settings.json |
50-
| `query_sessions` | Session metadata |
51-
| `query_tokens` | Token usage analysis |
63+
| `query_tool_frequency` | Tool usage counts (Read, Edit, Bash, etc.) |
64+
| `query_timeline` | Events in time window with filtering |
65+
| `query_commands` | Bash command breakdown with prefix filter |
66+
| `query_sessions` | Session metadata and token totals |
67+
| `query_tokens` | Token usage by day, session, or model |
68+
| `query_sequences` | Common tool patterns (n-grams) |
69+
| `query_permission_gaps` | Commands needing settings.json entries |
5270
| `get_insights` | Pre-computed patterns for /improve-workflow |
53-
| `get_status` | Ingestion status + DB stats |
71+
72+
## CLI Commands
73+
74+
All commands support `--json` for machine-readable output:
75+
76+
```bash
77+
session-analytics-cli status # DB stats
78+
session-analytics-cli ingest --days 30 # Refresh data
79+
session-analytics-cli frequency # Tool usage
80+
session-analytics-cli commands --prefix git # Command breakdown
81+
session-analytics-cli sessions # Session info
82+
session-analytics-cli tokens --by model # Token usage
83+
session-analytics-cli sequences # Tool chains
84+
session-analytics-cli permissions # Permission gaps
85+
session-analytics-cli insights # For /improve-workflow
86+
```
87+
88+
## Integration
89+
90+
### With /improve-workflow
91+
92+
The `get_insights` tool (or `session-analytics-cli insights`) provides pre-computed patterns:
93+
- Tool frequency for identifying high-value automations
94+
- Command frequency for settings.json additions
95+
- Tool sequences for workflow optimization
96+
- Permission gaps with ready-to-use suggestions
97+
98+
### With session-start hook
99+
100+
Can be used to auto-ingest on session start:
101+
```bash
102+
session-analytics-cli ingest --days 1 --json 2>/dev/null || true
103+
```
104+
105+
## Data Model
106+
107+
**Events table**: Individual tool uses with timestamps, tokens, commands
108+
**Sessions table**: Aggregated session metadata
109+
**Patterns table**: Pre-computed patterns for fast querying
110+
**Ingested files table**: Tracks file mtime/size for incremental updates
54111

55112
## Reference
56113

57114
Full implementation plan: `~/.claude/plans/precious-crunching-crescent.md`
58-
59-
Reference implementation: `~/Documents/projects/claude-event-bus/`

‎Makefile‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
.PHONY: check fmt lint test clean install uninstall dev venv
2+
3+
# Run all quality gates (format check, lint, tests)
4+
check: fmt lint test
5+
6+
# Check/fix formatting with ruff
7+
fmt:
8+
ruff format --check .
9+
10+
# Run linter with ruff
11+
lint:
12+
ruff check .
13+
14+
# Run tests
15+
test:
16+
pytest tests/ -v
17+
18+
# Clean build artifacts
19+
clean:
20+
rm -rf build/ dist/ *.egg-info .pytest_cache .ruff_cache
21+
find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
22+
23+
# Create virtual environment (requires Python 3.10+)
24+
venv:
25+
@if [ ! -d .venv ]; then \
26+
echo "Creating virtual environment..."; \
27+
PYTHON=$$(command -v python3.12 || command -v python3.11 || command -v python3.10 || echo "python3"); \
28+
$$PYTHON -m venv .venv && .venv/bin/pip install --upgrade pip; \
29+
fi
30+
31+
# Install with dev dependencies (for development)
32+
dev: venv
33+
.venv/bin/pip install -e ".[dev]"
34+
35+
# Full installation: venv + deps + LaunchAgent + CLI + MCP
36+
install: venv
37+
@echo "Installing dependencies..."
38+
.venv/bin/pip install -e .
39+
@echo ""
40+
@echo "Installing LaunchAgent..."
41+
./scripts/install-launchagent.sh
42+
@echo ""
43+
@echo "Adding to Claude Code..."
44+
@CLAUDE_CMD=$$(command -v claude || echo "$$HOME/.local/bin/claude"); \
45+
if [ -x "$$CLAUDE_CMD" ]; then \
46+
$$CLAUDE_CMD mcp add --transport http --scope user session-analytics http://localhost:8081/mcp 2>/dev/null && \
47+
echo "Added session-analytics to Claude Code" || \
48+
echo "session-analytics already configured in Claude Code"; \
49+
else \
50+
echo "Note: claude not found. Run manually:"; \
51+
echo " claude mcp add --transport http --scope user session-analytics http://localhost:8081/mcp"; \
52+
fi
53+
@echo ""
54+
@echo "Installation complete!"
55+
@echo ""
56+
@echo "Make sure ~/.local/bin is in your PATH:"
57+
@echo ' export PATH="$$HOME/.local/bin:$$PATH"'
58+
59+
# Uninstall: LaunchAgent + CLI + MCP config
60+
uninstall:
61+
@echo "Uninstalling..."
62+
./scripts/uninstall-launchagent.sh
63+
@echo ""
64+
@echo "Removing from Claude Code..."
65+
@CLAUDE_CMD=$$(command -v claude || echo "$$HOME/.local/bin/claude"); \
66+
if [ -x "$$CLAUDE_CMD" ]; then \
67+
$$CLAUDE_CMD mcp remove --scope user session-analytics 2>/dev/null && \
68+
echo "Removed session-analytics from Claude Code" || \
69+
echo "session-analytics not found in Claude Code"; \
70+
fi
71+
@echo ""
72+
@echo "Uninstall complete!"
73+
@echo "Note: venv and source code remain in place."

0 commit comments

Comments
 (0)