Add hierarchical clustering: L0 topics → L1 communities → L2 nodes by rongmiao926-hub · Pull Request #297 · Graphify-Labs/graphify

rongmiao926-hub · 2026-04-13T06:12:27Z

Summary

Adds two-pass Leiden hierarchical clustering to organize flat communities into a topic → community → node hierarchy
First pass (unchanged): standard Leiden → L1 communities
Second pass (new): contracts communities into super-nodes, runs Leiden on contracted graph → L0 topics
Fully backward compatible: existing cluster() unchanged, hierarchical_cluster() is additive

Changes

cluster.py: HierarchicalCommunities dataclass + hierarchical_cluster() function + _build_contracted_graph()
export.py: to_json persists hierarchy; to_html adds two-level collapsible legend with topic filtering; to_obsidian generates _TOPIC_*.md overview notes
serve.py: New get_topics and drill_down MCP tools; graph_stats shows topic count; node info includes topic
report.py: "Topic Overview" section with ASCII tree view; community headers show parent topic
analyze.py: cross_topic question type for edges spanning topic boundaries
skill.md: Pipeline updated to use hierarchical_cluster(G) with L0 topic labeling step
README.md: Updated clustering description

Backward compatibility

All 7 existing MCP tools behave identically
cluster() function signature and return value unchanged
New get_topics and drill_down tools are pure additions
If < 4 L1 communities exist, all are grouped into a single topic (graceful degradation)

Test plan

Tested on synthetic 24-node graph: 2 topics, 4 communities correctly produced
Full pipeline test: report, JSON, HTML, cross-topic questions all verified
Existing behavior preserved when hierarchy has only 1 topic

🤖 Generated with Claude Code

* fix: git hooks fail when graphify is installed via pipx When installed via pipx, the graphify module is only available in pipx's isolated venv, not the system python3. The git hooks (post-commit, post-checkout) hardcoded `python3` which cannot import graphify in this case. Detect the correct Python interpreter from the graphify binary's shebang line, matching the approach already used in skill.md Step 1. Falls back to python3 for system installs. * fix: handle env-style shebangs and improve interpreter detection - Use POSIX `command -v` instead of non-standard `which` - Parse `#!/usr/bin/env python3` shebangs correctly (previous `tr -d ' '` would produce `/usr/bin/envpython3`) - Add import validation fallback to python3 if resolved interpreter cannot import graphify

… buffer * fix: suppress graspologic ANSI output that breaks PowerShell scrolling graspologic's leiden() emits ANSI escape sequences (progress bars, colored warnings) that corrupt PowerShell 5.1's scroll buffer on Windows, disabling vertical scrolling. Redirect stdout/stderr to StringIO during leiden() calls to prevent any escape codes from reaching the terminal. Add 2 tests verifying cluster() produces no stdout/stderr output. Fixes Graphify-Labs#19 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add PowerShell troubleshooting section to Windows skill Document the PowerShell 5.1 scrolling issue and provide 4 workarounds: upgrade graphify, use Windows Terminal, reset terminal, or uninstall graspologic to use Louvain fallback. Fixes Graphify-Labs#19 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

- Register 'trae' and 'trae-cn' in _PLATFORM_CONFIG (skill-trae.md, ~/.trae/skills/ and ~/.trae-cn/skills/, claude_md=False) - Add CLI subcommands: graphify trae install/uninstall, graphify trae-cn install/uninstall (routes to _agents_install/uninstall) - Update help text with new platform entries - Create skill-trae.md (Agent-tool based extraction, AGENTS.md integration, no PreToolUse hook support per Trae limitations) - Update README.md and README.zh-CN.md with Trae platform docs Co-authored-by: lijinshuan <lijinshuan@bytedance.com>

…-safe

…rShell fix

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…utput Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ean README, save-result CLI

…y-Labs#109: extension drift, click detection, skill coverage, .graphify_python persistence Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…-Labs#93 Graphify-Labs#102 Graphify-Labs#109) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ify-Labs#126) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…Graphify-Labs#71) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…e relations in innerHTML (#sec) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…y-Labs#127) Tree-sitter resolves call targets directly from source — marking them INFERRED was incorrect. Cross-file class-level uses edges remain INFERRED. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…AST calls Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…Windows git hooks (Graphify-Labs#140) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…sh (Graphify-Labs#137, Graphify-Labs#148, Graphify-Labs#149) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…hify-Labs#105) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…he, graphifyignore parent discovery, MCP fixes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…sor, Gemini in CLI section

…cription Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…port Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…nstall Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…attribute Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…r prompt itself Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ort.py, bound collision loop Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…aceholder Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Graphify-Labs#195: skill.md requires general-purpose subagent type for extraction dispatch Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…phify-Labs#222 and bump to 0.4.2 - extract.py: use str(path) for node IDs to prevent same-basename collision (Graphify-Labs#211) - build.py: normalize from/to edge keys before KeyError (Graphify-Labs#216) - export.py: guard ZeroDivisionError when graph has no edges (Graphify-Labs#217) - hooks.py: remove stale CODE_EXTS filter, rebuild on any changed file (Graphify-Labs#222) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…, Graphify-Labs#221 into 0.4.2 - build/validate: accept NetworkX <=3.1 "links" key alongside "edges" (Graphify-Labs#212) - __main__: skip version check during install/uninstall, deduplicate paths (Graphify-Labs#220) - all file IO: explicit encoding="utf-8" to prevent crashes on Windows CJK locales (Graphify-Labs#204) - hooks: add newline="\n" on write to prevent CRLF shebang breakage on Windows (Graphify-Labs#204) - export: strip trailing .md from safe_name so "CLAUDE.md" doesn't become "CLAUDE.md.md" (Graphify-Labs#221) - report: add Community Hubs navigation block so Obsidian vault stays connected (Graphify-Labs#221) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…phify-Labs#226, Graphify-Labs#254 and bump to 0.4.3 - extract.py: resolve relative JS/TS imports to full-path IDs (fixes 0 import edges on TS codebases) (Graphify-Labs#256) - extract.py: resolve relative Python imports to full-path IDs (Graphify-Labs#256) - watch.py: merge fresh AST with existing semantic nodes instead of overwriting (Graphify-Labs#253) - hooks.py: add python fallback after python3 for Windows; exit 0 if neither found (Graphify-Labs#244) - analyze.py: guard stale _src/_tgt hints with node membership check (Graphify-Labs#226) - detect.py + extract.py: add .vue and .svelte to CODE_EXTENSIONS and _DISPATCH (Graphify-Labs#254) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Two-pass Leiden: first pass produces L1 communities (unchanged from existing behavior), second pass contracts communities into super-nodes and runs Leiden again to produce L0 topics that group related communities into broader themes. Changes: - cluster.py: HierarchicalCommunities dataclass + hierarchical_cluster() - export.py: hierarchy in graph.json, two-level HTML legend, _TOPIC_*.md in Obsidian - serve.py: get_topics and drill_down MCP tools, topic info in node/stats output - report.py: Topic Overview section with tree view, topic labels on communities - analyze.py: cross_topic question type in suggest_questions - skill.md: Steps 4/5/6 updated for hierarchical pipeline - README.md: updated clustering description Fully backward compatible: cluster() signature unchanged, all existing MCP tools work as before, hierarchy is purely additive. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ljinshuan · 2026-04-13T06:44:49Z

the same as #264 ？

Minidoracat and others added 30 commits April 8, 2026 19:39

docs: add graph.json + LLM workflow example to README

312ba0d

docs: add Japanese README

74e192d

fix: skill-droid.md missing from package-data, louvain kwargs version…

b64c231

…-safe

fix: XSS in legend innerHTML and shebang allowlist in hooks

1cc54b3

fix: shebang allowlist validation in all skill files

6a3b4bb

release: 0.3.15 — Trae platform, security hardening, pipx hooks, Powe…

0dc125d

…rShell fix

fix: switch star history chart to starchart.cc (no auth required)

30a66ec

v0.3.16: NetworkX <3.4 compat, .jsx support, pipx follow-up fix

23d88c5

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

v0.3.17: Julia support, smarter chunking, tree-sitter pin, progress o…

dd8235e

…utput Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs: add .jsx .tsx .jl to extensions table, bump to 20 languages

8dbf953

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

add Penpax link to README

1a50f2d

add Korean README and fix trae.com link across all READMEs

ec4f684

release 0.3.18: fix watch .graphifyignore, codex hook, trae link, Kor…

11dff7e

…ean README, save-result CLI

Apply PRs Graphify-Labs#82 Graphify-Labs#93 Graphify-Labs#102 Graphif…

29c639d

…y-Labs#109: extension drift, click detection, skill coverage, .graphify_python persistence Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Update CHANGELOG for v0.3.18 with PR fixes (Graphify-Labs#82 Graphify…

4d8cffe

…-Labs#93 Graphify-Labs#102 Graphify-Labs#109) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix install step: try plain pip before --break-system-packages (Graph…

f140d47

…ify-Labs#126) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Release v0.3.19

096a76f

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add OpenCode tool.execute.before plugin via graphify opencode install (…

3501605

…Graphify-Labs#71) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Update README for OpenCode tool.execute.before plugin

e1864d7

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix XSS in HTML viz: escape node labels, types, source files, and edg…

38255e7

…e relations in innerHTML (#sec) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Pin tree-sitter>=0.23.0, add version guard, confidence=EXTRACTED for …

b101a99

…AST calls Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix Codex hook JSON schema (Graphify-Labs#138) and use #!/bin/sh for …

6f9fc65

…Windows git hooks (Graphify-Labs#140) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add monthly PyPI downloads badge to README

a104a0d

Add Cursor support, fix _rebuild_code KeyError and node_link_data cra…

f770712

…sh (Graphify-Labs#137, Graphify-Labs#148, Graphify-Labs#149) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add Cursor to description, README intro, and PyPI keywords

0b8067c

Add Gemini CLI support and sponsor nudge at pipeline completion (Grap…

dcc402e

…hify-Labs#105) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

safishamsi and others added 28 commits April 9, 2026 22:45

Add Trendshift badge to README

e440539

Remove Trendshift badge (API not available)

cf39034

release 0.3.25: Aider + Copilot CLI, directed graphs, frontmatter cac…

1cbcee5

…he, graphifyignore parent discovery, MCP fixes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix MCP server path validation security issue (0.3.26)

863100c

update README to list all platforms including Aider, Copilot CLI, Cur…

a0a196e

…sor, Gemini in CLI section

fix gemini install missing skill file copy (0.3.27)

55964bc

update README Gemini CLI description to mention skill file copy

af3a3d2

switch downloads badge to pepy.tech to avoid shields.io rate limit

f7ee752

fix hook reinstall, CRLF labels, skill-windows missing commands (0.3.28)

210243f

Add video/audio corpus support with yt-dlp download and Whisper trans…

79acb7e

…cription Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Update README and descriptions to mention video/audio and YouTube sup…

f758911

…port Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add YouTube URL example to README usage, update repo about with pip i…

a2872ca

…nstall Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix CI: mock lazy anthropic import via sys.modules instead of module …

2c21bc0

…attribute Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Remove Anthropic API call from transcribe.py - agent generates Whispe…

699e996

…r prompt itself Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Bump to 0.4.0, set v4 as working branch, update CI badge and skill URL

68bb2bd

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add v4 to CI branch triggers

3ce9bda

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Security: fix YAML injection in ingest.py, </script> injection in exp…

fa388c8

…ort.py, bound collision loop Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add dedicated video and audio corpus section to README

209224d

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Remove YouTube URL references from README, use generic <video-url> pl…

0e45308

…aceholder Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix Graphify-Labs#188: collect_files() now respects .graphifyignore; fix

271ee0a

Graphify-Labs#195: skill.md requires general-purpose subagent type for extraction dispatch Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Bump to 0.4.1

625ca0e

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

add 0.4.2 CHANGELOG entry

b4d4ac8

add LinkedIn badge to README

013313f

update README: 22 languages, add .vue and .svelte to file types table

efce74d

safishamsi force-pushed the v4 branch from 64e4c64 to 5843ffc Compare April 23, 2026 18:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add hierarchical clustering: L0 topics → L1 communities → L2 nodes#297

Add hierarchical clustering: L0 topics → L1 communities → L2 nodes#297
rongmiao926-hub wants to merge 61 commits into
Graphify-Labs:v4from
rongmiao926-hub:v4

rongmiao926-hub commented Apr 13, 2026

Uh oh!

ljinshuan commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Uh oh!

Conversation

rongmiao926-hub commented Apr 13, 2026

Summary

Changes

Backward compatibility

Test plan

Uh oh!

ljinshuan commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants