← Back to projects

Developer Tools

CodeGraph

Explore a repository through syntax, relationships, and question-specific retrieval instead of sending all of its source to an agent.

Measured results

LLM context reduction
97.5%

≈737K → ≈18K tokens per query

Supported-source tokens compared with tool-returned context.

Measures retrieved context size, excluding model prompts and generated answers; not accuracy, billing, or latency.

Read context measurement and scope ↓

I designed and built CodeGraph independently during summer 2026, before the school year began. My work spans the full system: repository ingestion and parsing, the graph and retrieval pipeline, the API, and the interactive dashboard.

System
Repository exploration
Interface
React Flow graph + source drawer
Retrieval
Seven LangGraph tools over Neo4j
View CodeGraph repository ↗

The problem

An unfamiliar repository presents two related problems: finding relevant code and understanding how it connects. A full-source prompt is large, while isolated snippets can lose caller and dependency context. CodeGraph builds a navigable structural index and lets an agent choose smaller observations for the current question.

System architecture

A React dashboard submits a GitHub repository to FastAPI, displays a React Flow graph, and fetches source separately when a function is selected. The backend combines Tree-sitter parsing, Neo4j relationships, metadata embeddings, Leiden communities, and a LangGraph ReAct agent powered by Claude Sonnet 5.

Two paths through one repository index

  1. 01 / Interface

    React dashboard ↔ FastAPI

    Repository submission, graph exploration, source inspection, and questions.

    Ingestion path ↓

  2. 02 / Indexing

    GitHub archive → Tree-sitter → ETL

    Safe temporary extraction; supported-source discovery; syntax records; three-pass graph writes.

    Persist + enrich ↓

  3. 03 / Storage

    Neo4j graph · vectors · communities

    Repository → File → Function; inferred CALLS edges. OpenAI name/path embeddings and sampled community labels enrich the index.

    Question path: agent ↔ tools ↔ index ↓

  4. 04 / Retrieval

    LangGraph ReAct ↔ Claude Sonnet 5

    Seven tools return structure, direct callers, outgoing calls, unresolved external names, semantic matches, and community summaries. Tool observations inform the final answer.

Ingestion builds the repository index. Questions use graph and semantic retrieval, while the dashboard provides source inspection on demand.Graphic

From ingestion to context

  1. Acquire and discover

    Validate an HTTPS GitHub owner/repository URL, download its default-branch archive, and extract with path-containment checks. Discover Python, JavaScript/JSX, TypeScript/TSX, Go, and Java source.

  2. Count and parse

    Count supported-source tokens and capture definitions, source spans, and lexical call records with Tree-sitter. Up to 32 per-file stages are active; a per-language lock serializes shared parser use.

  3. Write and enrich

    Replace the old repository scope, then write files, functions with metadata embeddings, and calls in 100-record batches. Run undirected Leiden clustering and generate labels from sampled names and paths. These batches are separate transactions.

  4. Expose the index

    Stream newline-delimited progress records; the client buffers partial lines and requires a terminal completion record. Fetch graph data after success. The graph endpoint caps query rows at 200 and leaves raw source and vectors out of the bulk payload.

  5. Retrieve for the question

    Start a fresh system/user message pair. The agent chooses tools, receives formatted observations, and can call again before answering. Metadata-vector candidates are selected globally, then filtered by repository.

Engineering challenges

Connecting syntax to a usable graph

Repository exploration brings together function definitions, call relationships, and source locations. Consistent identities across parsing, storage, and lookup are central to making those views useful.

Making ingestion observable

Parsing and enriching a repository spans multiple stages. Streamed progress helps the interface communicate indexing status before the graph is ready to explore.

Combining structural and semantic retrieval

Graph queries expose relationships, while metadata embeddings find related names and paths. The two approaches support different ways of navigating unfamiliar code.

Correctness & validation

Tests cover parser behavior, API helpers, formatting, and repository scoping.

Backend tests
92 passed

Parser, formatting, and repository-scoping checks.

Frontend tests
13 Node tests passed

API-helper and utility checks; the Vite production build also passed.

Input handling includes path-safe archive extraction, parameterized database queries, and raw-body HMAC verification.

Results & measurement

LLM context / per query

≈737Kto≈18K

tokens per query

LLM context reduction
97.5%

Supported-source tokens compared with tool-returned context.

Baseline: total tokens across discovered, supported source files.

Selected context: tokens in the tool responses returned for the query.

Context reduction = (baseline − tool context) / baseline × 100

This measures retrieved context size, excluding model prompts and generated answers.

Design decisions & tradeoffs

Static syntax across languages

Supported repositories span several languages. Tree-sitter extracts definitions and call syntax without running repository code.

Benefit
One parsing pipeline supplies multi-language structure for exploration.
Accepted limitation
Syntax alone cannot resolve dynamic dispatch or runtime call targets.

Graph, vectors, and communities

Repository questions need relationship queries and metadata-vector search. Neo4j stores both, with Leiden communities for an architectural overview.

Benefit
One store supports relationship, semantic, and community views.
Accepted limitation
Communities are exploratory groups, not verified module boundaries.

Engineering lessons

Consistent identities connect the system

Function identities link parsing, graph relationships, and source lookup. Schema design is part of retrieval design.

Separate progress from consistency

Progress reporting and index consistency solve different problems. Long-running ingestion needs both a clear status model and a deliberate data lifecycle.

Design retrieval around the question

Structure, semantic matches, and source inspection provide complementary ways to understand a repository. Choosing the right view keeps exploration focused.

Repository

Explore the parser, graph pipeline, retrieval tools, and dashboard on GitHub.