Disclosure

AI use, and how we report what we find

Two separate obligations: being honest about how this was built, and being careful about what we tell the world.

01 — how this was built

AI assistance was used

  • Scaffolding and boilerplate
  • Drafting documentation
  • Drafting the labeler’s rules table — deterministic, no model at runtime
  • Suggesting Cypher that a human then reviewed

Human checkpoints

  • Every Cypher query run against a live 6.0.2 instance before commit
  • Every benchmark number read by hand from eval/results/*.json; every transcript from docs/live-evidence/
  • The gold label set hand-labelled, with no model input
  • Graph model and architecture decisions are ours

We can walk through any query, index and parameter choice on request. If a number cannot be traced to a file in the repo, treat it as a bug and tell us.

02 — what we learned

The short version, because it shaped the product more than any feature did.

We started with the wrong idea, and the research killed it

Our first plan was a router that trims MCP tool schemas to save context. The pain is real (tool definitions can consume a large share of a context window), but the fix does not need a graph — keyword search, embeddings or built-in tool search all work, and Anthropic already ships one. Under this hackathon's rules, a product that survives deleting its graph is disqualified. Ours would have. We dropped it.

The idea we landed on fails the same test in the other direction

“Is there a route from a stranger's words to my public data?” has no answer without edges, so the graph is doing the work, not decorating it.

We found documented behaviour that does not run

The engine's docs describe a shortest-path index absent from the pinned image, and a parameter-passing form that silently fails to bind. We wrote fallbacks and said so, rather than quoting a benchmark we could not reproduce.

Our first colour palette failed accessibility

The two colours carrying the entire explanation — private data and the leak path — measured below the AAA bar we had claimed. We replaced them; the palette check now measures every pair instead of trusting them.

A published paper already solved the easy half

We claimed nobody models tool dependencies. Someone does, on a public benchmark. We cite their result and we do not re-implement it — retrieval accuracy is not our product, and Benchmarks says exactly that. We leave that row out rather than claim it.

03 — reporting findings in other people's servers

Composition is not a vulnerability. Most of what we surface is a combination of servers that each behave correctly. We treat that distinction as a hard rule, not a courtesy.

We file an issue only when it is one server's fault

Either a single server holds all three legs, or its annotations are wrong enough to make clients mislabel it. A pair of correct servers is not reported to either maintainer.

Private channel first, always

We read SECURITY.md before anything else. Where a private channel exists, we use it and wait for a reply before mentioning it publicly.

Capabilities, never payloads

Our reports describe what a tool can read or write, suggest annotation fixes and scopes, and never include a working injection or steps against real users.

Every leaderboard entry is disputable

A maintainer can ask us to correct or remove their entry, with no argument. Open an issue, or use the private channel below.

Found a problem in LethalPath?

Please report it privately first, so we can ship a fix before it is public. We aim to acknowledge within 48 hours. 

# see SECURITY.md in the repo
# private channel: GitHub → Security → Report a vulnerability