Skip to content

Neo4j: never delete by default; gate purge on --eager and scope it to this analyzer #171

Description

@rahlk

Problem

codeanalyzer/neo4j/bolt.py:127-131 purges a changed module with two unlabelled statements:

tx.run("MATCH (x {_module: $m})-[r]->() DELETE r", m=module)
tx.run("MATCH (x {_module: $m}) ... DETACH DELETE x", ...)

MATCH (x {_module: $m}) matches any node in the database with that _module value, not just
python's. codeanalyzer-typescript and codeanalyzer-java both set _module to the file key on
their nodes, so where a python module and a sibling analyzer's module share a file key, a python
push deletes that sibling's nodes and their relationships.

It also has no index behind it: INDEXES covers py_callable_name, py_class_name, and a
fulltext, but nothing on _module. An unlabelled match on a non-indexed property is a full store
scan, run once per changed module.

Found while fixing the same class of bug in codeanalyzer-typescript
(codellm-devkit/codeanalyzer-typescript#116). There the equivalent statement was
MATCH (n) WHERE n._module IS NOT NULL AND NOT n:CanNode DETACH DELETE n, which described python's
and java's nodes exactly — both set _module, neither applies :CanNode. It surfaced only as an
out-of-memory error, because the delete was large enough to exhaust
dbms.memory.transaction.total.max and roll back. A smaller foreign graph would have been deleted
silently.

Worth noting python's orphan prune (bolt.py:154-156) is anchored correctly —
MATCH (:PyApplication {name: $app})-[:PY_HAS_MODULE]->(m:PyModule). It is the per-module purge
that is exposed. That anchored pattern is the better model, and typescript is adopting it.

Scope boundary

The two unlabelled statements in the per-module purge. Not in scope: the orphan prune, which is
already scoped to the application.

Goals

  • Anchor both statements on python-owned labels so a sibling analyzer's nodes cannot match
  • A test asserting a foreign node with a colliding _module survives a push
  • Consider an index on the anchoring label's _module — the current match is a full scan per module

Caveats and known risks

  • PySymbol is already a shared merge label across PyClass/PyCallable/PyExternal and could
    serve as part of the anchor, but it does not cover PyModule/PyAttribute/PyVariable/
    PyBodyNode, so the predicate likely has to enumerate labels.
  • Silent data loss: no error, no log line when this fires. Any occurrence to date would have gone
    unnoticed, so shipped graphs may warrant an audit.

Definition of done

A python push into a database holding a typescript or java graph with an overlapping _module
value leaves that graph byte-identical.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions