Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .claude/SCHEMA_DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -609,3 +609,30 @@ Sibling halves: codeanalyzer-java#255/#256, codeanalyzer-typescript#201/#202.
is where the comment model stops. Its closing comment names #203 as dropping its
comment goals for the same reason. `python-sdk`'s `get_all_comments` raising on the
Neo4j backend is now the permanent answer, not a workaround.

## 2026-09-14 — co-positioned call sites get a key disambiguator (#215)

- **A body key may carry `/N`.** `line:col` addressed a *position*, and two nested
`ast.Call` nodes can share one (`getattr(o, n)(x)`, `f()()()`), so one of them was
dropped from `body` and with it from `cfg`/`cdg`/`ddg` and the Neo4j projection.
The key sequence is now `line:col`, then `/2`, `/3`, … per further call site at
that position. `schema/ids.py::call_body_keys` is the one definition; L1, L2, the
dataflow builder, the defuse linker and the graph projection all re-derive the
pairing from it instead of rebuilding a key from a position.
- **The spelling is codeanalyzer-typescript's, adopted verbatim** (`callBodyKeys`,
`src/schema/l1Body.ts`): `/` and a 2-based counter, not `#` and not an
end-position key. A term coined twice is permanently wrong.
- **The outermost call keeps the bare key.** Call sites are recorded pre-order in
both analyzers, so this needs no extra rule — but it does change what a colliding
key resolves to: `11:18` used to be the inner `getattr`, and is now the
invocation. Non-colliding keys are untouched.
- **codeanalyzer-java is structurally immune.** `BodyNodeBuilder.anchorOfStatement`
keys a call at the invoked name token, so its nested calls never share an anchor.
Re-anchoring python that way was rejected: it moves every existing call key and
does not fix a call-of-call, which has no name token.
- **A call-of-call resolves to nothing.** `_callee_anchor` returns `None` when
`node.func` is an `ast.Call`, so the site carries `callee_signature=None` and
`method_name="<unknown>"` instead of claiming to call the inner callee. What the
dynamic call reaches is deliberately not inferred.
- **Neither version moves** — the 2026-09-07 hold stands. The payload shape is
unchanged; only the key space below the callable widens.
6 changes: 5 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,11 @@ declared callables by their `can://` tree id, imported/builtin targets by a
intra-callable `cfg`/`cdg`/`ddg`/`summary` edge endpoint: `"line:col"` for real
statements, `"@entry"`/`"@exit"` for the CFG bookends, `"@formal_in:0"` /
`"@formal_out"` for formals, and `"<callsite>/actual_in:0"` /
`"<callsite>/actual_out"` for actuals (parented to their call site). The
`"<callsite>/actual_out"` for actuals (parented to their call site). Two nested
calls can *start* at one position (`getattr(o, n)(x)`), so a call key may carry a
`/2`, `/3`, … disambiguator — outermost call first, the bare `line:col` (#215).
`codeanalyzer/schema/ids.py::call_body_keys` is the single definition of that
sequence; the spelling is codeanalyzer-typescript's, adopted verbatim. The
bijection `(signature, int node_id) ↔ (can:// id, local id)` is built once by
`codeanalyzer/dataflow/identity.py` and feeds **both** projections, keeping them
in lockstep.
Expand Down
28 changes: 19 additions & 9 deletions codeanalyzer/dataflow/builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
from __future__ import annotations

import ast
from collections import defaultdict
from pathlib import Path
from typing import Callable, Dict, List, Optional, Set, Tuple

Expand All @@ -43,7 +44,7 @@
from codeanalyzer.dataflow.pdg import build_pdg
from codeanalyzer.dataflow.sdg import ProgramGraphsIR, assemble_sdg
from codeanalyzer.dataflow.summaries import CallSite, FunctionInfo, compute_summaries
from codeanalyzer.schema.ids import stamp_body_ids
from codeanalyzer.schema.ids import call_body_keys, stamp_body_ids
from codeanalyzer.schema.py_schema import PyApplication, PyCallable, PyClass, PyModule
from codeanalyzer.utils import logger

Expand Down Expand Up @@ -291,6 +292,11 @@ def _span_of(source: str, node) -> Optional["Span"]:
continue
pycallable.body[local] = BodyNode(kind=node.kind, span=span)

keys_at: Dict[str, List[str]] = defaultdict(list)
for key, n in pycallable.body.items():
if n.kind == "call":
keys_at[key.split("/", 1)[0]].append(key)

# #115: anchor nested call vertices to their statement. A bare-call
# statement shares its key with its CFG node (handled above); a call
# nested inside a larger statement (`y = f(x)`) has its own key and
Expand All @@ -301,12 +307,16 @@ def _span_of(source: str, node) -> Optional["Span"]:
continue
stmt_local = im.local(node.id)
for call in _calls_in(node.ast_node):
call_key = f"{call.lineno}:{call.col_offset}"
child = pycallable.body.get(call_key)
if child is None or call_key == stmt_local:
continue
if child.kind == "call":
child.parent = stmt_local
# Every call node at this position, not just one: nested calls
# that share a start column are keyed `line:col`, `line:col/2`,
# ... and each of them needs the anchor (#215).
base = f"{call.lineno}:{call.col_offset}"
for call_key in keys_at.get(base, ()):
child = pycallable.body.get(call_key)
if child is None or call_key == stmt_local:
continue
if child.kind == "call":
child.parent = stmt_local

if want_cfg:
pycallable.cfg = [
Expand Down Expand Up @@ -378,7 +388,7 @@ def build_program_graphs(
calls_by_pos.setdefault(pos, (node.id, call))
calls_by_line.setdefault(call.lineno, (node.id, call))

for site in pycallable.call_sites or []:
for site_key, site in call_body_keys(pycallable.call_sites):
# Prefer the callsite's body-backfilled callee over Jedi's own
# callee_signature side channel: under cross-test parso/Jedi cache
# pressure that inference can silently degrade (full-suite-only
Expand All @@ -388,7 +398,7 @@ def build_program_graphs(
# Only a resolved INTERNAL target counts (id_to_sig misses on an
# external/unresolved callee); falls through to callee_signature
# exactly as before whenever the body doesn't have an answer.
body_node = pycallable.body.get(f"{site.start_line}:{site.start_column}")
body_node = pycallable.body.get(site_key)
target = (id_to_sig.get(body_node.callee) if body_node else None) or site.callee_signature
if not target:
continue
Expand Down
19 changes: 10 additions & 9 deletions codeanalyzer/neo4j/project.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,9 @@
PyVariableDeclaration,
)
from codeanalyzer.schema import model_dump
from codeanalyzer.schema.ids import application_id, external_id, global_ordinal, purl_pypi
from codeanalyzer.schema.ids import (
application_id, call_body_keys, external_id, global_ordinal, purl_pypi,
)
from codeanalyzer.schema.py_schema import PyDecorator, byte_offsets


Expand Down Expand Up @@ -183,12 +185,14 @@ def _project_program_graphs(
continue # unstamped callable — assign_ids must run first
owner = _sym(c.id) # the :PyCallable node, keyed by its can:// id
# ``callee_signature`` lives on ``PyCallable.call_sites``, not on the body
# node, so the graph joins the two on the call site's position (#203).
# node, so the graph joins the two on the call site's BODY KEY (#203,
# #215) -- re-derived from the same sequence L1 keyed ``body`` with, so
# two calls that start at one position keep their own signatures.
# ``argument_types`` is deliberately not joined: it is the legacy field #86
# split into ``PyCallArgument``, already carried as ``arguments_json``.
sig_by_pos = {
(cs.start_line, cs.start_column): cs.callee_signature
for cs in (c.call_sites or [])
sig_by_key = {
key: cs.callee_signature
for key, cs in call_body_keys(c.call_sites)
if cs.callee_signature
}
for local_key, node in (c.body or {}).items():
Expand All @@ -204,10 +208,7 @@ def _project_program_graphs(
{
"kind": node.kind,
**_span_props(span),
"callee_signature": (
sig_by_pos.get((span.start[0], span.start[1]))
if span else None
),
"callee_signature": sig_by_key.get(local_key),
"var": node.of,
"call_node": node.parent,
# Call-site detail (#120). The JSON emits one node per
Expand Down
38 changes: 37 additions & 1 deletion codeanalyzer/schema/ids.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
an application named ``python`` mints ``can://python/python/...``, so a test
for ``can://python/`` no longer means "a python id"; test the scheme instead."""
from __future__ import annotations
from typing import List, Optional
from typing import Iterable, Iterator, List, Optional, Tuple

SCHEME = "can://"

Expand Down Expand Up @@ -52,6 +52,42 @@ def external_id(app_id: str, module: Optional[str], name: str) -> str:
return f"{base}/{module}/{name}" if module else f"{base}/{name}"


def call_body_keys(sites: Iterable) -> Iterator[Tuple[str, object]]:
"""The body key of each call site, in recording order: ``line:col``,
disambiguated ``/2``, ``/3``, ... when nested calls share a start position
(#215).

`getattr(o, n)(x)` begins the outer application and the inner `getattr` at the
same column, so a bare ``line:col`` key keeps one of the two and the dynamic
invocation is lost. Call sites are recorded pre-order, so the bare key goes to
the OUTERMOST call and the nested ones take the suffixes. The spelling is
codeanalyzer-typescript's (``callBodyKeys``, ``src/schema/l1Body.ts``), adopted
verbatim; the ``/`` never collides with a param-vertex segment, which always
begins ``actual_``.

The SINGLE definition of the sequence -- L1 builds ``body`` with it, and L2, the
dataflow builder, the defuse linker and the Neo4j projection re-derive the same
pairing from it rather than re-deriving a key from a position.
"""
used = set()
for cs in sites or []:
base = f"{cs.start_line}:{cs.start_column}"
key = base
k = 2
while key in used:
key = f"{base}/{k}"
k += 1
used.add(key)
yield key, cs

def call_body_key(callable_, site) -> Optional[str]:
"""``site``'s body key within ``callable_`` — the pairing of
:func:`call_body_keys`, for a caller that holds one site rather than the list."""
for key, cs in call_body_keys(callable_.call_sites):
if cs is site:
return key
return None

def global_ordinal(callable_id: str, local_key: str) -> str:
"""The GLOBAL ordinal id of a body node from its LOCAL key: synthetic keys
(`@entry`, `@formal_in:0`) already carry the `@`; positional keys (`15:2`,
Expand Down
5 changes: 2 additions & 3 deletions codeanalyzer/schema/l1_body.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
"""L1 body population: materialize `call` nodes from existing call sites.
`callee` is left None here — the sanctioned null→id refinement happens at L2."""
from __future__ import annotations
from codeanalyzer.schema.ids import stamp_body_ids
from codeanalyzer.schema.ids import call_body_keys, stamp_body_ids
from codeanalyzer.schema.py_schema import PyApplication, PyClass, PyCallable, BodyNode, Span, byte_offsets

def _do_callable(source: str, c: PyCallable) -> None:
for cs in c.call_sites or []:
key = f"{cs.start_line}:{cs.start_column}"
for key, cs in call_body_keys(c.call_sites):
span = Span(start=(cs.start_line, cs.start_column),
end=(cs.end_line, cs.end_column),
bytes=byte_offsets(source, cs.start_line, cs.start_column, cs.end_line, cs.end_column)) if source else None
Expand Down
7 changes: 4 additions & 3 deletions codeanalyzer/schema/l2_callees.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,18 @@

Two resolution sources feed the backfill: Jedi's `callee_signature` on the
call site itself, and the defuse linker's returned map (keyed by caller
signature + "line:col"). The linker's resolutions are deliberately NOT written
signature + the call site's body key, which carries a `/N` disambiguator when
nested calls share a start position -- #215). The linker's resolutions are deliberately NOT written
into `callee_signature` — the symbol table round-trips through the analysis
cache, and a persisted resolution would resurface on a warm run as a Jedi
edge, silently changing provenance."""
from __future__ import annotations
from codeanalyzer.schema.ids import call_body_keys
from codeanalyzer.schema.py_schema import PyApplication, PyClass, PyCallable


def _do_callable(c: PyCallable, sig_to_id: dict, resolutions: dict) -> None:
for cs in c.call_sites or []:
key = f"{cs.start_line}:{cs.start_column}"
for key, cs in call_body_keys(c.call_sites):
jedi_sig = cs.callee_signature
if jedi_sig and jedi_sig.startswith("typing."):
# A decorator-typed callable resolved to its annotation, not a
Expand Down
11 changes: 7 additions & 4 deletions codeanalyzer/semantic_analysis/defuse_linker.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,14 @@
import builtins as _py_builtins
from typing import Dict, List, Optional, Tuple

from codeanalyzer.schema.ids import call_body_key
from codeanalyzer.schema.py_schema import PyCallable, PyCallEdge, PyClass, PyModule

__all__ = ["defuse_linker_edges"]

# (caller signature, "line:col" of the call site) -> resolved callee signature
# (caller signature, the call site's body key) -> resolved callee signature.
# The key is `call_body_key`, not a bare "line:col": two nested calls can start
# at one position and only one of them is the resolution being recorded (#215).
Resolutions = Dict[Tuple[str, str], str]

_MAX_CHAIN = 16 # assignment-chain hops before giving up (cycle safety net)
Expand Down Expand Up @@ -1095,7 +1098,7 @@ def bump(src: str, dst: str) -> None:
oracle.vote(sig, site)
bump(caller.signature, sig)
resolutions[
(caller.signature, f"{site.start_line}:{site.start_column}")
(caller.signature, call_body_key(caller, site))
] = sig

# Calls Jedi's extractor never recorded as sites at all (with-
Expand Down Expand Up @@ -1398,7 +1401,7 @@ def _method_on(t, method, qual):
oracle.vote(sig, site)
bump(caller.signature, sig)
resolutions[
(caller.signature, f"{site.start_line}:{site.start_column}")
(caller.signature, call_body_key(caller, site))
] = sig
made_progress = True
else:
Expand Down Expand Up @@ -1452,7 +1455,7 @@ def _method_on(t, method, qual):
oracle.vote(sig, site)
bump(caller.signature, sig)
resolutions[
(caller.signature, f"{site.start_line}:{site.start_column}")
(caller.signature, call_body_key(caller, site))
] = sig
made_progress = True
remaining = still
Expand Down
30 changes: 22 additions & 8 deletions codeanalyzer/syntactic_analysis/symbol_table_builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,20 +130,27 @@ def _infer_callee(
return None, False

@staticmethod
def _callee_anchor(node: ast.Call) -> Tuple[int, int]:
"""Position of the callee *name* for Jedi inference.
def _callee_anchor(node: ast.Call) -> Optional[Tuple[int, int]]:
"""Position of the callee *name* for Jedi inference, or ``None`` when the
callee is not a name at all.

An ``ast.Call``'s own ``lineno``/``col_offset`` is the first
character of the whole call expression — for an attribute call
``receiver.method(...)`` that is the receiver token, and Jedi
would infer the receiver's type instead of the invoked method
(issue #80). Anchor attribute calls inside the attribute name —
its last character, so one-character names stay in range; other
callee shapes keep the call-expression start.
callee shapes keep the call-expression start. A callee that is itself a
call has no name to anchor on, so it yields ``None`` (#215).
"""
func_expr = node.func
if isinstance(func_expr, ast.Attribute):
return func_expr.end_lineno, func_expr.end_col_offset - 1
if isinstance(func_expr, ast.Call):
# `getattr(o, n)(x)`: the callee IS a call, so the expression start is
# the INNER call's name and inferring there labels this site a call to
# `getattr` -- the thing that produced the callee, not the callee (#215).
return None
return node.lineno, node.col_offset

@staticmethod
Expand Down Expand Up @@ -766,11 +773,18 @@ def _call_sites(self, fn_node: ast.FunctionDef, script: Script) -> List[PyCallsi
func_expr = node.func

method_name = "<unknown>"
anchor_line, anchor_col = self._callee_anchor(node)
callee_signature, is_constructor = self._infer_callee(
script, anchor_line, anchor_col
)
return_type = self._infer_call_return_type(script, anchor_line, anchor_col)
anchor = self._callee_anchor(node)
if anchor is None:
# A dynamic invocation: the site is recorded, and it resolves to
# nothing. Guessing a target here is what produced a graph full of
# calls to `builtins.getattr` (#215).
callee_signature, is_constructor, return_type = None, False, None
else:
anchor_line, anchor_col = anchor
callee_signature, is_constructor = self._infer_callee(
script, anchor_line, anchor_col
)
return_type = self._infer_call_return_type(script, anchor_line, anchor_col)

receiver_expr = None
receiver_type = None
Expand Down
Loading