Skip to content

L2 call graph is not byte-reproducible: Jedi overload resolution for open() varies run to run #146

Description

@rahlk

Is your feature request related to a problem? Please describe.

Two byte-identical invocations of the analyzer produce different call graphs. The variance is small but real, and it is not the sharding defect from #145 — it survives with --pycg-shard sequential, with --ray, and at any --pycg-max-iter.

Measured on test/fixtures/whole_applications/flask (82 files), -a 2 --no-venv --pycg-shard --pycg-shard-ceiling 25 --clear-cache, two runs per configuration:

config edges A / B differing edges
sequential, --pycg-max-iter 50 1263 / 1263 1
sequential, --pycg-max-iter 3 1263 / 1263 2
--ray, --pycg-max-iter 50 1168 / 1168 3
--ray, --pycg-max-iter 3 1168 / 1167 2

0.1–0.3% of edges. Ray adds no meaningful variance, so this is not a scheduling artifact.

Almost every differing edge involves file-IO types — _TextIOBase, _BufferedIOBase, FileIO. The reproducible instance is src/flask/cli.py:1004:

with open(startup) as f:
    eval(compile(f.read(), startup, "exec"), ctx)

open() is overloaded in typeshed (text mode → TextIOWrapper, binary → BufferedReader). Jedi's overload resolution returns a different candidate set on different runs, so f.read() resolves to either:

can://python/flask/src/flask/cli.py/shell_command() -> …/@external/builtins._TextIOBase/read
can://python/flask/src/flask/cli.py/shell_command() -> …/@external/builtins._BufferedIOBase/read

SymbolTableBuilder._first_definition is already a deterministic picker — min(definitions, key=lambda d: (d.full_name or "", d.name or "")). Given both candidates it would always choose _BufferedIOBase (B < T). The instability is in the candidate set Jedi hands us, not in our selection.

PYTHONHASHSEED=0 is already pinned for the driver (#99) and propagated into Ray workers via runtime_env.env_vars (core.py:48), so hash randomisation is not the cause.

Describe the solution you'd like

Not stated in the original issue.

Describe alternatives you've considered

Not stated in the original issue.

Additional context

Caveats and known risks

Definition of done

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions