fix(python-uv): preserve compiled bytecode after packaging - #926
xujiantop-crypto wants to merge 6 commits into
Conversation
github-actions
Bot
added
pr/external
stage/needs-triage
![aws-sam-tooling-bot[bot]](https://avatars.githubusercontent.com/b/1290?s=60&v=4){kind=small}
xujiantop-crypto
changed the title
|
|
||
| # Add configuration arguments | ||
| target_python = None | ||
| if config.compile_bytecode: |
There was a problem hiding this comment.
[GENERAL] Bytecode compilation is effectively enabled for every build, and there is no way to turn it off.
UvConfig.compile_bytecode still defaults to True (utils.py:107, and the new test test_uv_config_defaults now asserts self.assertTrue(config.compile_bytecode)), while no caller ever constructs a UvConfig with arguments: PythonUvWorkflow._setup_build_actions() builds PythonUvBuildAction without config, and the action falls back to self.config = config or UvConfig(). So on every build this now:
- shells out to
uv python find, - pins
--pythonto the discovered host interpreter, and - runs
compileall -fover the entire installed dependency tree.
Before this PR nothing was compiled, because to_uv_args() never emitted a bytecode flag — uv's own default is no compilation. The PR description states the opposite ("preserve the existing effective default by keeping bytecode compilation disabled unless explicitly enabled"), so the shipped default appears unintended.
The impact is concrete: compileall -f over a large dependency set (e.g. boto3/botocore, numpy) adds noticeable build time, and the generated __pycache__ trees roughly double the on-disk footprint of pure-Python dependencies, which counts against Lambda's 250 MB unzipped package limit. Functions currently near that limit could start failing to deploy after a plain sam build.
If opt-in was the intent, flip the default:
compile_bytecode: bool = False,If default-on is intentional, please say so explicitly, since it changes artifact size and build time for all existing users of this workflow.
| # Use the exact interpreter found by UV instead of repeating a version request that | ||
| # could resolve differently when the install command runs. | ||
| if target_python: | ||
| args.extend(["--python", target_python]) |
There was a problem hiding this comment.
[GENERAL] The --python pin is no longer needed and makes an optional optimization change how the install itself runs.
_compile_bytecode() invokes the discovered interpreter directly:
command = [python_executable, "-m", "compileall", "-f", "-q", "--invalidation-mode", "unchecked-hash", target_dir]The resulting .pyc files depend only on that interpreter and the installed sources — not on which interpreter uv used. Meanwhile the installed sources are already governed by --python-version and --python-platform (both handlers always pass platform="linux" plus an architecture). So --python contributes nothing to the bytecode outcome, but it does override uv's own interpreter selection for the whole uv pip install invocation — including the interpreter used for any sdist build — replacing whatever uv would have chosen (e.g. an active virtualenv) with whatever uv python find happens to return.
Dropping the pin keeps the install behavior identical to today and confines the new code path to the compilation step:
if python_version:
args.extend(["--python-version", python_version])with target_python used only for _compile_bytecode().
| if rc == 0 and stdout: | ||
| return stdout.strip() | ||
| diagnostic = (stderr or "").strip() or (stdout or "").strip() or "no diagnostic output" | ||
| LOG.warning("Could not locate target Python %s via uv (exit code %d): %s", python_version, rc, diagnostic) |
There was a problem hiding this comment.
[GENERAL] A missing target interpreter is the normal case for cross-version builds, so LOG.warning here will surface a scary message on healthy builds.
LOG.warning("Could not locate target Python %s via uv (exit code %d): %s", python_version, rc, diagnostic)Building a python3.9 function on a host that only has 3.12 installed — or any build in an image whose interpreter differs from the runtime — hits this path, and --no-python-downloads guarantees uv will not fill the gap. Since bytecode compilation is a best-effort optimization whose absence has no effect on the produced artifact, this belongs at LOG.debug (or LOG.info), otherwise users see a warning in sam build output for a build that succeeded exactly as intended. Note this is amplified by the default in comment 1: with compilation on by default, most cross-version builds will emit it.
The same applies to the "Target Python version is unavailable" warning at line 155 — _extract_python_version() raises when there is no runtime, so a falsy python_version reaching install_requirements() only happens for direct library callers.
1 participant
Issue #, if available: Fixes #924
Fixes #925
Description of changes
UvConfig.compile_bytecodeoption effective in real builds and retain its documented default ofTrueuv python find --no-project --no-python-downloads <version>--python, while retaining--python-versionfor dependency resolution--no-compile-bytecodetouv pip install, preventing uv,UV_COMPILE_BYTECODE, or project uv configuration from producing timestamp-based bytecode that SAM packaging would invalidatepython -m compileall -f -q --invalidation-mode unchecked-hash <target>so packaged.pycfiles remain valid after ZIP timestamps are normalized-fis intentional: it replaces any timestamp-based.pycalready present in an incremental target instead of allowingcompileallto skip it.Description of how you validated changes
SubprocessUvandUvRunnercoverage: 34 tests passedtest_packager.pyandtest_utils.pyrun: 49 tests passed; 6 existing Windows environment-dependent tests failed because the suite expects a Unixlsexecutable and the host temporary directory rejects file creationPythonUvBuildActionconfiguration enables bytecode compilationsample.cpython-313.pyc.pycheader and confirmedflags=1(unchecked-hash), rather than timestamp invalidation.pycremained loadable after the source timestamp/content no longer matchedgit diff --checkpassedChecklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.