Skip to content

Sync template to version 0.1.20 #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
IvanHCenalmor wants to merge 1 commit into from
Closed

Sync template to version 0.1.20 #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 66 additions & 9 deletions .github/workflows/update_on_notebook_change.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ on:
- 'notebooks/**/requirements.yaml'
workflow_dispatch:

# This enables to run a single workflow at a time
# One run of this workflow at a time per branch. The src workflow has its own group: the two can run at the same time,
# because each one rebases its commit onto the latest branch before it pushes (a shared group kept only one waiting run, so a
# third push cancelled the waiting run of the other workflow).
concurrency:
group: notebooks-global-lock-${{ github.ref }}
cancel-in-progress: false
Expand Down Expand Up @@ -98,6 +100,12 @@ jobs:
.tools/python/merge_requirements.py \
python .tools/python/merge_requirements.py --source-dir notebooks --output requirements.txt

# requirements*.txt -> requirements*.lock.txt: every package the installer installs with pip, with its exact version (uv, valid on
# Windows, macOS and Linux). A lock file is rewritten only when its requirements file, Python or JupyterLab version changed.
- name: Lock requirements
run: |
bash .tools/bash/run_with_context.sh artifacts/error_log_document.log "lock_requirements.py" .tools/python/lock_requirements.py python .tools/python/lock_requirements.py

- name: Check for changes
id: check-changes
run: |
Expand Down Expand Up @@ -138,17 +146,66 @@ jobs:
- name: Commit updated files
id: auto-commit
if: steps.check-changes.outputs.has_changes == 'true' && steps.test-conda-env.outcome == 'success'
uses: stefanzweifel/git-auto-commit-action@b863ae1933cb653a53c021fe36dbb774e1fb9403 # v5
with:
commit_message: GitHub Action - Updated files because of notebook and/or requirements changes
file_pattern: requirements*.txt environment.yaml notebooks/* setup.py
env:
BRANCH: ${{ github.ref_name }}
COMMIT_MESSAGE: GitHub Action - Updated files because of notebook and/or requirements changes
# The run checked out the commit that triggered it (the scripts above compare it with its parent); the branch may have moved
# since (another workflow committed, or a new push). A rejected push is therefore retried after a rebase onto the latest branch.
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git ls-files -z --modified --deleted --others --exclude-standard -- 'requirements*.txt' environment.yaml 'notebooks/*' setup.py | xargs -0 -r git add -A --
if git diff --cached --quiet; then
echo "No changes to commit."
exit 0
fi
git commit -m "$COMMIT_MESSAGE"
for attempt in 1 2 3 4 5; do
if git push origin "HEAD:refs/heads/$BRANCH"; then
exit 0
fi
echo "The push was rejected (attempt $attempt of 5): rebasing onto the latest $BRANCH."
sleep $((attempt * 5))
git fetch origin "$BRANCH"
if ! git rebase --autostash "origin/$BRANCH"; then
git rebase --abort || true
echo "::error::The commit of this run conflicts with the latest $BRANCH; run this workflow again (Actions > Run workflow)."
exit 1
fi
done
echo "::error::The push was rejected 5 times; run this workflow again (Actions > Run workflow)."
exit 1

# Only when the merged requirements could not be installed: a failure of any other step (a script, the push) never reverts a commit.
- name: Revert user commit that broke requirements
if: failure() && github.event_name == 'push'
if: failure() && github.event_name == 'push' && steps.test-conda-env.outcome == 'failure'
env:
BRANCH: ${{ github.ref_name }}
BAD_COMMIT: ${{ github.sha }}
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git reset --hard
git clean -fd
git revert "${{ github.sha }}" -m 1 --no-edit
git push origin HEAD:${{ github.ref_name }}
for attempt in 1 2 3 4 5; do
git fetch origin "$BRANCH"
git checkout -B "$BRANCH" "origin/$BRANCH"
# A merge commit is reverted against its first parent (-m 1); any other commit is reverted as it is.
if [ "$(git rev-list --parents -n 1 "$BAD_COMMIT" | wc -w)" -gt 2 ]; then
REVERT_ARGS=(-m 1)
else
REVERT_ARGS=()
fi
if ! git revert --no-edit "${REVERT_ARGS[@]}" "$BAD_COMMIT"; then
git revert --abort || true
echo "::error::$BAD_COMMIT broke the requirements but could not be reverted automatically (later commits change the same lines): revert it by hand."
exit 1
fi
if git push origin "HEAD:refs/heads/$BRANCH"; then
exit 0
fi
echo "The push of the revert was rejected (attempt $attempt of 5): retrying on the latest $BRANCH."
sleep $((attempt * 5))
done
echo "::error::The revert of $BAD_COMMIT could not be pushed: revert it by hand."
exit 1
45 changes: 35 additions & 10 deletions .github/workflows/update_on_src_change.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@ on:
- 'src/**'
workflow_dispatch:

# This enables to run a single workflow at a time
# One run of this workflow at a time per branch. The notebook workflow has its own group: the two can run at the same time,
# because each one rebases its commit onto the latest branch before it pushes (a shared group kept only one waiting run, so a
# third push cancelled the waiting run of the other workflow).
concurrency:
group: notebooks-global-lock-${{ github.ref }}
group: src-change-lock-${{ github.ref }}
cancel-in-progress: false

jobs:
Expand All @@ -18,13 +20,13 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: write

steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
fetch-depth: 0 # Fetch all history for all branches and tags

- name: Update src change file
run: |
cat > .tools/meta/src_change.yaml <<EOF
Expand All @@ -33,9 +35,32 @@ jobs:

- name: Commit updated files
id: auto-commit
uses: stefanzweifel/git-auto-commit-action@b863ae1933cb653a53c021fe36dbb774e1fb9403 # v5
with:
commit_message: GitHub Action - Updated src change file

file_pattern: .tools/meta/src_change.yaml

env:
BRANCH: ${{ github.ref_name }}
COMMIT_MESSAGE: GitHub Action - Updated src change file
# The run checked out the commit that triggered it; the branch may have moved since (another workflow committed, or a new push).
# A rejected push is therefore retried after a rebase onto the latest branch.
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -A -- .tools/meta/src_change.yaml
if git diff --cached --quiet; then
echo "No changes to commit."
exit 0
fi
git commit -m "$COMMIT_MESSAGE"
for attempt in 1 2 3 4 5; do
if git push origin "HEAD:refs/heads/$BRANCH"; then
exit 0
fi
echo "The push was rejected (attempt $attempt of 5): rebasing onto the latest $BRANCH."
sleep $((attempt * 5))
git fetch origin "$BRANCH"
if ! git rebase --autostash "origin/$BRANCH"; then
git rebase --abort || true
echo "::error::The commit of this run conflicts with the latest $BRANCH; run this workflow again (Actions > Run workflow)."
exit 1
fi
done
echo "::error::The push was rejected 5 times; run this workflow again (Actions > Run workflow)."
exit 1
5 changes: 4 additions & 1 deletion .template_sync/manifest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
template_version: 0.1.19
template_version: 0.1.20
migrations:
- from: 0.0.0
to: 0.0.1
Expand Down Expand Up @@ -63,3 +63,6 @@ migrations:
- from: 0.1.18
to: 0.1.19
script: migrations/v0_1_18_to_v0_1_19.py
- from: 0.1.19
to: 0.1.20
script: migrations/v0_1_19_to_v0_1_20.py
Loading
Loading