Conversation
Signed-off-by: ryo-ebata <r.ebata@hajimari.inc>
ryo-ebata
marked this pull request as draft
September 18, 2026 03:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Persist GraphQL collector progress per task, raw scope, nested collector and input so restarting DevLake can resume from the last committed page instead of clearing raw data and starting over.
Does this close any open issues?
Closes #9142.
Validation
go test -race -count=1.bash backend/test/graphql-resume-crash.shandDEVLAKE_CRASH_DRIVER=postgres bash backend/test/graphql-resume-crash.sh.Compatibility and limits
The migration adds
_devlake_graphql_collector_states; task-less callers retain legacy behavior. Known input/code/page-setting changes are rejected and require a new task. Unknown GraphQL partial errors no longer complete a persisted task even when IgnoreQueryErrors is set. Inputs are staged to disk before fetching to detect iterator failure and manifest drift.Raw integrity uses row counts, not content checksums. Arbitrary connection/environment changes and upstream cursor/result-set changes are not snapshot-guaranteed. The subtask progress display remains per-attempt; persisted data/cursors determine what is refetched. No live GitHub/Linear service or production database was used for the integration tests.
See
backend/test/graphql-resume-coverage.mdfor the full audit matrix and reproduction instructions. Local results do not imply upstream GitHub Actions approval or success.