Skip to content

Add Evaluate(Ref) Gateway API - #3137

Merged
tonistiigi merged 5 commits into
moby:masterfrom
jedevc:evaluate-gateway
Oct 12, 2022
Merged

tonistiigi merged 5 commits into
moby:masterfrom
jedevc:evaluate-gateway

Conversation

@jedevc

@jedevc jedevc commented Sep 29, 2022 •

Copy link
Copy Markdown
Member

This PR is a replacement for #2947 (cherry-picking that commit for consistency, so that calling Evaluate over each ref in a result is the same as setting Evaluate: true in the SolveRequest).

This can be invoked on a ref using ref.Evaluate() in a frontend - this allows for on-demand unlazying a reference, instead of needing to wait until export. Previously, as a hack, this could be done using StatFile, however this has the disadvantage of needing to pull the image to be able to mount it, which isn't required to ensure that the image is unlazied.

Additionally, the previous Evaluate field in the SolveRequest was not sufficient, as this causes the Solve to block - we may want to perform other operations, before then determining whether to Evaluate, as in docker/buildx#1197.

Comment thread frontend/gateway/grpcclient/client.go Outdated
Comment thread client/build.go Outdated
Comment thread frontend/gateway/gateway.go Outdated
func (lbf *llbBridgeForwarder) Evaluate(ctx context.Context, req *pb.EvaluateRequest) (*pb.EvaluateResponse, error) {
ctx = tracing.ContextWithSpanFromContext(ctx, lbf.callCtx)

_, err := lbf.getImmutableRef(ctx, req.Ref, "/")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not part of your PR, but why does getImmutableRef() take a path ? 🤯

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀 seemingly to print a deceiving error message apparently...

func (lbf *llbBridgeForwarder) getImmutableRef(ctx context.Context, id, path string) (cache.ImmutableRef, error) {
	lbf.mu.Lock()
	ref, ok := lbf.refs[id]
	lbf.mu.Unlock()
	if !ok {
		return nil, errors.Errorf("no such ref: %v", id)
	}
	if ref == nil {
		return nil, errors.Wrap(os.ErrNotExist, path)
	}

Technically it's possible for a value in Result.Ref to be nil, but printing file does not exist: <path> for that doesn't feel like the right message - empty ref feels like a more apt description of what's gone wrong. Definitely an edge case, a sane frontend should never produce nil refs like that.

I'll add a commit on top of the stack in this PR to tidy up the method, good catch.

Comment thread frontend/gateway/grpcclient/client.go Outdated
Comment thread frontend/gateway/forwarder/forward.go Outdated
Comment thread solver/llbsolver/bridge.go Outdated
Signed-off-by: Justin Chadwell <me@jedevc.com>
Always use the value of the evaluate field to force result generation,
which previously was not performed for frontends. This improves API
consistency, and ensures the value is used regardless of whether
the solver uses a frontend, or a raw definition.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
The StatFile fallback needs reworking to ensure that all refs in the
result are evaluated, not just the first one found.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Remove path argument from llbBridgeForward.getImmutableRef, as it is
only used to print a deceiving error message. Contents of the
Result.Refs map should not be nil, and sane frontends should not produce
them - printing that the file cannot be found doesn't indicate that this
is an abnormal edge case.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants