Repository navigation
[Bug]: No run metrics on kubernetes backend when privileged: true #3398
Description
Steps to reproduce
- Deploy a
minikubecluster:minikube start --driver=kvm2 --container-runtime=containerd1 - Start any run with
privileged: true - Check metrics:
dstack metrics <run-name>
Actual behaviour
NAME STATUS CPU MEMORY GPU
dev-environment running - - -
Expected behaviour
No response
dstack version
0.20.0
Server logs
WARNING dstack._internal.server.background.tasks.process_metrics:139 Failed to connect to job dev-environment-0-0 to collect metricsAdditional information
$ ssh <run-name>
# curl -w "%{http_code}\n" http://localhost:10999/api/metrics
could not read memory usage: open /sys/fs/cgroup/memory.current: no such file or directory
500
Footnotes
-
Cannot reproduce with
dockercontainer runtime ↩
Activity
[-][Bug]: No run metrics on `kubernetes` backend[/-][+][Bug]: No run metrics on `kubernetes` backend when `privileged: true`[/+]un-def commentedon Dec 17, 2025
See: https://github.com/containerd/containerd/blob/d3ee5d6088b0e5444e26fc88e87bd0a0322d9c76/internal/cri/server/container_create.go#L927
See: https://github.com/kubernetes/enhancements/blob/0e409b47497e398b369c281074485c8de129694f/keps/sig-node/20191118-cgroups-v2.md#cgroup-namespace
[runner] Get container cgroup path from procfs
[runner] Get container cgroup path from procfs
[runner] Get container cgroup path from procfs (dstackai#3402)