node-cache-pruner reads every cache's size even when a running pod means it cannot prune #146

node-cache-pruner.sh runs du -sk over every profile's cache on every round, before it checks whether a runner pod mounts the cache. It can only empty a cache when no pod mounts it, so on a node that is always running jobs nearly every round reads the size and then does nothing with it.

On a node with a consumer SATA SSD and a shared pnpm store of about 300k files, that read is expensive: the jobs' page cache churn evicts the inode tables between rounds, so each du re-reads them, and with ext4's inode readahead a round read 5 to 6 GiB at about 90 MiB/s for about a minute. While it ran, /proc/pressure/io full on the node was 14 to 32%, and over a six-minute sample the pruner's windows held 19% and 54% of the node's whole I/O stall on two such nodes. The pruner container's cgroup io.stat showed about 3.5 TiB read in a little over two days at the default CHECK_SECONDS of 300.

Listing the pods first and reading the size only when the cache could be emptied (no pod mounts it, or it is retired) would remove almost all of it on a busy node. The hard-limit report still needs a size while pods run; that could be read on a slower cadence of its own. A consumer can raise github_runner_arc_node_cache_prune_seconds meanwhile, which only spreads the same reads out.