Skip to content

Latest commit

Β 

History

23 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Squint 🦝🐾

Squint is a GPU-aware Slurm monitor for your terminal. Read-only, zero-config, runs anywhere.

Squint Cover License: Apache 2.0

It's read-only by design: it only ever runs squeue / scontrol, plus nvidia-smi for GPU telemetry (DCGM next). Nothing to trust it with, nothing it can break. Point it at a cluster and look.

βš™οΈ Install

πŸš€ Release πŸ› οΈ Build & Test

    # Install Squint
    brew install hiteshsahu/tap/squint   # (planned)
    
    go install github.com/hiteshsahu/squint@latest

▢️ Run

With Mocks

No cluster handy? It ships with a mock source, so squint runs on your laptop out of the box.

    squint          # mock data β€” runs anywhere

Squint Dashboard Mock

Keys:

  • q quit
  • r refresh (auto refresh every 2s)
  • ↑/↓/PgUp/PgDn/wheel to scroll.

With Live Slurm Cluster

    squint --live   # read your real Slurm cluster

No real cluster to point --live at?

Caravan bundles a real local Slurm cluster (controller + GPU compute nodes) into one CLI

macOS / Linux

     # start a local Slurm cluster in Docker/Podman
    caravan cluster up     
    # run a gpu workload
    caravan submit workloads\long_running_example.sh
    # monitor jobs                   
    squint --live

Windows (PowerShell)

    .\caravan.exe cluster up
    $env:GOOS = "linux"; $env:GOARCH = "amd64"; go build -o squint.linux .
    docker cp squint.linux slurmctld:/usr/local/bin/squint
    docker exec -it slurmctld squint --live

Why squint

Because cluster state is often hidden behind pages of squeue output.

Squint turns that into something you can understand at a glance.

Slurm is powerful but feels like infrastructure from another era squeue is hard to read, "why is my job pending?" is a dark art, and nothing shows you what the GPUs are actually doing. squint is the live, GPU-native view of your cluster, in your shell, right now.

Two things nothing else in the Slurm ecosystem does well:

1. Find the Squatter GPU

Every GPU on every node, colored by utilization and mapped to the job that owns it so the eight A100s allocated to a job using none of them light up red.

GPU heatmap with idle-shaming.

Squint

That's the "who's squatting?" view every platform team wants and no Slurm tool gives them.

2. "Why is my job pending?" in plain English.

Reason=(QOSMaxGRESPerUser) becomes "You've hit your GPU quota for this QOS β€” wait for a running job to finish, or submit somewhere with more headroom." For every common reason code, with the fix.

Squint


πŸ‘¨β€πŸ’» DEVELOP

  • Requires Go 1.22+

βš™οΈ Install dependencies

    # Install dependencies
    go mod tidy

πŸ§ͺ Build & Test

Tests are run as part of CI itself.

   # Build 
   go build ./...
   
   # Formatting go file
   gofmt -w . 
   
   # Linting
   go vet ./... 
   

▢️ Run

Mock Run

  • Run with mock gpu data
  • Any Machine: macOS, Windows, Linux
    go run .

Live SLURM Run

  • Run with Real GPU data
  • SLURM must be installed β€” squeue/scontrol/slurmd/munge present.
  • Only Linux or Winodws with WSL
    go run . --live          # Run with real Slurm
    

Keys: q quit Β· r refresh. Polls every 2s.

πŸ§ͺ Testing

Run test

    go test ./... 
    
    # Test with coverage                      
    go test -race -coverprofile=coverage.out ./...
    
    # See code coverage
    go tool cover -func=coverage.out

How Squint is Built


            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
            β”‚                 Squint TUI                 β”‚
            β”‚   GPU heatmap β€’ jobs panel β€’ dashboard     β”‚
            β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                               β”‚
                               β–Ό
            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
            β”‚              Source Layer                  β”‚
            β”‚     Mock Source  β€’  Live Source            β”‚
            β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                               β”‚
                               β–Ό
            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
            β”‚           Queue & GPU Insights              β”‚
            β”‚  Pending-Reason Translation                β”‚
            β”‚  Job β†’ Node β†’ GPU Allocation                β”‚
            β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                               β”‚
                               β–Ό
            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
            β”‚                  Slurm                     β”‚
            β”‚      squeue β€’ scontrol β€’ nvidia-smi        β”‚
            β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
            

πŸ“ Folder structure

The Source interface is the whole seam: Mock and Live both implement it, and the TUI never knows which one it's talking to.

squint/
    β”œβ”€β”€ main.go                         # CLI entrypoint (--live flag, Bubble Tea program)
    β”œβ”€β”€ internal/
    β”‚   β”œβ”€β”€ model/
    β”‚   β”‚   └── types.go                # Job, GPU, Node, Snapshot
    β”‚   β”‚
    β”‚   β”œβ”€β”€ source/
    β”‚   β”‚   β”œβ”€β”€ source.go               # Source interface + Mock (runs anywhere)
    β”‚   β”‚   β”œβ”€β”€ live.go                 # Live: squeue/scontrol + jobβ†’nodeβ†’GPU attribution
    β”‚   β”‚   β”œβ”€β”€ slurm.go                # squeue/scontrol output parsing
    β”‚   β”‚   β”œβ”€β”€ pending.go              # Slurm reason-code β†’ plain-English translator
    β”‚   β”‚   └── telemetry.go            # nvidia-smi GPU telemetry
    β”‚   β”‚
    β”‚   └── tui/
    β”‚       β”œβ”€β”€ app.go                  # Bubble Tea model: poll, fetch, keys
    β”‚       └── view.go                 # Lip Gloss rendering: heatmap + jobs panel
    β”‚
    β”œβ”€β”€ go.mod
    β”œβ”€β”€ README.md
    └── LICENSE

The Source interface is the whole seam:

  • Mock β€” deterministic demo data, runs anywhere, no Slurm needed.
  • Live β€” real squeue/scontrol, plus nvidia-smi telemetry when available (DCGM next).
  • The TUI never knows the difference.

πŸ—ΊοΈ Roadmap

squint grows up one rung at a time β€” each earns the right to the next.

  • L0 β€” Observe (here) β€” read-only GPU-aware TUI.
  • L1 β€” Act β€” cancel / hold / requeue from the TUI; job-done desktop & Slack pings.
  • L2 β€” Declare β€” clean job specs with pre-submit validation; squint rerun <jobid>.
  • L3 β€” API β€” a friendly daemon over Slurm with a real job-state event stream.

It also emits Prometheus metrics, so it can feed a longer-term observability stack rather than replace one.


Contributing

Issues, ideas, and pull requests are welcome.

If Squint helped you understand your cluster a little faster, consider giving the project a ⭐.


License

Β© 2026 Hitesh Kumar Sahu Β· Licensed under Apache 2.0

Releases

Packages

Contributors

Languages