# node_observations.csv

Times are local (America/New_York) ISO-8601, one sample every 5 minutes. Memory is MB.
A GPU is counted **idle** only if its node is not DOWN/DRAIN/FAIL/MAINT and not in a
reservation. `gpu_idle_requestable` and `gpu_idle_unreachable` use one fixed definition (a
job needing 8 cores and 10.2 GB); the web page recomputes them from the raw columns for
whatever CPU count you choose. `cpu_free` and schedulable
memory are `CPUEfctv - CPUAlloc` and `RealMemory - AllocMem`; `FreeMem` is deliberately
not used, being OS-level and inclusive of reclaimable cache.

## Columns — one row per pmg1 GPU node per sample (26,505 rows)

| column | meaning |
|---|---|
| `timestamp` | when the sample was taken |
| `node` | Slurm node name |
| `gpu_type` | `a6000`, `h100`, `l40`, `l40s` |
| `partitions` | pipe-separated partitions the node belongs to |
| `state` | Slurm node state string, verbatim |
| `gpu_total` / `gpu_in_use` | GPUs configured / allocated to jobs |
| `gpu_idle` | idle and healthy (0 if the node is down) |
| `gpu_idle_requestable` | of those, how many a 1-GPU/8-core job could actually get |
| `gpu_idle_unreachable` | `gpu_idle` minus `gpu_idle_requestable` |
| `gpu_down` | free GPUs on a node that is down, drained or reserved |
| `cpu_total` / `cpu_in_use` / `cpu_free` | cores schedulable / allocated / free |
| `mem_total_mb` / `mem_in_use_mb` | `RealMemory` / `AllocMem` |
| `jobs_with_gpu` / `jobs_without_gpu` | running jobs on this node, by whether they hold a GPU |
| `cores_held_by_gpu_jobs` / `cores_held_by_nongpu_jobs` | cores those two groups hold |