History-First Reliability-Gated Fusion for Sampled-Peak System and GPU Memory Utilization Prediction in HPC Clusters
Abstract
Abstract Predicting sampled-peak resource utilization aids capacity planning in high-performance computing (HPC) clusters, but scheduler requests do not capture changes in executable logic and history-only predictors may fail as workloads change. We present a history-first reliability-gated cascade for predicting system-memory and GPU-memory (VRAM) utilization ratios from executable scripts, submissiontime requests, pseudonymous user identity, and event-time history. A LoRA-adapted Qwen2.5-Coder-1.5B-Instruct backbone encodes scripts with a 4096-token context. Before inference, a resource-specific classifier bypasses the encoder when eligible exact-script history is judged reliable; otherwise, Qwen is invoked and, when history exists, a post-inference regressor blends its output with robust history under a dispersion-aware trust region. History includes only jobs completed before submission; cascade fitting and policy selection use chronologically separated validation blocks. We evaluate 19,321 jobs retained after completion-boundary purging from an 11.5-month production trace across five training seeds. The cascade achieves memory MAE / RMSE / Acc@10% of 0.0818 ± 0.0008 / 0.1777 ± 0.0032 / 79.46% ± 0.45 percentage points (pp) and VRAM values of 0.0823 ± 0.0017 / 0.1634 ± 0.0004 / 77.11% ± 1.33 pp (mean ± sample standard deviation). Relative to Hierarchical Same-Script Last-1, mean MAE decreases by 4.83% for memory and 13.70% for VRAM. Paired day-block bootstrap intervals…
Read the paper
Related studies
- Diarylethenes for Memories and Switches
- Long-Term Potentiation and Memory
- Social Memory Studies: From “Collective Memory” to the Historical Sociology of Mnemonic Practices
- The Cognitive Neuroscience of Working Memory
- Memory aging and brain maintenance
- Effector and Memory CTL Differentiation
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.