MLOps and LLMOps as AI Production Engineering: An Evidence-Centric Framework for Controlled Change in Production AI Systems
Abstract
Production AI systems are socio-technical systems whose behavior emerges from interacting states across data, code, execution environments, model or provider state, prompts, retrieval context, tools, policies, evaluators, infrastructure, and human intervention. This paper develops an evidence-centric framework that treats Machine Learning Operations (MLOps) and Large Language Model Operations (LLMOps) as disciplines for controlled change in such systems. The framework integrates evidence from recent MLOps architecture studies, LLMOps research, generative-AI evaluation literature, current platform documentation, security guidance, and AI governance frameworks. Five contributions are developed: (1) a multidimensional architecture lens that extends infrastructure-friction versus operational-yield analysis into an explicit trade-off space; (2) executable provenance, which defines reproducibility as reconstruction of materially causal execution state; (3) a composable MLOps/LLMOps architecture separating evidence, evaluation, orchestration, compute, knowledge, serving, telemetry, policy, and human control; (4) evaluation as a release-control mechanism in which the evaluator is itself versioned; and (5) a capability-coverage model for organizational readiness. The paper derives eight falsifiable hypotheses and specifies a controlled benchmark protocol spanning supervised ML, parameter-efficient fine-tuning, retrieval-augmented generation (RAG), and tool-using agents. An…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.