How Can We Train Deep Learning Models Across Clouds and Continents? An Experimental Study
Abstract
This paper aims to answer the question: Can deep learning models be cost-efficiently trained on a global market of spot VMs spanning different data centers and cloud providers? To provide guidance, we extensively evaluate the cost and throughput implications of training in different zones, continents, and clouds for representative CV, NLP, and ASR models. To expand the current training options further, we compare the scalability potential for hybrid-cloud scenarios by adding cloud resources to on-premise hardware to improve training throughput. Finally, we show how leveraging spot instance pricing enables a new cost-efficient way to train models with multiple cheap VMs, trumping both more centralized and powerful hardware and even on-demand cloud offerings at competitive prices.
Read the paper
Related studies
- Understanding the factors influencing fall armyworm (Spodoptera frugiperda J.E. Smith) damage in African smallholder maize fields and quantifying its impact on yield. A case study in Eastern Zimbabwe
- Alarming situation of emerging H5 and H7 avian influenza and effective control strategies
- Livestock and food security: vulnerability to population growth and climate change
- Atmosphere–ocean oxygen and productivity dynamics during early animal radiations
- Characterization of the global distribution and diversified plasmid reservoirs of the colistin resistance gene mcr-9
- The global seroprevalence of Toxoplasma gondii in pigs: A systematic review and meta-analysis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.