Cloud vendors’ commercial models poorly serve scientists, forcing them to struggle for value amid tightening budgets, according to research.
Modern science - from bioinformatics to astrophysics - depends heavily on sophisticated computer modeling, yet cloud providers’ business-focused models clash with how scientific projects consume computing resources, argue Vanessa Sochat and Daniel Milroy, both post-doctoral researchers in computing at Lawrence Livermore National Laboratory.
“Standard business models based on commercial traffic call for persistent services and long-term discounts for purchasing resources. In contrast, scientific runs are typically short and infrequent. A scientist might need a cluster with specialized, high-precision hardware a few times a month to run a large si…
Cloud vendors’ commercial models poorly serve scientists, forcing them to struggle for value amid tightening budgets, according to research.
Modern science - from bioinformatics to astrophysics - depends heavily on sophisticated computer modeling, yet cloud providers’ business-focused models clash with how scientific projects consume computing resources, argue Vanessa Sochat and Daniel Milroy, both post-doctoral researchers in computing at Lawrence Livermore National Laboratory.
“Standard business models based on commercial traffic call for persistent services and long-term discounts for purchasing resources. In contrast, scientific runs are typically short and infrequent. A scientist might need a cluster with specialized, high-precision hardware a few times a month to run a large simulation,” the pair’s paper says.
“In the case of larger scientific simulations, cost models based on saving money with preemptible instances can be risky, as the failure of one instance can lead to failure of the entire simulation because of the rigidity of Message Passing Interface (MPI),” it adds
The researchers argue that while businesses can plough income back into future cloud resources on an ongoing basis, scientific projects can be finite, depending on the grand situation. Meanwhile, institutions which are home to scientific projects are unlikely to have a strategy for getting better deals out of cloud providers.
Grant-funded researchers can buy cloud resources for specific projects, but they typically can’t guarantee repeat purchases or influence how their institutions spend on computing infrastructure more broadly. When vendors offer credits to support research groups, they’re investing in what they hope will become profitable long-term partnerships. These expectations, however, often go unmet — research groups simply don’t wield enough influence over institutional procurement to deliver the sustained business vendors anticipate.
- Unhappy with the cloud costs? You’re not alone
- Developer pockets $2M in savings from going cloud-free
- When hyperscalers can’t safeguard one nation’s data from another, dark clouds are ahead
- DoD spins up supercomputer to accelerate biothreat defense
Even the cloud’s promised elasticity and “on-demand” resources can fail scientists at critical moments when the compute turns out not be available.
While cluster creation is reasonably quick… problems arise when capacity is not available. “In a recent performance study, the inability to meet the minimum number of required resources incurred a charge of $4,000 while waiting for nodes that were never allocated. Vendors charging for idle resources is not intentional, but results from the deficiency of the cost and allocation models,” the paper states.
The researchers urge collaboration between cloud providers and scientists to develop models that serve both profitability and discovery.
“If the cloud could provide stronger guarantees about when work can occur, scientists can accommodate these future resources. We must advocate for integrated cost models that satisfy market needs without leaving scientific discovery behind,” the paper concludes.