DODAS_CMS: Dynamic On Demand Analysis Services

Use Case overview: 

CERN does not have the computing or financial resources to crunch all of the data on site, so in 2002 it turned to grid computing to share the burden with computer centres around the world. Recently cloud computing has come to public attention due to the promise of providing as much computing power as people require, simplifying management and reducing TCO. Main goal is to demonstrate that it is possible to extend computing centres outside local computing facilities in order to be able to scale as much as the experiment requires, either during normal periods or when approaching an important deadline. The Compact Muon Solenoid (CMS) is a generalpurpose detector at the Large Hadron Collider (LHC). It has a broad physics programme ranging from studying the Standard Model (including the Higgs boson) to searching for extra dimensions and particles that could make up dark matter.

Electronic circuits record the passage of each particle through a detector as a series of electronic signals, and send the data to the CERN Data Centre (DC) for digital reconstruction. The CMS experiment is one of the largest international scientific collaborations in history, involving 4300 particle physicists, engineers, technicians, students and support staff from 182 institutes in 42 countries.



This use case is based on Dynamic On Demand Analysis Services (DODAS) which is designed to provide a high level of automation in terms of provisioning, creating, managing and accessing a pool of heterogeneous computing and storage resources, by generating clusters on demand for the execution of HTCondor workload management system. DODAS allows to seamlessly join the HTCondor Global Pool of CMS to enable the dynamic extension of existing computing resources. A benefit of such an architecture is that it provides high scaling capabilities and self-healing support that results in a drastic reduction of time and cost, through setup and operational efficiency increases.

The Challenge

The principal challenges for the jobs we want to run on the resources managed through DODAS are

  • »» simple instantiation via command line a large number of machines
  • »» the resources allocated must be connected to the GEANT research networks to avoid bottlenecks when connecting to local computing resources to read or write data
  • »» the computing efficiency must be comparable to that of our local resources
  • »» simple user access not requiring new credentials and federated via eduGAIN.

Benefits and impact

CMS is one of the LHC experiments in the WLCG collaboration. The Worldwide LHC Computing Grid (WLCG) project is a global collaboration of more than 170 computing centres in 42 countries, linking up national and international grid infrastructures. Demonstrating that HNSciCloud can satisfy the requirements of LHC experiments has an enormous potential impact on cloud adoption in the particle physics environment. It could also be a driver for non-LHC experiments that do not have the human resources to move their computing on the cloud, by providing guidelines and best practices in order to simplify this process.

Tasks of the cloud access interface:
  1. individual user accounts and user groups
  2. budget allocation and usage monitoring with cut-off per group
  3. startup and shutdown of cloud clusters
  4. file transfer between local and cloud clusters
  5. login and job startup on cloud cluster
  6. solutions must be cost efficient
  7. network access must provide 5MB/s per job
  8. every job should be able to access 8 cores, at least 2GB RAM and 100GB of disk space










Supporting organization: 

The National Institute for Nuclear Physics (INFN) is the Italian research agency dedicated to the study of the fundamental constituents of matter and the laws that govern them, under the supervision of the Ministry of Education, Universities and Research (MIUR). It conducts theoretical and experimental research in the fields of subnuclear, nuclear and astroparticle physics. All of the INFN’s research activities are undertaken within a framework of international competition, in close collaboration with Italian universities on the basis of solid academic partnerships spanning decades. Fundamental research in these areas requires the use of cutting-edge technology and instruments, developed by the INFN at its own laboratories and in collaboration with industries.




Use Case: 
High Energy Physics

Who's behind HNSciCloud?

HNSciCloud Pre-Commercial Procurement is driven by ten leading research organisations from across Europe.