Parallelization of multipacting simulation codes

Progress in particle physics has traditionally been achieved by a symbiosis of experiments at the energy and intensity frontiers and model-building. This has led to the current situation of the Standard Model (SM) representing our best knowledge of particle physics, but which leaves a number of open questions to be resolved. These include the composition of dark matter, the dynamic origin of the observed matter–antimatter asymmetry, and the fine-tunings needed for a light Higgs and no observable CP violation in the strong sector. Furthermore, the SM is itself known not to be complete by the existence of massive neutrinos and offers no explanation for the smallness of their masses. There thus exist many problems requiring hitherto unobserved particles and physics processes and, consequently, appropriate facilities are needed.

ESS Bilbao is a strategic center of international reference in Neutronic Technologies that provides knowledge and added value through in-kind contributions to the European Spallation Source, a singular scientific infrastructure that uses nuclear spallation, a process in which neutrons are liberated from heavy elements by impacting them with high energy protons. This source of neutrons is critical for many domains of research spanning the realms of physics, chemistry, geology, biology and medicine. The ESS is being built in the Swedish city of Lund through an exceptional international effort and will support research on a wide range of topics.

The team at ESS Bilbao also works on the construction of their own particle accelerator, which will facilitate regional research and development in high energy physics. In order to accelerate protons to near light speeds, these particle accelerators are made up of a wide range of components to deal with progressively higher energy particles. In addition, the design includes many electronics for monitoring, control and diagnostics of the beams.

An important step when designing and building these components is to simulate the behaviour of the particles inside them to ensure they behave as expected and that no pathological effects will occur in the accelerator component. Now, a team of researchers presents ¹ the improvement and optimization of a finite differences particle simulation program used by ESS Bilbao to design and manufacture some essential components. In this work, the scientists have worked with one of the programs used for these simulations, which is mainly used to analyse a crucial physical process called multipacting.

The multipactor effect or, simply, multipacting, is an avalanche discharge of electrons generated by the synchronization between the intense alternating electric field used to accelerate the particles and the emission of secondary electrons on surfaces exposed to electrons accelerated by this field in vacuum conditions. It begins when an electron that is inside one of the mentioned components collides with one of the surfaces of the component and as, a consequence of the collision, it gets absorbed by the surface, but the energy of the impact rips out other electron(s) from the surface. These electrons, in turn, can collide and pull out even more electrons. If this event is repeated, a cloud of electrons can be created, forming what is called multipacting or multipactor effect.

Analysing the conditions under which this avalanche of electrons is most likely to form in each component can be of utter importance because during regular operation it is essential to prevent it from happening. Otherwise, it could influence the particles in the accelerator, which in turn affects the experiments that are being conducted, or even worse, it could damage equipment. However, on some occasions it might be desirable to produce the avalanche on purpose under controlled conditions, as it can be used for maintenance work such as, for instance, fine-polishing the surface of a component.

Nevertheless, if experiments are to be as reliable as possible, it is needed to simulate very large numbers of electrons and also perform numerous repetitions. For this reason, the execution time of a program of these characteristics that simulates the path of each one of the electrons in series can be very high. The researchers have developed ways to parallelize the execution of the original program, so it can take advantage of the computing resources offered by the ATLAS supercomputer at the Donostia International Physics Center.

First, they developed a parallel version of the multipacting simulation and were able to accelerate the execution up to 35 times with 48 or 56 cores. When this first parallelization effort was tried for multi-power simulations, they found that parallelism was severely limited with a maximum of 20 times speed-up. Thus, they introduced a new method to improve the parallelization efficiency for this second use case. This method uses a shared processor pool for all simulations of electrons (OnePool). OnePool improved scalability by pushing the speed-up to over 32 times.

While scalability did not seem to extend beyond that number of cores, the obtained acceleration is adequate since it translates hours into minutes and days into hours, hugely accelerating the design process. Moreover, the parallelization has enabled the simulation of components that before were hitting the CPU time quotas of the supercomputer.

Author: César Tomé López is a science writer and the editor of Mapping Ignorance

Disclaimer: Parts of this article may have been copied verbatim or almost verbatim from the referenced research paper/s.