GPU Day 2023

We would like to present a short introduction to the ALICE Analysis Facility and WSCLAB projects in our datacenter. After introducing the ALICE Analysis Facility, we are discussing the corresponding IT infrastructure design and capacity in detail.
Hardware components are aging, so monitoring is an important method to keep infrastructure healthy and to prolong cluster lifetime.
We selected a monitoring solution fitting for these environments. We created server types (worker node, storage) and defined entities in our monitoring system. In some cases, monitoring checks are just basic, others are advanced and some are even more complex to make sure we know the most important details in almost real time.
We created a separate VLAN network for monitoring in order to minimize interference with real worker node traffic.
Power consumption and electricity bills are important factors nowadays. In order to see the details, we are using a visualization solution for power usage statistics based on each rack.
To scale up the monitoring system efficiently, we are using automation tools for node preparation and installation.
Historical data is also very valuable, so we integrated a database solution into our monitoring workflow.
Our roadmap for future developments includes: continuous disk tests (S.M.A.R.T.), scheduled backup for monitoring data, proper alerting based on pre-defined warning and critical levels, iterative time-based optimization for running checks, HTCondor service monitoring, monitoring GPU RAM, GPU-utilization and temperature for GPULAB and smart alerting for complex cases.

Following up on the similarly named talk in 2021, the evolution of standards in HPC haven't slowed down a bit. If anything, research and adoption have accelerated. There is a fair amount to be excited over around Khronos standards and open technologies.

Following major upgrades of the ALICE detector during the second long shutdown, the data-taking capabilities increased by two orders of magnitude. Consequently, the ALICE distributed infrastructure was improved to enable analysis on exceptionally large data samples and to optimise the analysis process. This was carried out by developing new tools for Run 3, O2 and Hyperloop. The O2 analysis framework allows for distributed and efficient processing of the new amount of data, while the Hyperloop train system, fully integrated with O2, enables distributed analysis on the Grid and dedicated analysis facilities. Hyperloop was developed using Java, JavaScript and a modern versatile framework, React. This allowed the implementation of responsive and adaptable features facilitating the user experience. The new train system has already been in use since the start of Run 3, with extensive documentation available. This talk will discuss the requirements and the used concepts, providing details on the actual implementation. An overview of the current status of the analysis in Run 3 will also be presented.

Research infrastructures play a key role in large-scale and collaborative research. Each country should have its own network of these. In Hungary recently a list of the top 50 research infrastructures was put together in close connection with the researcher community. How can these help to participate in international collaborations? How can they be best utilized by the researchers? The presentation will provide insight into a current best practice in answering these questions.

Hastlayer (https://github.com/Lombiq/Hastlayer-SDK) is an open-source tool to increase the performance and decrease the power consumption of .NET applications by accelerating them with FPGAs. It converts standard .NET Common Intermediate Language (CIL) bytecode into equivalent VHDL constructs which can be implemented in hardware. The cloud-available Xilinx Alveo family of boards are supported for high-performance use-cases, as well as the Zynq 7000 family of FPGA SoC devices for low-power embedded systems like drones and nanosatellites. In this talk, we'll introduce Hastlayer and how it can be used, our results showing up to 2 orders of magnitude speed and power efficiency increases, as well as the collaboration partners we seek from academia and other industry players.

The talk will highlights some works of the Department of Computational Sciences: new advances in the analysis of the interaction of systems, in particular causal discovery. Towards the end a new initiative, the foundation of the Society of Scientific Computing will be announced.

We give a short introduction to a new Hungarian online community dedicated to GPU programming. Our aim is to connect developers and researchers speaking Hungarian to get to know each other and be able to discuss, learn and share expertise in GPU technologies, programming, and related fields, including computer graphics, education, and more.

Recently, employing photonic quantum computers for machine learning purposes has gotten more attention, hence the need for efficient simulation of photonic quantum machine learning is key in this research area. To ease simulation, automatic differentiation of photonic quantum circuits is essential, for which a Piquasso implementation is presented.

Cystic fibrosis (CF) is a hereditary disorder caused by mutations in the cystic fibrosis transmembrane conductance regulator (CFTR) protein, resulting in the loss of function of this ion channel in the plasma membrane of epithelial cells. CFTR is composed of two transmembrane domains and two intracellular nucleotide-binding domains (NBDs), with the disordered R-domain wedged between them. Phosphorylation of the R-domain is a prerequisite for the opening of the channel, causing it to dissociate from the NBDs. Understanding the mechanism of different mutations requires knowledge of the protein structure. However, with over 350 known CF mutations, it is unfeasible to develop a treatment for all rare mutations. To address this challenge, we propose a general molecular-level intervention to facilitate the activation of the channel. By designing a "miniprotein" that binds to the R-domain and inhibits its insertion between the two NBDs, an increase in CFTR activity would become possible. As a first step of this approach, in silico methods were used to model the structure of the complexes formed between the phosphorylation sites of the R-domain and the nucleotide-binding domain 1 (NBD1). Protein structure predictions were performed using AlphaFold-Multimer neural network, which is one of the currently available best methods for predicting the structure of protein complexes. Selected complex structures modeled by AlphaFold-Multimer were studied using molecular dynamics simulations. The models provide information about possible conformations of R-domain and NBD-1 complexes, thus contributing to the selection of target sites for the redesigned “miniproteins”.
Acknowledgements: This research was funded by the ÚNKP-22-2-III-BME-239 New National Excellence Program of the Ministry for Innovation and Technology, NRDIO/NKFIH grant numbers K127961, K137610, and Cystic Fibrosis Foundation (CFF) grant number HEGEDU20I0.

We have investigated the synchronization transition of the Shinomoto-Kuramoto model on networks of the fruit-fly and two large human connectomes. This model contains a force term, thus is capable of describing critical behavior in the presence of external excitation. By numerical solution we determined the crackling noise durations with and without thermal noise and showed extended non-universal scaling tails characterized by the exponent 2 < τt < 2.8, in contrast with the Hopf transition of the Kuramoto model, without the force τt = 3.1(1). Comparing the phase and frequency order parameters we have found different synchronization transition points and fluctuation peaks as in case of the Kuramoto model, related to a crossover at Widom lines. Using the local order parameter values we also determined the Hurst (phase) and β (frequency) exponents and compared them with recent experimental results obtained by fMRI. We show that these exponents, characterizing the auto-correlations are smaller in the excited system than in the resting state and exhibit module dependence.

In this talk, I will present my experiences building a range of Computational Fluid Dynamics applications on LUMI that use a variety of parallelization approaches, such as HIP, OpenMP offload, and SYCL. Given the relatively new AMD GPU platform, I discuss issues and limitations that affect development, debugging, and performance analysis. I show a contrasting analysis of performance as a function of compilers and applications. For a few applications, I also show and discuss strong and weak scalability up to 2000 GPUs and a problem size of half a trillion cells. Overall the MI250X GPU is a strong competitor to the NVIDIA A100 GPU given its raw performance. However, there are still significant usability issues, making development challenging.

Mirages (fata morgana) can appear for example above water bodies in our natural environment, when certain atmospheric conditions are satisfied, e.g. the water is warmer than the ambient air, creating a near-surface temperature-gradient resulting in a quick variation of the refractive index of air. The size of the visible mirage carries information on the overall difference in temperature. Extracting it requires one to have a complex deconvolution based on various environmental parameters, which poses a challenge in the case of a general application. Here, we developed a machine learning-based tool to extract the temperature difference information from photographs taken of mirages.

Laser-induced fusion with simultaneous volume ignition, a spin-off from relativistic heavy ion collisions, was suggested, where implanted nano antennas regulate the light absorption in the fusion target [1,2]. Recent studies of the resilience of the nanoantennas in vacuum [3] and UDMA-TEGDMA medium [4] concluded that the lifetime of the plasmonic effect is longer in the medium, however, less energy was observed in the UDMA-TEGDMA copolymer, due to the smaller resonant size of the gold nanoantenna. Here we show how the plasmonic effect behaves in an environment fully capable of ionization, surrounded by Hydrogen atoms. We treated the electrons of gold in the conduction band as strongly coupled plasma. The results show that the protons close to the nanorod's surface follow the collectively moving electrons rather than the incoming electric field of the light and this screening effect is also dependent on the laser intensity.

[1] Csernai LP, Csete M, Mishustin IN, Motornenko A, Papp I, Satarov LM, et al. Radiation-Dominated implosion with flat target. Phys Wave Phenomena (2020)

[2] Csernai LP, Kroó N, Papp I. Radiation dominated implosion with nano-plasmonics. Laser Part Beams (2018)

[3] Papp I, Bravina L, Csete M, Kumari A, et al. Kinetic model evaluation of the resilience of plasmonic nanoantennas for laser-induced fusion. PRX Energy (2022)

[4] Papp I, Bravina L, Csete M, Kumari A, Kinetic model of resonant nanoantennas in polymer for laser-induced fusion, Frontiers in Physics, 11, 1116023 (2023)

Closed quantum systems are described by a deterministic, linear evolution of their quantum states. Observation of a system breaks this rule: we gain information and the description becomes probabilistic. Provided we have multiple copies of a system in the same quantum state, we can design a protocol consisting of a joint unitary operation on all of them and a subsequent measurement on all but one of the outputs. Accepting only certain measurement outcomes, we arrive, in general, at a nonlinear quantum transformation on the remaining system. These nonlinear quantum transformations play an important role in quantum state distillation, aiding quantum communication schemes. We consider such iterated nonlinear protocols for qubits, involving a generalized CNOT, a measurement, and a single-qubit Hadamard gate processing n inputs in identical quantum states. In our work, we study the asymptotic properties of this special class of iterated nonlinear maps using numerical methods. We determine the fixed points, and relevant fixed cycles and show the presence of a universal phase transition related to the disappearance of the fractalness of borders of different basins of attraction. The transition point is marked by a repelling fixed point, which has an increasing purity, tending to 1 as we increase the order n of the map. The generalized Julia set, defined as the set of border points between the basins of attraction for fixed purity initial states, has a constant fractal dimension as a function of the purity, above a critical value. Remarkably, we found that this fractal dimension is universal in this family, it is independent of the order n of the map.

The GigaBit Transceiver (GBT) and the low power GBT (lpGBT) link architecture and protocol have been developed by CERN for physical experiments in the Large Hadron Collider as a radiation-hard optical link between the detectors and the data processing FPGAs (https://gitlab.cern.ch/gbt-fpga/). This presentation shows the details of how to implement a large array of GBT/lpGBT links (up to 48 x 4.8/5.12/10.24 Gbps) on a custom data acquisition board based on the Intel Arria 10 FPGA.

GUARDYAN (GpU Assisted Reactor Dynamic ANalysis) is a GPU-based dynamic Monte Carlo code developed at the Budapest University of Technology and Economics. As an emerging calculation technology, dynamic Monte Carlo computes the neutron population evolution by calculating the direct time dependence of the neutron histories in multiplying systems. GUARDYAN targets to solve time-dependent problems related to fission reactors with the main focus on simulating and analyzing short transients. The key idea of GUARDYAN is a massively parallel execution structure making use of advanced programming possibilities available on CUDA-enabled GPUs. The GUARDYAN has been recently coupled with the SUBCHANFLOW subchannel thermal-hydraulics solver to carry out dynamic calculations with thermal-hydraulics feedback. The coupling is compared against other dynamic Monte Carlo-based neutron transport codes (SEPRENT, TRIPOLI-4) with SUNCHANFLOW coupling in TMI 3x3 transient scenario. The results of the comparison showed an exact match.

Some applications of Monte Carlo simulation require the generation of a large number of small grids of bits (e.g. tiles or boards of 32 by 32 bits) with a given number of bits set to one, the rest is set to zero. Conventional sequential algorithms generate them by randomly selecting empty sites until the given number of set sites is reached, which is not a client solution. In this talk, we present a solution that uses the special capabilities of either CPU or GPU to generate them at a high speed. In the CUDA implementation, the tiles are generated warpwise (with the use of warp shues) thereby eliminating the loop divergence of threads. This realization allows us also to reduce the use of shared memory, the computation is performed in registers, and only the PRNG states are stored in shared memory.

The Artificial Intelligence National Laboratory Hungary (MILAB) is a research consortium of 11 institutions in Hungary. MILAB researchers use GPUs to accelerate the training of deep neural networks for machine vision tasks in object detection, surveillance, and medical imaging applications. We train and evaluate deep neural networks for natural languages processing tasks, such as sentiment analysis, NER, and text summarization, in particular for the development of the HuSpaCy system. We also use GPUs to train and evaluate deep reinforcement learning models for robotics applications, such as autonomous navigation and manipulation.

Proton therapy is a promising method for cancer treatment, where the energy deposit may be focused on the cancerous cells, hence providing a much faster method with fewer side effects than traditional radiation therapy. However, a good diagnostic tool, a "CT" with protons is missing to reach the desired accuracy. Here, we model a new toolchain, where we generate events, detected by the planned detector for various phantoms, identifying trajectories and the energies and direction of the charged particles behind the phantom. The information extracted is the input for image reconstruction algorithms. Due to the complexity of the identification we designed neural network-based models to speed up the process.

The application of deep learning in gigapixel whole slide image analysis has shown promising results in terms of accuracy and efficiency compared to traditional image analysis techniques. Transformer-based models as the current state-of-the-art algorithms are capable of identifying and classifying various structures and patterns within the tissue, providing insights into the underlying pathology and helping in the diagnosis and treatment of diseases. However analyzing pathological whole slide images with vision transformers requires specialized hardware with high computational power, high-speed memory, and interconnects. High-performance GPUs and TPUs are commonly used for this purpose, as they are designed specifically for processing large amounts of data in parallel. In this presentation, we demonstrate an efficient pipeline for image preprocessing and the application of deep learning models at TB scale for digital histopathology.

Position control is a fundamental task of mechatronics, where the goal is to drive an object to a desired position or maintain its trajectory on a given path. The digital implementation of control leads to a sampled-data system, resulting in piecewise smooth dynamical properties due to the sudden changes of the control input at the sampling instants. In the controlled mechanical system, dry friction causes another nonsmooth effect because the Coulomb-friction force has a jump discontinuity at motion reversals. These two phenomena appear simultaneously in a mechatronic system; however, the sampling and friction-caused switchings are independent, making the analysis difficult.

In this work, the position control of a single-degree-of-freedom (1DoF) system in the presence of friction is presented with an iterated function system (IFS) based approach. An IFS consists of a finite number of mappings on a metric space primarily used to describe fractals. We present that the discrete-time model of the discussed 1DoF nonsmooth system can be transformed into an IFS on the system's state space.

The numeric evaluation of the IFS can be challenging because a large set of initial states must be examined to describe the system accurately. To address this problem, we present a GPU-based simulation implemented in OpenCL to parallelize the function evaluations.

Supported by the ÚNKP-22-3-1-BME-336 New National Excellence Program of the Ministry for Culture and Innovation from the source of the National Research, Development and Innovation Fund

In HPC, a common method is decomposing large equation systems into a batched problem of small equation systems. Such small systems are Tridiagonal/pentadiagonal matrix systems frequently arise in finite-difference methods for solving multi-dimensional PDEs in various applications. Such systems are, for instance, present in computational fluid dynamics (CFD) for flow solvers based on implicit high-order finite-difference schemes, like the Alternating Direction Implicit (ADI) method. In this talk, I will present our initial results of developing a scalable batch-pentadiagonal solver library for large CPU and GPU clusters for ADI applications. We use the hybrid Thomas-Jacobi and Thomas-PCR algorithms using the Thomas algorithm locally to create a reduced system and a distributed solver on the reduced system. We show the cost difference between the local communications of the approximate solver (Jacobi) and the one-to-one communications with increasing distance of the exact solver algorithm (PCR). We show the scaling behavior of these algorithms on LUMI up to 1024 GPUs in one direction.

Examining the dynamics and spatial properties of individual stars and stellar populations in satellite galaxies surrounding the Milky Way through the lenses of galactic archeology is crucial to understand galactic evolution. By gaining insight into this process, it can provide us with valuable information about the large-scale and long-term characteristics of both ordinary matter and dark matter. Thorough studies can utilize both photometric and spectroscopic observations that complementarily support each other. Here we focused on the spectroscopic aspect of this problem, by investigating how well autoencoder-based neural networks (AEs) perform in the processing and analysis of stellar spectra. We show that AEs are capable of learning the characteristics of a stellar spectrum to a considerable extent, even in noisy and low 𝑆/𝑁 conditions. Moreover, they can be extremely valuable tools during the preprocessing stage of spectroscopic data analysis, where otherwise finding the black body continuum of a noisy spectrum is challenging.