Author name: Alan O'Cais

Code for Thought is a podcast for the growing number of researchers & scientists who code, with episodes in English, German and French. In the latest episode of the podcast, October 15 2024, MultiXscale project members Kenneth Hoste and Alan O’Cais discuss the European Environment for Scientific Software Installations (EESSI) and how it can help make scientists more productive…and reduce the technical burden of working with HPC resources. Click here to access the episode.

EESSI’s CI workflows are available on GitHub Actions and as a GitLab CI/CD component. Enabling this is as simple as adding EESSI’s CI to your workflow of choice, giving you access to the entire EESSI software stack optimized for the relevant CPU architecture(s) in your runner’s environment. If you are developing an application on top of the EESSI software stack, for example, this means you don’t need to invest heavily in configuring and maintaining a CI setup: EESSI does that for you so you can focus on your code. With the EESSI CI workflows you don’t have to worry about figuring out how to optimize build and runtime dependencies as these will be streamed seamlessly to your runner’s environment. Using the CI component in GitLab To showcase this, let’s create a simple R package that just outputs a map of the European Union and Norway, and colours the participating countries in the MultiXscale CoE. We’ll make a package eessirmaps that relies on popular R packages ggplot2, sf, and rnaturalearth to render and save this map. Installing GIS tools for R can be somewhat cumbersome, which could become trickier if it has to be done in a CI environment. This is because sf requires system packages libgdal-dev and libproj-dev, which would add yet another step, complicating our CI workflow. Thankfully, EESSI makes a lot of the packages dependencies available to us from the start, as well as a fully functioning version of R, and the necessary package dependencies to boot! As far as setup goes, this results in a simple CI workflow: Note how we simply include the EESSI GitLab CI component and set up a blank directory for our user R libraries. Remember, because of EESSI, the environment that you develop in will be exactly the same as the one the CI is run in. Apart from the rnaturalearthdata R package, all the other dependencies are taken care of by the R-bundle-CRAN/2023.12-foss-2023a EESSI module. This is true for the system and R package dependencies. Then we simply have to install our package to the CI environment and call the multixscale_map() function to produce the plot, which is uploaded as an artifact from the CI environment. We can then retrieve the artifact archive, unpack it and obtain the map.

Alan O ‘Cais presented MultiXScale and EESSI as part of the Nordic Industry Days 2024 – Supercomputing the gateway to AI, which took place 2-3 September 2024 in Copenhagen. It was a collaboration between the EuroCC competence centres in Denmark, Norway, Finland, Iceland, and Sweden, as well as Dansk Industri. More on the agenda and topics can be found here https://enccs.se/events/nordic-indust….

EESSI has been nominated for the HPCwire Readers’ Choice Awards 2024, in the “Best HPC Programming Tool or Technology” category. You can help us win the award by joining the vote. To vote, you should: 1) Fill out and submit the form to register yourself as an HPCWire reader and access your ballot; 2) Access your ballot here; 3) Select your favorite in one or more categories; 4) Submit your vote by filling in your name, organisation, and email address (to avoid ballot stuffing), and hitting the Done button. Note that you are not required to vote for all categories, you can opt for only voting for one particular nominee in only one of the categories. For example, you could vote for European Environment for Scientific Software Installations (EESSI) in category 13: Best HPC Programming Tool or Technology.

Thanks to the work developed under MultiXscale CoE we are proud to announce that as of 22 July 2024, Extrae v4.2.0 is available in the EESSI production repository software.eessi.io, optimized for the 8 CPU targets that are fully supported by version 2023.06 of EESSI. This allows using Extrae effortlessly on the EuroHPC systems where EESSI is already available, like Vega and Karolina. It is worth noting that from that date Extrae is also available in the EESSI RISC-V repository risv.eessi.io. Extrae is a package developed at BSC devoted to generate Paraver trace-files for a post-mortem analysis of applications performance. Extrae is a tool that uses different interposition mechanisms to inject probes into the target application so as to gather information regarding the application performance. It is one of the tools used in the POP3 CoE. The work to incorporate Extrae into EESSI started early in May. It took quite some time and effort but has resulted in a number of updates, improvements and bug fixes for Extrae. For full details of the work, see the extended EESSI blog post.

The International Post-Exascale Project (InPEx) is a pioneering initiative bringing together the brightest minds in the field of high-performance computing, from researchers and engineers to policy organizations and funding bodies. It’s workshops, accompanied by workgroups of experts dedicated on critical topics for Exascale, are designed to foster international collaboration and co-design, essential in the journey towards and beyond Exascale computing. The technical manager of MultiXscale (AlanO’Cais) was invited to participate in the 2024 InPEx Workshop, and in particular contributed to the “Software production and management” topic of the workshop. Lively discussions were held as part of that topic with a general agreement that creating a software installation description format could be very useful to encourage collaboration between the various software installation tools being used “in the wild” (at this particular workshop EasyBuild/EESSI, Spack and Guix were represented).

One of the milestones that we have in MultiXscale is to be able to run the EESSI test suite on at least two different architectures. In the context of EuroHPC that means running on different partitions of the available EuroHPC Supercomputers.  Our initial effort focused on getting the test suite portable between two different supercomputers: Karolina and Vega (the CPU partitions of both are a Zen2 architecture). More recently we have spent time getting the same test suite working on a more “exotic” architecture, the ARM A64FX architecture of Deucalion (currently in pre-production). This has some additional complications for us as CernVM-FS is not yet natively available there. The performance/scalability plots we have measured for the ESPResSo application of MultiXscale are below. For full technical details of how we carried this out (and how you can repeat it for yourself!), please take a look at the EESSI blog post on this milestone. We look forward to reporting increased performance for ESPResSo in the future as we implement some of the ideas suggested in our recent deliverable on this topic.