User Tools

Site Tools


eso_pipelines

This is an old revision of the document!


ESO pipelines

ESO offers pipeline software for their instruments (see https://www.eso.org/sci/software/pipelines/) However, installing this software directly on to the system, is usually not possible, since they don't support every version of Fedora or RedHat.

We have two methods to make this software available to users:

  1. a local install on a desktop, for a specific pipeline (if supported) and
  2. through apptainer containers

Local install

Until early 2025, this was the only available method, which would work in most cases, but not all. If it works, it could be slightly easier and perhaps faster.

If you want a pipeline installed on your Linux desktop system, please request it through the helpdesk form and specify on which computer you are, and which pipeline(s) you want to have installed.

Apptainer

We have recently adapted ESO's apptainer setup to work with our system. Apptainer is a container system, were an application runs in a specially tailored environment, possibly with tools from another Linux system, and mostly isolated from the actual operating system.

The main benefit is, that software that is supported on any Linux version, can be supported in this way. In our case this means, that the softwar eis now available on the desktops, as well as on compute nodes and the vdesk cluster.

There are however a couple of drawbacks: if will be less efficient with cp, memory and disk space, and you may have to pass some additional options to bypass parts of the isolation, in order to make data available to programs running in the container.

We currently have a couple of apptainers in /software/eso, the one called pipelines.sif has a large selection of pipelines (see below). To load the apptainer and start a command from it:

apptainer exec /software/eso/pipelines.sif [command]

e.g.

apptainer exec /software/eso/pipelines.sifesoreflex molecfit

The apptainer always makes the current directory and the user's home available to commands running inside it. If you need access to data in other directories, these can be bound to the apptainer to make them available:

apptainer exec --bind /my/data/dir:/data /software/eso/pipelines.sif esoreflex gravity

This binds /my/data/dir on the real system to a path /data inside the apptainer.

ESO's site about apptainer has some more information about usage and options, see https://www.eso.org/sci/software/apptainer.html (skip the part about installation if you run on Sterrewacht workstations, that has already been done for you)

eso_pipelines.1743772928.txt.gz · Last modified: by jansen