Online Analysis Resources¶

In order to support near realtime analysis, computing resources from Maxwell can be reserved for a beamtime. These reserved resources are only available for the beamtime and also allow to access the beamline filesystem from the Maxwell node. User access is only possible with a specific user account via the use of SSH public keys.

The reserved resources can be used interactively and/or via sbatch SLURM jobs. For additional details about SLURM, have a look at the Maxwell space.

These resources are reserved and allocated via the startBeamtime script. See below on how to reserve and use the resources.

Resource Allocation via startBeamtime¶

Online resources can be allocated from the startBeamtime script, either with Pre-Start or without Pre-Start. Additionally, it is possible to add options for resource allocation, e.g. nodes with GPUs or dedicated nodes for the beamline.

See startBeamtime --help for all available options:

% startBeamtime --help
usage: startBeamtime  [-h] [--beamtimeId BEAMTIMEID] [--beamline BEAMLINE]
                      [--verbose] [--pre-start] [--online]
                      [--feature FEATURE] [--resources PARTITION] [--asapo]
                      [--archive]

optional arguments:
  -h, --help            show this help message and exit
  --beamtimeId BEAMTIMEID
                        beamtime-id of your experiment
  --beamline BEAMLINE   beamline that belongs to the beamtimeId
  --verbose             more verbose output
  --pre-start           BETA: Reserve Maxwell Online Resources
  --online              BETA: Reserve Maxwell Online Resources without Pre-
                        start
  --feature FEATURE     BETA: Specify additional node features, e.g. GPU for a GPU node
  --resources PARTITION
                        BETA: Specify resource location, either common or
                        dedicated
  --asapo               Initialize ASAP::O
  --archive             Enable Archiving for Industry beamtime

2 modes of resource reservation are available, with or without pre-start.

Resource Selection¶

The flags --resources PARTITION and --feature FEATURE allow you some basic resource selection.

If none of the flags are specified, a single CPU node is taken from the common resources.

With --feature gpu, you will get a single node with a GPU (NVIDIA P100 as of 2025-03-03).

Some beamlines have acquired Maxwell resources for beamtimes, which allows them to use more resources. With --resources dedicated, these resources will be used. The reservation in Slurm will use all available nodes from these dedicated resources. Use --nodes NUMBER to reduce this, if desired.

One notable exception are beamtimes with proposal type P aka Industrial Beamtimes. These beamtimes have to specify always --resources dedicated due to licensing constraints.

The --feature FEATURE supports the full feature syntax of Slurm. This allows a very granular selection of resources.

With Pre-Start¶

Pre-start is used before the beamtime starts. Storage resources are then only allocated on the core fileystem and a shared node from Maxwell will be allocated. This allows you to setup your scripts, pipelines etc. in advance of a beamtime.

See below how to start the beamtime

# startBeamtime --beamtimeId 12345678 --beamline p00 --pre-start
<snap>
2021-08-12 09:46:01,464 >  [Maxwell Online Analysis Resources]-----------------
2021-08-12 09:46:09,678 >  [Maxwell Online Analysis Resources]-----------------finished.
2021-08-12 09:46:09,685 >  Beamtime has been pre-started.
Maxwell Online Analysis Resources have been configured for preparation phase.

Storage resources were only created for the Core-Filesystem:
/asap3/petra3/gpfs/p00/2021/data/12345678

For information about the user account and associated SSH key, please check the
shared folder in the Core-Filesystem

Now you can access the beamtime via the core filesystem and start with preparation. Once the experiment and real beamtime starts, startBeamtime has to be called again:

# startBeamtime --beamtimeId 12345678 --beamline p00

This allocates the storage resources on the beamline filesystem and also mounts it on the Maxwell node. The reservation will be updated as well, so check beamtime-metadata-.json for changed reservation- and node names!

The SSH key is copied to the beamline filesystem in the shared folder. Any server with access to the beamline filesystem can now access the Maxwell resources as well.

Without Pre-Start¶

If you do not want to use the pre-start option, you can use the --online flag:

# startBeamtime --beamtimeId 12345678 --beamline p00 --online

This will skip the pre-start phase and start directly with the online phase.

Creating multiple reservations¶

If you have dedicated resources with enough nodes, you can create multiple reservations instead of a single reservation.

To use multiple reservations, use the --nodes NUMBER[:NUMBER[:NUMBER...]] flag. NUMBER specifies the number of nodes for the reservation, the colon is a delimiter and allows to specify additional reservations.

Specifying --nodes 2:2 would create 2 reservations, each with 2 nodes. See below for additional examples:

To reserve nodes with specific features, use --feature FEATURE[:FEATURE]. It follows the same rules as --nodes:

If you use multiple reservations, the name of the first reservation will be always beamtimeId. Any additional reservation will have an incrementing counter added to the reservation name beamtimeId_N, where N starts with 1 and is incremented. For a beamtime 12345678 with 2 reservations, the reservation names will be 12345678 and 12345678_1.

It's recommended to parse this information from the beamtime metadata file.

Using Allocated Resources¶

After you have started your beamtime with Online Resources, first check the Beamtime Metadata File. The onlineAnalysis key contains all relevant information:

• Name of the reserved node(s)
• Name of the reservation, required to use for jobs with sbatch
• Beamtime specific account name
• Relative path to the SSH private key

To start using the resources, you must use the account specified in userAccount and the private key from sshPrivateKeyPath:

# ssh -l<userAccount> -i <sshPrivateKeyPath> <reservedNode>

The key is available on both, the beamline- and core filesystem. On the reserved node, the beamline filesystem is mounted on /beamline//current

You should always read and write data from the core filesystem. Only write small results back to the beamline filesystem, e.g. if they are needed for changing the setup of the running experiment.