Jupyter notebook intro

Quickstart & some stuff you should know about Jupyter Notebooks in the NAF

Jupyter Notebooks are one popular interface to access batchcomputing and/or the storage and software setup of the NAF.

Point your browser to naf-jhub.desy.de (it is accessible from in- and outside the DESY network likewise)

We currently provide three types of notebook setups for you

  • Regular: Provides 12 GB memory, 1 core and runs for 12h
  • Medium: Provides 20 GB memory, 2 cores and runs for 6h
  • Large: Provides 48 GB memory, 2 cores and runs for 3 h

alt text

Runtime and memory usage are being enforced

Consuming more than the allowed ressources will stop your Jupyter Notebook immediately without any warnings - keep an eye on your memory consumption (and limit) in the lower right corner of the notebokk!

alt text

GPU usage with Jupyter Notebooks

  • In order to use a GPU node you need to have the registry ressource 'nafgpu' !
  • A medium sized notebook (Provides 20 GB memory, 2 cores and runs for 6h) will be started for you automatically

Jupyterhub AFS logs

When a Jupyterhub server is obtained, it logs some information to your AFS Home, specifically

.jupyterhub.condor.err
.jupyterhub.condor.out

These files likely contain useful messages if you run into problems. For example, one of the more common issues is getting a timeout when trying to start a server.

If there are no new messages, you probably did not even connect to a worker

  • no slots available
  • project permissions might be wrong or missing

If there are messages corresponding to your attempt:

  • server startup might have failed due to conflicting python modules in your custom library (in .local/)

Installing python modules into AFS

The NAF Jupyterhub runs with Python 3.12, chosen for being the newest available version.

The system python on the worker nodes is currently 3.9, though, requiring a little more care when locally installing modules.

If you add your modules from a Jupyterhub terminal with pip install be sure to use the correct version:

  • pip for version 3.9
  • pip-3.12 for version 3.12

Run your pip -V to double check the python version if you select plain pip.

Alternatively open a console from JupyterLab with the Python Kernel you want to add modules to which should map pip to the correct version.