Skip to content

Creating Jupyter Kernels

the jupyterHub (naf or maxwell) launches a batch job starting a "private" jupyter singleuser server. The singleuser server is the instance executing your jupyter notebook.

The startup of the singleuser server, and the interaction with the jupyter notebooks, requires that the versions of the various python modules involved are compatible.

The singleuser server runs in a user context, so it will find the modules installed in ~/.local/lib/python[version]/lib/site-packages, and these modules take precendence. So if you have for example incompatible versions of pydantic, tornado, ipywidgets installed (e.g. with pip install --user), some of the functionalities appear to be broken, and the singleuser server might not start at all.

To avoid such problems, it's much better not to install any modules in ~/.local but rather use a virtual or conda environment.

Recipe

lets assume you have already modules installed for python3.12. You can easily create a virtual environment from that:

# get a list of python modules installed in userspace
python3.12 -m pip freeze --user > user.modules

# deactivate the use of ~/.local
export PYTHONNOUSERSITE=1

# create and activate a virtual environment
python3.12 -m venv superapp
source superapp/bin/activate
which python3
   ~/superapp/bin/python3

# install the packages 
python3 -m pip install -r user.modules

# create a jupyter kernel
python3 -m pip install ipykernel
python3 -m ipykernel install --name superapp --display-name "SuperApp (py3.12)" --user
   Installed kernelspec superapp in ~/.local/share/jupyter/kernels/superapp

# test if everything works as expected, you are still in the virtual env
python3.12 my-precious-code.py

# everything all right, so
# rm -rf ~/.local/lib/python3.12 or
mv ~/.local/lib/python3.12  ~/.local/lib/save.python3.12

unset PYTHONNOUSERSITE

This should create a kernel named SuperApp which you can select from jupyter to run your code. You can even replace the icons in ~/.local/share/jupyter/kernels/superapp by icons of your choice, to "personalize" the setup.