1616356 : Cupy driver error on gpu node

Created: 2026-07-10T18:40:39Z - current status: new

Anonymized Summary:

Issue: A user encountered errors while trying to use the xfel IPython kernel (ipykernel). The primary issues are:

  1. Conflicting CuPy Installations: Multiple versions of CuPy (cupy-cuda12x, cupy-cuda13x) are detected, causing potential functionality issues.
  2. NumPy Version Conflict: Modules compiled with NumPy 1.x fail due to incompatibility with NumPy 2.4.6.
  3. Matplotlib Import Error: Attempting to import matplotlib.pyplot triggers an AttributeError (_ARRAY_API not found) and ImportError (numpy.core.multiarray failed to import).

The user reports being unable to resolve the CuPy conflict via pip uninstall/reinstall.

Solution Steps:

  1. Resolve CuPy Conflicts:
  2. Uninstall conflicting CuPy packages manually: bash pip uninstall cupy-cuda12x cupy-cuda13x
  3. Verify no residual installations exist: bash pip list | grep cupy

  4. Reinstall the correct version aligned with the system’s CUDA toolkit (e.g., cupy-cuda12x).

  5. Downgrade NumPy:

  6. Downgrade to NumPy 1.x to ensure compatibility: bash pip install "numpy<2"

  7. Rebuild Environment (if needed):

  8. If the issue persists, consider recreating the virtual environment or container to avoid dependency conflicts.

  9. Check Permissions:

  10. Ensure the user has write permissions to modify Python packages in the target directory (~/.local/lib/python3.11/).

  11. Alternative:

  12. Use Conda to manage dependencies cleanly: bash conda create -n new_env python=3.11 numpy=1.* cupy=cuda12x conda activate new_env

Note: Coordinate with administrators if modifications require elevated privileges.