Reverse proxy for pip, conda and others

we have setup a proxy to be able using pypi and conda packages even from protected networks.

The proxy is hosted on a load-balanced alias max-pyproxy.desy.de, using two hosts (max-pyproxy001,002) in the background.

conda/mamba

modify ~/.condarc to include

proxy_servers:
   http: max-pyproxy.desy.de:8899
   https: max-pyproxy.desy.de:8899

and work with mamba/conda as usual. The conda configuration ssl_verify: False should usually not be needed.

pip

there is a dedicated pypi proxy available on nims.desy.de. That's certainly the preferred choice. To use it:

pip3 install -i https://nims.desy.de/devpi/root/pypi/+simple/ flask

Alternatively you can use the proxy:

pip install --proxy http://max-pyproxy.desy.de:8899 numpy

# or
export HTTPS_PROXY=http://max-pyproxy.desy.de:8899
export HTTP_PROXY=$HTTPS_PROXY  # should not be needed
pip install numpy

curl & wget

# wget
wget -e use_proxy=yes -e https_proxy=http://max-pyproxy.desy.de:8899  https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh

# curl
curl -x http://max-pyproxy.desy.de:8899 -L https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh

using a conda mirror

there are hardly any public conda mirrors available, and mirroring channels from anaconda.org turned out to be slow and tedious. There is however an excellent mirror available at prefox.dev.

Adding the following lines to ~/.condarc should redirect packaging pulling to repo.prefix.dev:

default_channels:
  - https://repo.prefix.dev/conda-forge

channel_alias: https://repo.prefix.dev/

This also works in combination with the reverse proxy.