PIP

PIP

Applies To: Diskover 2.5.x+
Platforms: RHEL / Rocky Linux 8 & 9, Ubuntu / Debian 22 & 24

Overview

This guide covers how Python packages (pip) are managed across the Diskover platform. Both the Diskoverd worker daemon and Diskover Admin web application rely on Python libraries installed via pip, and keeping these dependencies properly installed and isolated is essential for a healthy Diskover deployment.

Starting with Diskover 2.5.x, all Python dependencies are installed into a dedicated virtual environment (/opt/python-venv-diskover) rather than the system Python. This page explains why that matters, how the venv is created, and how to install or update pip packages.

When to use this guide:

• Setting up Python and pip on a new Diskover host (web or worker)

• Troubleshooting pip dependency issues on an existing deployment

• Installing or upgrading individual Python packages

Note: Diskover also offers Ansible playbooks that handle the pip install of packages, Python version selection, and venv creation automatically. If you'd like to use Ansible to manage this process, reach out to the Diskover team for assistance.

Understanding Python & pip in Diskover

pip is the standard package manager for Python. Diskover uses pip to install the Python libraries that Diskoverd and Diskover Admin depend on — things like Elasticsearch clients, Flask, Celery, cryptography, and many others. These dependencies are defined in requirements.txt files shipped with each component.

Why a virtual environment?

Modern Linux distributions (RHEL 9, Ubuntu 22+) mark the system Python as "externally managed," which means running pip install against it will fail or require the --break-system-packages flag. Beyond that, installing Diskover's dependencies into the system Python risks conflicting with OS packages or other applications on the same host.

A Python virtual environment (venv) solves this by creating an isolated copy of the Python interpreter with its own site-packages directory. Diskover's venv lives at /opt/python-venv-diskover and is shared by all Diskover components on a given host — Diskoverd, Celery, and Diskover Admin all use the same venv.

How it fits together:

Official PyPI repository: Diskover sources pip modules from https://pypi.org/

Diskover Python Virtual Environment

Starting with Diskover 2.5.x, all Python packages are installed into a dedicated virtual environment at /opt/python-venv-diskover. This isolates Diskover's dependencies from the system Python and avoids conflicts with OS-managed packages.

Python version requirements

Diskover requires Python 3.11 or newer for the virtual environment:

• If your system Python is 3.12 or newer, you can use the system version directly.

• Otherwise, install Python 3.11 and use it to create the venv.

You can check your system version with:

python3 -V

Create the Diskover venv

RHEL / Rocky Linux:

If the system Python version is older than 3.12, install the target Python version first:

dnf install python3.11 python3.11-devel gcc

If the system Python is already 3.12+, install the development headers only:

dnf install gcc python3-devel

Create the virtual environment (replace 3.11 with your target version if different):

python3.11 -m venv /opt/python-venv-diskover

Ubuntu / Debian:

Install the venv module and development headers:

apt install python3-venv gcc python3.11-dev

Create the virtual environment:

python3.11 -m venv /opt/python-venv-diskover

Upgrade the core packaging tools inside the venv:

/opt/python-venv-diskover/bin/pip install --upgrade pip setuptools wheel

Add the venv to your system PATH

Create a profile script so the venv is available to all users and services:

cat > /etc/profile.d/diskover-venv.sh << 'EOF'
# Add Diskover venv to PATH for CLI usage
export PATH="/opt/python-venv-diskover/bin:$PATH"
EOF
chmod 0644 /etc/profile.d/diskover-venv.sh

Source it in your current session (or log out and back in):

source /etc/profile.d/diskover-venv.sh

Verify the venv is active:

which python3
# Expected: /opt/python-venv-diskover/bin/python3

which pip
# Expected: /opt/python-venv-diskover/bin/pip

Install PIP (Legacy — System Python)

Note: The commands in this section install pip into the system Python. For Diskover 2.5.x+ deployments, use the virtual environment approach above instead.

By default for your version of Python3 it is possible that pip is simply not installed. Here is a quick command to ensure it's available on that specific Python3 version.

• RHEL: python3 -m ensurepip

• Debian: apt install python3-pip

Upgrade PIP

Inside the Diskover venv (recommended):

/opt/python-venv-diskover/bin/pip install --upgrade pip

System Python (legacy):

python3 -m pip install --upgrade pip

Install Python Packages with PIP

Diskover Admin and Diskoverd each ship a requirements.txt file listing their Python dependencies. All packages should be installed into the Diskover venv.

Using the venv (recommended)

Diskover Admin:

/opt/python-venv-diskover/bin/pip install -r /var/www/diskover-admin/etc/requirements.txt

Diskoverd (Task Worker):

/opt/python-venv-diskover/bin/pip install -r /opt/diskover/requirements.txt

Installing a single package

To install or pin a specific package version:

/opt/python-venv-diskover/bin/pip install Flask==2.2.5

System Python (legacy)

These commands install into the system Python and are provided for reference only. Use the venv commands above for Diskover 2.5.x+ deployments.

• Diskover Admin: python3 -m pip install -r /var/www/diskover-admin/etc/requirements.txt

• Diskoverd (Task Worker): python3 -m pip install -r /opt/diskover/requirements.txt

• Single package: python3 -m pip install Flask==2.2.5

Quick Reference