pipx package conflicts with global Python installs: detection and fix

Our nightly CI job started importing pandas from the system site‑packages even though we installed a fresh copy with pipx. No crash, but the DataFrame API behaved like an older 1.3 release. The culprit: a stray PYTHONPATH entry that let the global pandas shadow the isolated pipx version.

# Example showing the issue
# running inside a pipx‑installed mytool
import pandas as pd  # expected pandas 2.2 from the pipx venv
print('pandas version:', pd.__version__)
# Output showed 1.4.3 – that's the global install!

# later we load a CSV
df_tmp = pd.read_csv('data.csv')
print(f'df_tmp: {len(df_tmp)} rows')
# spent 30min debugging this

pipx creates a virtual environment, but it still respects the interpreter’s PYTHONPATH. If the host machine has a directory on PYTHONPATH that contains a site‑packages folder, the import machinery will search there before the pipx venv’s lib. That means pandas (or any other library) from the global install can be imported, leading to version drift. This mirrors the standard Python import resolution order defined in PEP 302 and documented in the Python import system specification. Related factors:

  • PYTHONPATH includes a directory with a stale site‑packages copy
  • pipx was invoked with –system-site-packages (or the environment variable PIPX_DEFAULT_PYTHONPATH is set)
  • Global packages were installed with pip install --user which places them on the user site‑directory that is also on sys.path

To diagnose this in your code:

bash
# 1. Verify which pandas is being used inside the pipx command
pipx run mytool -- python -c "import pandas, inspect, sys; print(pandas.__file__); print('sys.path first entry:', sys.path[0])"
# Expected: path inside ~/.local/pipx/venvs/mytool/lib/pythonX.Y/site-packages/pandas/__init__.py
# If you see something like /home/user/.local/lib/python3.11/site-packages/pandas/__init__.py, the global package is leaking.

# 2. Check PYTHONPATH in the pipx environment
pipx run mytool -- python - <<'PY'
import os, sys
print('PYTHONPATH:', os.getenv('PYTHONPATH'))
print('sys.path contains user site?', any('site-packages' in p for p in sys.path))
PY

Fixing the Issue

First, make sure the pipx venv is truly isolated. The safest route is to clear any PYTHONPATH that points at a global site‑packages directory and reinstall the tool without the --system-site-packages flag.

# 1. Unset PYTHONPATH for the current shell (or remove the offending entry from ~/.bashrc)
unset PYTHONPATH

# 2. Remove the conflicting global package if you don't need it system‑wide
pip uninstall pandas  # runs against the user‑level pip, not the pipx venv

# 3. Reinstall the tool with a clean environment
pipx uninstall mytool
pipx install mytool  # default creates a fully isolated venv

If you must keep the global pandas for other scripts, you can tell pipx to ignore the system site‑packages when creating the venv:

pipx install mytool --system-site-packages=false

Alternatively, pin the exact pandas version inside the tool’s dependencies so pipx will upgrade the isolated copy even if the global one is older:

# In mytool's pyproject.toml
[project]
dependencies = ["pandas>=2.2"]

After reinstalling, run the same detection command again – the printed path should now be under ~/.local/pipx/venvs/mytool/.... The DataFrame API will line up with the expected version and the CI job stops pulling in the stale global pandas.

The gotcha that bit us was a stray export PYTHONPATH=$HOME/.local/lib/python3.11/site-packages line in our Docker entrypoint. Removing that line restored true isolation.

  • Unsetting PYTHONPATH but forgetting the same variable is set inside the Dockerfile, so the leak reappears on the next container start.
  • Running pipx install --system-site-packages assuming it will speed things up – it actually re‑introduces the exact conflict you’re trying to avoid.
  • Installing the same package globally with pip install --user and then expecting pipx to ignore it; the user site‑directory is always on sys.path unless explicitly excluded.

Frequently Asked Questions

Q: How do I fix this issue?

See the solutions section above for multiple approaches.

We removed the stray PYTHONPATH export, purged the global pandas, and re‑installed the tool with pipx. The pipeline now imports pandas 2.2 from the isolated venv and finishes in under a minute – the SLA is back to green.

→ Fix How to resolve pip dependency conflicts in pandas projects → Why semver breaks with pandas API changes → Fix pandas SettingWithCopyWarning false positive → Fix pandas to_datetime timezone conversion issues