Skip to content

pyfix: find broken Python packages from the list pip already keeps

The problem

A Python program that worked yesterday fails today with cannot import name X. Nothing was upgraded. Often a package has lost files underneath it: a disk cleaner deleted "duplicate" empty __init__.py files, a dedupe hardlinked two copies so an edit to one changed the other, an install stopped halfway. The error points at the symptom, so people reinstall the wrong thing or rebuild the whole environment.

The method

pip already wrote down what a healthy install looks like.

  • RECORD is a free manifest. Every package pip or uv installs gets *.dist-info/RECORD: one line per file, path,sha256=...,size. A listed file missing from disk is damage.
  • Hashes catch quiet edits. --hash compares each file with its recorded hash, so a file changed by hand or through a hardlink shows up even when it still imports.
  • Bytecode is not damage. .pyc and __pycache__ are rebuilt on import and skipped. Console scripts in bin/ are checked: a missing one is a broken command.
  • The right interpreter, never a guess. lib/python3.12/site-packages says which Python owns it; only bin/python3.12, or a python3 that answers 3.12, is used to repair it.
  • The smallest repair. --fix reinstalls only the damaged packages, pinned to the installed version, --force-reinstall --no-deps. Nothing upgrades. pip if the environment has it, else uv for that interpreter.
  • Local installs are reported, not pinned. A package installed from a folder or URL cannot come back from an index under name==version.

Tried on a real virtual environment with one file removed and one edited: both found, both reinstalled at the same versions, the package imported again, and a second run came back clean. On ten untouched real environments, --hash found nothing, as it should.

Sample

The check, in Python:

for line in open(f"{dist_info}/RECORD"):
    path, digest, _size = line.rsplit(",", 2)   # a path may hold commas
    if path.endswith(".pyc") or "__pycache__" in path:
        continue
    full = os.path.normpath(os.path.join(site_packages, path))
    if not os.path.lexists(full):
        missing.append(path)
    elif check_hash and "=" in digest:
        algo, want = digest.split("=", 1)
        if b64_hash(full, algo) != want:     # urlsafe base64, no padding
            changed.append(path)

Use it:

pyfix ~/projects            # read-only: which environments, which packages
pyfix --hash ~/projects     # also compare every file with its hash
pyfix --fix ~/projects      # reinstall damaged packages, same versions

Exit 0 means all whole (or fixed), 1 means damage is left, so it can run on a schedule.

Limits

  • Only packages with a RECORD are checked; OS-managed dist-packages and old egg installs are left to their own tools.
  • --hash reads every installed byte: seconds for a few environments, minutes for hundreds.
  • Files a package writes after install, or files someone added, are not judged.
  • --fix needs the same version still downloadable; a yanked version fails and says so.

Code

methods/pyfix/ in the methods repository: the script, a README and a test.