Since #1152, a plain-ASCII requirements.txt whose coding line names an unmodelled codec (iso-8859-15, cp1250, gbk…) is read as absent: scan finds nothing (exit 0), and vex / rollback can't see an existing hosted pin #1212

[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).

Summary

#1152 (fix for #1119 / #1120) sends every requirements reader through utils::requirements::decode. When the file has a PEP 263 coding line, decode maps the codec name through coding_line_codec, which only knows UTF-8, ASCII, Latin-1 and cp1252. For any other name it returns None, so the whole file is treated as unreadable. That includes common ASCII-compatible codecs such as iso-8859-15 / latin-9, cp1250, mac-roman and gbk.

This applies even when the file is pure ASCII, which is the usual case when a coding header was copied from a template. In that case every ASCII-compatible codec decodes the bytes the same way, and pip reads the file fine. v4.0.0 and the parent commit 03b9418 read these files as UTF-8 and handled them correctly.

Impact (main 793edd4)

• scan / lock-only discovery finds no packages: scannedPackages: 0, status: success, exit 0, and no warning. A fresh checkout looks clean while pip installs the vulnerable release.
• scan --mode hosted rewrites nothing and exits 0 with no warning.
• On a project that's already hosted (wired by v4.0.0 or an earlier main), vex exits 2 with "cannot read requirements.txt: not text pip's decoding rules can read". rollback exits 1 with manifest_not_found. The hosted pin can't be attested or unwound until the user deletes the coding line.

Repro

mkdir proj && cd proj
printf '# -*- coding: iso-8859-15 -*-\nsix==1.16.0\n' > requirements.txt   # pure ASCII
python3 -m venv /tmp/empty           # empty VIRTUAL_ENV keeps it lock-only
pip install --dry-run -r requirements.txt     # pip 20.3.4 and 26.2.1: resolves six==1.16.0
VIRTUAL_ENV=/tmp/empty socket-patch scan --json --dry-run
#   main 793edd4: batch request has no components, scannedPackages 0, exit 0
#   03b9418 / v4.0.0: batch asks for pkg:pypi/six@1.16.0, scannedPackages 1

# already-hosted variant (requirements.txt wired by 03b9418 / v4.0.0):
#   # -*- coding: iso-8859-15 -*-
#   six @ https://patch.socket.dev/patch/pypi/six/1.16.0/…/six-1.16.0-py2.py3-none-any.whl#sha256=…
socket-patch vex --product pkg:pypi/app@1.0.0   # main: exit 2 ("cannot read requirements.txt …"); 03b9418: not_affected, exit 0
socket-patch rollback --yes --json              # main: exit 1, manifest_not_found; 03b9418: exit 0, file restored

The same result came back for cp1250, latin-9, mac-roman and gbk. # -*- coding: utf-8 -*- still works. All runs used a local mock patch API with a real patched wheel, and each repro ran at least twice.

Expected vs actual

• Expected: requirements files are read the way pip decodes them. Fix requirements files read as absent when not UTF-8 (#1120, #1119) #1152's commit message says "utils::requirements::decode follows pip's auto_decode". pip's auto_decode decodes with whatever codec the coding line names, so these files install normally (verified: pip 20.3.4 on py3.8 and pip 26.2.1 on py3.11 both install six from them, and both install the hosted patched wheel from the rewritten file). docs/ecosystems.md lists requirements.txt as supported in every mode.
• Actual: a codec outside the four modelled ones makes the file read as absent, or as unreadable, and scan does this silently. That's exactly the Lock-only scans still drop a requirements.txt pip decodes through a PEP 263 coding line (latin-1): "No pypi packages found", exit 0, while the same project with a venv refuses candidate_file_unreadable #1119 symptom, now triggered by files that used to work.

Matrix (Linux)

macOS / Windows weren't probed. The code path is OS-independent.

First bad commit: 793edd4 (#1152).

Suspect code

• crates/socket-patch-core/src/utils/requirements.rs:57: coding_line_codec(&name)? returns None for any codec the reader doesn't model, even when the bytes are ASCII (or valid UTF-8) and every ASCII-compatible codec would decode them identically.
• crates/socket-patch-core/src/utils/requirements.rs:109: coding_line_codec covers only UTF-8 / ASCII / Latin-1 / cp1252.

A possible direction: when the codec isn't modelled and the bytes are pure ASCII, decode them as ASCII. Or fail loudly with a warning rather than reading the file as absent. I haven't filed a fix.