Scope the project-mode Deno crawl to the JSR packages deno.lock records (#1216)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #1216

Summary

In project mode, DenoCrawler::crawl_all walked the whole $DENO_DIR/npm/jsr.io cache as soon as cwd had a Deno marker. scan, agent apply and VEX therefore saw every JSR package any project on the machine had cached. A project with a deno.lock now gets only the JSR packages its lock records, each looked up in the cache. --global, --global-prefix and projects without a usable deno.lock still walk the cache.

Why

This is the Deno child of tracking issue #595 (register E05, living document doc/06 "Cache crawls are not project-scoped"). It follows the NuGet (#1183), cargo (#1205) and Go (#1209) children.

• B 0
• U 1: closes the Deno checklist item of Tracking: scope project-mode cache crawls to what the project resolves #595.
• D 1: one lock_section reader for deno.lock's version-dependent layout, shared by the crawl and VEX discovery (deno_npm_keys), and one JSR locate lookup shared by find_by_purls and the crawl.
• S 3: the cache walk is replaced by lookups. Timings are below.
• R L/M: the change is limited to the crawl's candidate set.

What changed

• crawlers/deno_crawler.rs:
  • locate(cache, purl) does the lookup find_by_purls already did (purl grammar, the traversal guard and the is_dir check). It is now shared by find_by_purls and the scoped crawl.
  • jsr_lock_scope(cwd) reads <cwd>/deno.lock FIFO-safely with read_regular_to_string. It returns the pkg:jsr/ purl of each jsr key. It returns None when there is no lock, the lock is not JSON, or it has no version field, and the crawl then walks the cache.
  • lock_section(lock, "npm" | "jsr") reads a section from any lock version: top-level in v4/v5, packages.<s> in v3 and <s>.packages in v2.
  • crawl_all uses the lock scope in project mode and keeps the walk otherwise.
• vex/discover/deno.rs: deno_npm_keys now calls lock_section, and its private copy of the layout logic is deleted.
• There are two commits: the extraction (no behavior change), then the scoped crawl.

Size

	+	−
Production	108	43
Tests	142	0

Behavior

These are the only changes:

• Project mode, readable deno.lock with a version: cached JSR packages that the lock doesn't record are no longer crawled.
  • A lock without a jsr section, such as an npm-only project, crawls no JSR packages.
  • scan --prune therefore treats manifest entries for unlocked cached JSR packages as uninstalled.
• Lock keys: keys whose coordinates would traverse out of the cache are rejected by the existing is_safe_jsr_component guard.
• Unchanged: global and prefix crawls, deno.json-only projects, malformed locks, find_by_purls, and the VEX npm-section read.

Test evidence

• New tests, red on main → green here:

  • crawl_all_local_lock_scopes_to_locked_jsr_packages (v5, v4 and v3 locks)
  • crawl_all_local_lock_without_jsr_reports_nothing
  • crawl_all_local_lock_skips_uncached_and_unsafe_entries

  To get the red result I forced the scope off, which reproduces main's walk: these 3 FAILED and the other crawl tests passed.

• New tests, green on both (they pin the fallbacks):

  • crawl_all_walks_without_a_usable_lock_or_in_global_mode
  • crawl_all_local_without_lock_walks_the_cache

• Shared-reader test: lock_section_reads_every_lock_version covers both callers' sections (npm and jsr) across v2 to v5. The existing deno_npm_keys_cover_every_lock_version stays green.

• cargo clippy --workspace --all-features -- -D warnings: clean.

• cargo test -p socket-patch-core --lib: 5852 passed. The 4 failures are the known root-sandbox ones (copy_tree::relax_loop_must_not_traverse_symlinked_root, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, pypi_poetry::wire_write_failure_…, pypi_requirements::wire_failure_…).

• crawler_deno_e2e: 13 passed.

• CLI suites:

  • e2e_vex_lockfile: 343 passed
  • scan: 118 passed
  • in_process_alternate_installers: 26 passed
  • in_process_get_hosted_ecosystems: 10 passed
  • repair: 124 passed. The 2 failures (repair_exits_zero_and_stays_quiet_when_lock_file_unremovable, repair_cleanup_failure_is_reported_in_json_and_silent_modes) also fail on main in this sandbox because it runs as root. They pass in CI.

• Timing (debug test build, 20-run mean): a 3,000-package JSR cache with 140 locked packages took 48.1 ms to walk and 7.5 ms with the lock-scoped lookup.

Risk

Low to medium. The candidate set shrinks only when a real deno.lock is present. No CLI suite depended on the whole-cache Deno crawl in a locked project, since the VEX fixtures lock the JSR packages they stage. lock_section stays in the crawler for now because formats/mod.rs is changed by open PRs; it can move to a formats::deno model later.

🤖 Generated with Claude Code

https://claude.ai/code/session_01T1eqQrD34pmJbFhR6LkLx1

---

Note

Medium Risk
Changes which packages appear in scan/VEX for locked Deno projects (smaller candidate set) and parses untrusted deno.lock paths, though existing fail-closed path guards are preserved.

Overview
Project-mode Deno/JSR discovery no longer enumerates the entire $DENO_DIR/npm/jsr.io cache when the workspace has a valid deno.lock. DenoCrawler::crawl_all now reads the lock’s jsr keys (lock versions 2–5 via a shared lock_section helper), turns them into pkg:jsr/… PURLs, and resolves each with targeted cache lookups instead of walking the tree. Cached JSR packages from other projects are excluded; npm-only locks yield no JSR packages.

--global, --global-prefix, missing/unreadable locks, and JSON without Deno’s version field still perform a full cache walk. JSR path resolution is centralized in locate (used by both the scoped crawl and find_by_purls), with the same traversal guards for lock-derived coordinates.

VEX Deno discovery reuses lock_section for npm lock keys, removing duplicated lock layout parsing.

^{Reviewed by Cursor Bugbot for commit a3e5f30. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a3e5f30. Configure here.}

[Mikola Lysenko (mikolalysenko)]

[burn-down] Ready for review at a3e5f30c1. CI 519/519 green (438 success, 81 skipped, incl. ci-ok and the gradle hosted windows compat legs); mergeable, no conflicts. Bugbot reviewed a3e5f30 with no issues; no open threads. Reviewer focus: projects without a usable deno.lock, plus --global/--global-prefix, still walk the whole JSR cache, as before.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Final review brief

What it does. In project mode, the Deno/JSR crawl now looks up only the JSR packages <cwd>/deno.lock records, instead of walking the whole $DENO_DIR/npm/jsr.io cache. Cached packages from other projects stop showing up in scan, apply and VEX. Global and prefix crawls, and projects without a usable lock, still walk. A shared lock_section reader replaces the VEX module's private copy of the lock-layout logic.

Risk: low. The set only narrows when a real deno.lock with a version field is present. Every other case (no lock, malformed JSON, non-UTF-8, no version) falls back to the walk. The traversal guards are reused unchanged. The intended behaviour change: scan --prune now treats cached JSR packages the lock doesn't list as uninstalled.

Look here

• deno_crawler.rs:107: chooses between the lock-scoped lookup and the walk.
• deno_crawler.rs:162: locate, the shared lookup with the is_safe_jsr_component guard.
• deno_crawler.rs:197: jsr_lock_scope and its fallbacks.
• deno_crawler.rs:214: lock_section, which reads lock versions 2-5.
• deno_crawler.rs:639: the tests for unsafe and uncached lock entries.

Verified

• I read the full diff of both commits against main.
• lock_section reproduces the deleted deno_npm_keys logic exactly, so VEX discovery is unchanged.
• Lock-derived purls (pkg:jsr/@std/path@0.220.0) are byte-identical to build_jsr_purl, so manifest and scan purls still match.
• The lock is read with read_regular_to_string, so a FIFO or a directory can't hang it.
• A traversal key (@std/fs@../../@other/x/2.0.0) is tested.
• The three scoped tests fail on main. The tests that set DENO_DIR are #[serial].
• The other crawl_all callers and the Docker JSR e2e (--global) are unaffected.
• No debug leftovers. CHANGELOG.md is untouched. The branch merges cleanly onto current main.
• CI: 520/520 check runs on the head (438 success, 82 skipped), including ci-ok and clippy. Bugbot is clean and there are no threads.

Changes I made: none.

Open questions (not blocking): in a Deno workspace member whose deno.lock sits at the workspace root, the crawl still walks the whole cache. That is the old behaviour, so it's a possible follow-up, not a regression.

Auto-merge (squash) is armed, so approving sends this straight to the merge queue.

---

Generated by Claude Code