Repository navigation
fix: avoid LAPACK deadlock and validate older JAX compatibility - #280
Merged
Merged
Conversation
This was referenced Oct 4, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Release integration's old
<0.11override selected JAX0.10.2 and the exact-wheel replay captured all four Eigen workers blocked in LAPACKParallelBatchMap -> CholeskyFactorization -> lapack_dpotrf_ffi. Keep the release baseline at>=0.11.2,<0.12, whose open-source build removes that path. Preserve the 115-pin incident manifest and bounded native diagnostic.Add an independent compatibility workflow for0.9.2 and0.11.2 using the original TestPyPI wheels, Python3.12.14, exact workspace/Hands commits, three original-likelihood trials, numerical/sampler checks and the historical FFT witness. The original control/candidate comparison remains manual and preserves its inconclusive result. Package policy in companion Nerves/Hands/consumer PRs retains older JAX while excluding affected releases; this validation baseline is not a global user minimum.
API Changes
Validation dependency floor changes; diagnostic workflows and evidence added. No Python public symbols, workspace API migration, timeout increases, quarantines or readiness-rule changes. See full details below.
Test Plan
Interleaved comparison37206724174 remains inconclusive: three controls and three candidates passed, so it does not measure failure-rate improvement. Previous-head Python3.12/3.13 CI37207278755 passed; cancelled37207278790 contributes no evidence. Original runner image and original wheel/FITS hashes were not retained.
Evidence and limits
Native capture in Heart replay37205459198 shows the 0.10.2 LAPACK/Eigen worker-pool deadlock. Tagged source lacks this path in 0.9.2 and disables it in OSS builds from 0.11.1. Only 0.10.2 was observed deadlocking; other exclusions are source-based precautions.
Original likelihood passes three CPU trials and one actual CUDA trial on each of 0.9.2/0.11.2; independent likelihood, finite-difference gradient, NUFFT, Optax and short BlackJAX execution checks pass on both backends. Historical FFT workaround is retained and its reproducer passes both versions. All 25 wheel resolver cases pass; fresh normal resolution without prereleases also selects protected synthetic stable wheels. Local Python3.12.10 differs from incident3.12.14.
Broader point-source gradient diagnostics hit the unchanged300s cap on BOTH versions after earlier checks pass; cause remains unresolved. Retained range does not certify every older version. No full release validation was rerun. Detailed receipts and compatibility CI: HeartPR280.
Development authorization
Heart remains RED: release validation FAILED (stage integrate). Live human authorization: “I authorize development investigation and repair despite the current RED reason above. Follow start-dev and record the task-specific authorization wherever required.” Compatibility plan approved: “ok yes do it properly then”. Independent Sol review CLEAN; applicable tests and targeted smoke evidence above pass. This authorizes development PRs only. Merge requires explicit human
/prm; no release is authorized. Umbrella: #274.Full API Changes (for automation & release notes)
Changed Behaviour
Migration
Generated by the PyAutoLabs agent workflow.
Coordinated PRs
PyAutoNerves, PyAutoHands, PyAutoFit, PyAutoGalaxy, PyAutoHeart, PyAutoArray, PyAutoLens, PyAutoCTI. Nerves must publish before repaired consumer wheels.