Repository navigation
feat: SparseTerms oversample=q fine precision-operator and dirty-image grids - #621
Merged
Merged
Conversation
…e grids `sparse_terms_from_chunks(..., oversample=q, oversample_pad=0.25)` accumulates two grids q times finer than the image pixel in the same streaming pass, for components analytic in the uv-plane (points, small Gaussians; Discussion #13 item 1, @HRSAstro): - precision_operator_fine: K(lag) = sum w cos(2 pi u.lag), full native shape plus a ceil(pad * max(N)) lag pad, W~'s wraparound order, Nyquist kept; - dirty_image_fine: D(x) from the phase-shifted noise-weighted data, twice the field, centred, native orientation. The type-1 core of nufft_precision_operator_via_nufft_from is lifted into _type1_real_grid_from (W~ pinned bit for bit against the old inline loop). The fine grids are preallocated and added in place per chunk; q must be even and the transformer TransformerNUFFT. SparseTerms gains precision_operator_fine, dirty_image_fine, oversample, oversample_pad; __add__ checks oversample provenance and refuses fine grids on one side only. Pass-through on Interferometer.from_stream and apply_sparse_operator_from_chunks; one memory-estimate log line. Refs #620 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Oct 7, 2026
feat: persist SparseTerms oversample fine grids in dataset.fits (opt-in)
PyAutoLabs/PyAutoGalaxy#650
Merged
The two oversample ValueError tests build their inputs via
_streaming_inputs(), which constructs a TransformerNUFFT and so needs
nufftax; they failed (7 cases) on the unittest-nojax CI leg. Guard them
with pytest.importorskip("nufftax") like their siblings.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional
oversample=qtosparse_terms_from_chunks. It was requested by @HRSAstro (pyuvimage) in Discussion #13, comment item 1. With it set, the same streaming pass also accumulates two grids q times finer than the image pixel:precision_operator_fine:K(lag) = Σ w cos(2π u·lag)over the full native shape plus aceil(oversample_pad · max(N))lag pad (default 25 %), in the same wraparound order asW~, with the Nyquist row/column kept.dirty_image_fine:D(x) = Re Σ c exp(2πi u·x)from the same phase-shifted, noise-weighted data asdirty_image_native. It covers twice the field, centred, in native orientation.For a unit point at
pthese give every sparse-likelihood term of a uv-analytic component: the cross termK(x_i − p), the point-point termK(p − p')and the data termD(p). pyuvimage can then drop its own pass. The column-terms helper (review item B2) is a deferred follow-up and is not in this PR.The type-1 core of
nufft_precision_operator_via_nufft_fromis lifted into_type1_real_grid_from. A new test pinsW~bit for bit against a verbatim copy of the old inline loop. The fine grids are preallocated and added in place per chunk (quadrant-wise ifftshift add), so no full-size grid is allocated per chunk. They inherit the equal-sigma check andpool_noise_map=Truepooling from #619. One info line logs a memory estimate.The dependency
sparse_noise_map_pooling_optionis merged as #619 (2c5cb69).Measured cost (warm call, CPU, 100×100 native, 2 × 1e5-visibility chunks, fresh process each):
At 400 px and q=8, K is 512 MB and D 328 MB, plus about 4 GB of nufftax work grid per call (5-6 GB peak). The cost is documented, with the advice to use chunks of at least ~1e6 visibilities with
oversample.Closes #620 (on merge of this PR and PyAutoLabs/PyAutoGalaxy's companion PR).
API Changes
Additive only. Defaults are unchanged and
W~is bit-for-bit unchanged.sparse_terms_from_chunks(..., oversample=None, oversample_pad=0.25);Interferometer.from_stream(..., oversample=None, oversample_pad=0.25);apply_sparse_operator_from_chunks(..., oversample=...)via its accumulator kwargs.SparseTermsgains optionalprecision_operator_fine,dirty_image_fine,oversample,oversample_pad, and ahas_fine_gridsproperty.SparseTerms.__add__now also checksoversample/oversample_padprovenance, and refuses when exactly one side carries fine grids.oversamplemust be a positive even integer, and the transformer aTransformerNUFFT. Otherwise it raisesValueError/InversionException.See full details below.
Test Plan
(iq, jq)==nufft_precision_operator[i, j]within the masked extent (mixed rtol/atol 1e-10), and fine K == brute force everywhere, pad and Nyquist included.dirty_image_nativeon unmasked pixels, and fine D == brute-force DFT everywhere.phase_centreshifts D by exactly the shift and leaves K bit-identical; pooled == pre-pooled bit for bit.__add__sums fine grids and refuses mismatchedoversample/oversample_pad, fine-on-one-side and fine-shape mismatch; odd/zero/negative/float/bool q, negative pad and the DFT transformer raise; memory log line; pass-through onfrom_stream/apply_sparse_operator_from_chunks.W~refactor bit-for-bit pin (7x7 and 16x16 fixtures, one-shot and chunked).test_autoarray: 2014 passed, 4 xfailed. Fulltest_autogalaxy(with the companion PR): 1357 passed.autolens_workspace/scripts/interferometer/features/datacube/modeling_array_free.py(from_stream / summed SparseTerms / phase_centre) passes on this branch.Full API Changes (for automation & release notes)
Added
autoarray.inversion.inversion.interferometer.inversion_interferometer_util.sparse_terms_from_chunks(..., oversample: Optional[int] = None, oversample_pad: float = 0.25)— accumulatesprecision_operator_fine/dirty_image_fine.aa.Interferometer.from_stream(..., oversample=None, oversample_pad=0.25)— pass-through.aa.SparseTerms.precision_operator_fine,.dirty_image_fine,.oversample,.oversample_pad(all defaultNone),.has_fine_grids.inversion_interferometer_util.fine_grid_shapes_from(shape_native, oversample, oversample_pad=0.25)— the two fine-grid shapes.inversion_interferometer_util._type1_real_grid_from(...)(private) — reusable type-1 NUFFT real-grid core with in-place accumulation.Changed
aa.SparseTerms.__add__—oversample/oversample_padjoin the provenance check; terms where exactly one side carries fine grids now raiseInversionException(previously no such fields existed).Migration
Requested by @HRSAstro — https://github.com/orgs/PyAutoLabs/discussions/13 (https://github.com/orgs/PyAutoLabs/discussions/13#discussioncomment-18741903).
Tier: judge — merge mode: human /prm. Heart readiness at ship: YELLOW (manifest drift: shared-standards blocks; release validation stale), acknowledged.
Generated by the PyAutoLabs agent workflow.
🤖 Generated with Claude Code