Affected Repositories

• PyAutoArray (primary)
• PyAutoGalaxy (FITS persistence; depends on the PyAutoArray PR)

Branch Survey

Suggested branch: feature/sparse-terms-oversampled-fine-grids

Implementation Steps

1. autoarray/inversion/inversion/interferometer/inversion_interferometer_util.py:
   • New _type1_real_grid_from(uv_wavelengths, values, n_modes, pixel_scale_radians, *, eps, chunk_size, centred, zero_nyquist, out=None). It returns Re nufft2d1(-x, +y, values, n_modes, eps, +1), with x = 2π u Δ, y = 2π v Δ, chunked over visibilities, ifftshifted to wraparound order unless centred, Nyquist zeroed if zero_nyquist. With out it adds in place (quadrant-wise for the shift), with no per-chunk full-grid allocation. nufft_precision_operator_via_nufft_from calls it with centred=False, zero_nyquist=True: same operations, same order, bit-for-bit.
   • sparse_terms_from_chunks(..., oversample: Optional[int] = None, oversample_pad: float = 0.25). q must be a positive even integer and the transformer a TransformerNUFFT (exc.InversionException / ValueError otherwise, checked before any work).
     • Fine K: (2(Ny+p)q, 2(Nx+p)q), (Ny, Nx) = full native shape, p = ceil(oversample_pad * max(Ny, Nx)), values w = 1/σ_r², spacing Δ/q, wraparound order like W~, zero_nyquist=False.
     • Fine D: (2Ny q, 2Nx q) centred (origin at [Ny q, Nx q], native orientation), from the phase-shifted c = d_r/σ_r² + i d_i/σ_i². Built as the core on conj(c), which flips both axes to the native orientation.
     • Both preallocated once and += per chunk. One SparseTerms built at the end via dataclasses.replace. One memory-estimate log line.
   • SparseTerms: append precision_operator_fine, dirty_image_fine, oversample, oversample_pad (Optional, None). __add__ adds oversample, oversample_pad to the provenance loop (numeric compare), refuses when exactly one side carries fine grids, checks fine shapes, and sums the fine grids.
2. autoarray/dataset/interferometer/dataset.py: pass oversample / oversample_pad through Interferometer.from_stream and apply_sparse_operator_from_chunks.
3. PyAutoGalaxy autogalaxy/interferometer/model/analysis.py: SPARSE_TERMS_SCALARS_ORDER gets oversample, oversample_pad appended (append-only, NaN = unrecorded), plus header OVERSAMP. HDUs PRECISION_OPERATOR_FINE / DIRTY_IMAGE_FINE are written only when present and include_fine_grids=True (default False). Loader autogalaxy/aggregator/interferometer/interferometer.py reads them via _has_hdu; pre-change files give None.
4. Docs: the memory/cost note (400 px, q=8: K 512 MB + D 328 MB plus ~4 GB nufftax work grid; use large chunks, ≳1e6 visibilities, with oversample).

Tests (B1 list)

(i) fine K at lags (iq, jq) == nufft_precision_operator[i, j] within the extent (mixed rtol/atol); (ii) fine D at native pixel centres == dirty_image_native; (iii) chunked == one-shot for both grids; (iv) phase_centre shifts D, not K; (v) __add__ refuses mismatched q and fine/no-fine; odd q / DFT raise; (vi) PyAutoGalaxy FITS round trip, including a legacy file loading as None. W~ bit-for-bit pinned by the existing NUFFT-vs-brute parity tests.

Key Files

• autoarray/inversion/inversion/interferometer/inversion_interferometer_util.py
• autoarray/dataset/interferometer/dataset.py
• test_autoarray/inversion/inversion/interferometer/ (sparse terms tests)
• autogalaxy/interferometer/model/analysis.py
• autogalaxy/aggregator/interferometer/interferometer.py
• test_autogalaxy/interferometer/model/test_analysis_interferometer.py

Deferred follow-up (not in scope): B2 point_column_terms_from(...) helper (FFT Gaussian smoothing + quintic spline).

SparseTerms oversample=q: accumulate fine precision-operator and dirty-image grids for analytic uv-plane components (streaming P6)

Type: feature
Target: PyAutoArray
Repos:

• PyAutoArray
• PyAutoGalaxy
  Themes:
• interferometer
• sparse-operator
• community
  Autonomy: supervised
  Priority: normal
  Status: draft
  Filed: 2026-10-07
  Difficulty: medium
  Consequence: judge
  Depends-on: sparse_noise_map_pooling_option (merge first; same repo claim)
  Witness: with sparse_terms_from_chunks(..., oversample=8) (NUFFT path), (i) precision_operator_fine sampled at lags (i q, j q) equals nufft_precision_operator[i, j] within the masked extent (mixed rtol/atol per iiu:529-531), and (ii) dirty_image_fine sampled at native pixel centres equals dirty_image_native; today oversample does not exist.
  Review-minutes: 20
  Unattended: ready
  Source: GitHub Discussion https://github.com/orgs/PyAutoLabs/discussions/13 (external contributor @HRSAstro, pyuvimage), comment Streaming visibilities for memory efficiency .github#13 (comment) item 1; technical review 2026-10-07 (Item B1, accept-with-change).

Request (verbatim)

1. Analytic components on the streamed path

sparse_profile_terms_from covers light profiles evaluated on the image grid, but a component that is analytic in the uv-plane (a point source, or a small Gaussian for a marginally resolved source) needs its terms at sub-pixel positions. All three terms of such a column are values of two image-plane functions that can be accumulated in the same streaming pass:

• K(Δ) = Σ w cos(2π u·Δ), i.e. the precision operator at an arbitrary lag
• D(x) = Re Σ (d_re/σ_re² + i d_im/σ_im²) exp(2πi u·x), i.e. the dirty image at an arbitrary position

For a unit point at p, the cross-term with the mesh is Mᵀ k_p with k_p[i] = K(x_i − p), the point-point term is K(p − q) and the data term is D(p). [...]

In pyuvimage we accumulate both on grids 8× finer than the image, with one extra type-1 NUFFT per batch for each, and read them with a quintic spline. [...]

• The precision-operator grid needs lags beyond one field width (we pad by a quarter field), and the dirty-image grid needs twice the field. Otherwise Gaussian-widened columns near the edge pick up the wrap-around.
  [...] An optional oversample=q in sparse_terms_from_chunks that accumulates these two fine grids would be enough for us to drop our own pass. A helper that returns the column terms for given positions and widths would make it general. Our implementation is in pyuvimage/streamed_points.py if it's a useful reference.

What

iiu = autoarray/inversion/inversion/interferometer/inversion_interferometer_util.py (origin/main lines). nufft_precision_operator_via_nufft_from (iiu:454-624) already builds W~ as one type-1 NUFFT onto a (2Ny, 2Nx) grid over the masked extent (iiu:2220), spacing hard-wired to the image pixel (iiu:580), Nyquist zeroed (iiu:621-622). A fine grid needs a small refactor, not a flag. The maths in the request is right; K uses w only, so it inherits the equal-sigma assumption (hence the dependency).

Scope: B1 only. Deferred follow-up (not in scope): B2 point_column_terms_from(terms, positions, widths, mapping_matrix=None, *, spline_order=5) helper (FFT-space Gaussian smoothing, quintic spline; not JAX-traceable) — file separately if we want point components in PyAutoLens's own sparse fits.

Plan

1. Lift the W~ core loop into _type1_real_grid_from(uv, values, n_modes, pixel_scale_radians, eps, chunk_size, centred, zero_nyquist); W~ calls it unchanged (existing tests pin it).
2. API: sparse_terms_from_chunks(..., oversample: Optional[int] = None, oversample_pad: float = 0.25), pass-through on from_stream / apply_sparse_operator_from_chunks. Require even q (native centres land on fine grid points) and the NUFFT transformer, else raise.
3. Grids: precision_operator_fine shape (2(Nk+pad)q)^2, pad = ceil(0.25*max(Nk)), Nk = full native shape (points may sit anywhere in the mask), wraparound order like W~, zero_nyquist=False (spline reads across lag N). dirty_image_fine shape (2Ny q, 2Nx q) centred (origin at [Ny q, Nx q]) from the phase-shifted c = d_re/s_re^2 + i d_im/s_im^2 exactly as iiu:2233-2236.
4. SparseTerms (iiu:1880, frozen dataclass): append precision_operator_fine, dirty_image_fine, oversample, oversample_pad (all Optional, None). __add__ (iiu:1951): add oversample/oversample_pad to the provenance loop (iiu:1977-1984) and refuse when exactly one side carries fine grids (pyuvimage silently drops them; refusing is our pattern).
5. Accumulation: preallocate the two fine arrays and += in place across chunks; build one SparseTerms at the end (do NOT + a fresh SparseTerms per chunk, iiu:2272, for 100s of MB arrays).
6. Memory/cost (document; consider a memory-estimate log line): 400x400, q=8, 25 % pad → K fine 8000^2 f64 = 512 MB, D fine 6400^2 = 328 MB, plus nufftax 2x-upsampled work grid 16000^2 c128 = 4.1 GB (K) / 2.6 GB (D): peak ~5-6 GB; q=4 ~1.5 GB; contributor's ~112 px at q=8: 40 MB + 26 MB, <0.5 GB work. Two extra type-1 NUFFTs per chunk with FFT cost independent of chunk length (seconds per chunk at 400/q8): document "use large chunks (>= ~1e6 vis) with oversample". JAX: each (K, n_modes, eps) compiles once; ragged last chunk adds signatures — pad to a fixed bucket with zero weights only if measured to matter.
7. PyAutoGalaxy persistence (autogalaxy/interferometer/model/analysis.py:40-66, writer ~:69, loader autogalaxy/aggregator/interferometer/interferometer.py:62-111): HDUs PRECISION_OPERATOR_FINE, DIRTY_IMAGE_FINE only when present; append oversample, oversample_pad to SPARSE_TERMS_SCALARS_ORDER (append-only, NaN = unrecorded) and header OVERSAMP; loader uses _has_hdu. Decision: fine grids are NOT written into dataset.fits by default (~840 MB per search output at 400/q8) — config flag to opt in; consider aa.SparseTerms.output_to_fits/from_fits as the cleaner home.

Tests: (i)/(ii) the witness; (iii) chunked == one-shot for both grids; (iv) phase_centre shifts D, not K; (v) __add__ refuses mismatched q and fine/no-fine; (vi) PyAutoGalaxy FITS round trip incl. an old file without the new HDUs/scalars loading as None; odd q / DFT transformer raise.

Risks: memory and per-chunk FFT cost at large grids; extra JAX compiles per chunk shape; FITS bloat if the default flips.