Skip to content

v0.9.18: db, search improvements - #8912

Merged
waleedlatif1 merged 19 commits into
mainfrom
staging
Oct 11, 2026
Merged

waleedlatif1 merged 19 commits into
mainfrom
staging

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

waleedlatif1 and others added 12 commits October 10, 2026 13:20
* fix(inbox): scope reply threading to the receiving workspace

* fix(inbox): resolve the inherited chat inside the guarded run and cover the scoped persist
* fix(realtime): scope batch-add upserts to the caller's workflow

Block, edge, and subflow ids are global primary keys, so the
ON CONFLICT (id) DO UPDATE in batch-add-blocks and batch-add-edges could
overwrite a row owned by another workflow. Route every upsert through
upsertWorkflowRows, which confines the update to the socket's workflow
with setWhere and aborts the transaction when RETURNING shows a skipped
conflict.

* fix(realtime): type the scoped upsert helper against its table
…check linear (#8893)

* fix(file-parsers): cap untrusted number formats and make time-format check linear

* fix(file-parsers): keep the elapsed-format check on the uncapped format

* fix(file-parsers): leave dates with over-long formats untouched instead of rescanning them
A live search waits for every account before fusing results, and accounts ran
four at a time, so an organization with five to eight connected accounts paid
a second full provider round trip on every unscoped search. Production
seven-account searches took about 2.1 s at p50 against about 0.6 s for one
account. Each account is a single provider call, so eight at once stays well
inside provider and token-refresh limits.
…8890)

## Summary
- A Lucid live read made about 11 MCP calls one after another: metadata, a metadata-only manifest, one fetch per page region (up to 8), then metadata again. At 1–2 s per call, reads ran past the 15 s live read deadline and failed with `MCP error -32001: TimeoutError`.
- The first metadata call and the manifest now run together, and region fetches run four at a time. That's three rounds of latency instead of up to 11.
- Validation still walks the responses in page and region order, and the before/after document-change checks and the content size cap are unchanged.

## Test plan
- [x] New `lucid-mcp.test.ts`: region fetches overlap and are assembled in page and region order even when they finish in reverse. It fails on the old code (max in-flight = 1).
- [ ] CI
* feat(projects): prepare workspace project column compatibility

* test(projects): clean up compatibility fixtures atomically

* feat(projects): move membership to workspace column with rollout bridge

* test(projects): align transfer mocks with workspace membership

* improvement(workspaces): present API responses explicitly

* fix(projects): support PostgreSQL 16 column migration

* fix(projects): allow multi-batch copies to finish

* fix(projects): reconcile memberships during schema push

* fix(projects): use the direct connection for push reconciliation

* test(billing): wait for parked transactions before racing writers

* test(billing): freeze the clock for exact deadline assertions

* ci(projects): use the standard PostgreSQL test coverage

* fix(projects): support trigger-free column rollout

* fix(projects): retain healthy indexes during schema push

* fix(projects): correct rollout fixtures and schema-push index

* fix(projects): switch membership authority after compatible rollout

* test(projects): guard optional authority fixtures explicitly

* test(projects): seed authority in billing lock-order mock

* fix(projects): bound authority barriers and verify push replay

* docs(projects): record native constraint rollout decisions

* fix(projects): recognize completed rollout without its marker

* test(projects): model expansion authority in workspace move fixtures
…onfluence matches (#8888)

## Summary
- `read_document` turned every provider-side read failure into the generic "Knowledge operation failed", because `NativeSearchError` escaped `readLiveDocument` without being classified. Reads now return errors the model can act on:
  - a 404 or 410 becomes `not_found`, with a hint to search again or read a different result;
  - a revoked grant becomes `unauthorized`, naming the provider to reconnect;
  - rate limits, 5xx responses, provider timeouts, MCP request timeouts and the 15 s read deadline become a retryable `LiveReadError`.
- The Assistant `read_document` tool now reports `retryable` (and `retryAfterSeconds` when set), the same way `search_workspace` does. The Search MCP `read_document` returns the classified message instead of the generic text.
- Confluence search labeled every non-blogpost CQL hit a `page`. Native CQL that matched attachments, comments, whiteboards, folders or databases therefore produced references whose v2 page read returns 404. Those kinds are now dropped, and the result message says so.

## Test plan
- [x] `application.test.ts`: covers 404, reconnect, rate limit, 503 and MCP timeout. All five fail with the fix reverted.
- [x] `atlassian.test.ts`: covers native CQL matches that a page read can't open. It fails with the fix reverted.
- [x] Focused suites pass locally: application, atlassian, policy, workspace-search, mcp server.
- [ ] CI
…ing the workspace (#8908)

* perf(files): resolve name references through an index instead of listing the workspace

A reference that misses the exact-name lookup fell back to loading every
live file in the workspace and matching in memory. A workflow's new-file
existence check always misses, so each file it wrote paid a full read.

Index the displayed name (the SQL twin of normalizeDisplaySegment) and
read only the rows the matcher can pick, keeping its results unchanged.

* refactor(files): share the whitespace class and split the display-name tests

* chore(db): format the generated migration snapshot

* test(files): explain the resolver's real candidate query and fix fixtures

Mock displaySegmentKey in the central schema mock, store size_bytes on the
raw fixture rows, and assert the plan of the full candidate query (id arm,
display-name arm, order) rather than a reduced one.

* fix(files): break uploaded-at ties by id when picking a reference candidate
…ument writes (#8909)

* perf(knowledge): lock the knowledge base row FOR NO KEY UPDATE on document writes

Document uploads, document hard deletes, connector creation, and knowledge
base restore locked the parent knowledge_base row FOR UPDATE. Every embedding
insert takes FOR KEY SHARE on that row through its foreign key and holds it
for the whole indexing transaction, and FOR UPDATE conflicts with KEY SHARE,
so these writes queued behind any indexing pass in the same knowledge base
(and vice versa). A hard delete could also deadlock against an indexing pass
on one of its own documents.

FOR NO KEY UPDATE still conflicts with itself, FOR SHARE (connector sync
saves and completion), FOR UPDATE (KB moves, tag mutations), and the
UPDATE/DELETE of the row, so every serialization these locks provided is
kept. No path under these locks writes knowledge_base.id, the only key
column (the other unique indexes are partial). This matches soft deletion
and connector cleanup, which already use FOR NO KEY UPDATE.

* test(knowledge): uploads and hard deletes commit while an embedding holds the KB key-share lock
… writes (#8910)

* perf(auth): stop blocking requests on best-effort last-used timestamp writes

The API key lastUsed update and the desktop device lastSeenAt update are
display-only, but every request awaited them. When the primary's commits wait
on synchronous replication, these single-row writes stall for the whole
episode and hold the authenticated request (and the desktop inbox poll) with
them. Concurrent requests on the same key also queued behind the stalled
writer's row lock, each holding a pool connection.

Both writes are now fire-and-forget with logged failures, following the
sandbox image touch precedent. The API key write is additionally debounced
per process with an LRU keyed by key id over the existing staleness window,
so a hot key issues at most one in-flight write per process instead of one
per request.

* fix(auth): never overlap last-used and last-seen writes for the same key

A process-local TTL debounce alone let a write still pending past the
window, or evicted from the cache, be followed by another write queued on
the same row lock. createDetachedTouch tracks in-flight keys apart from the
optional debounce, so a key never has two writes pending. The desktop
last-seen write takes no in-process debounce: the offline check reads it.

* fix(core): release a detached touch key even if its write throws synchronously
…t key (#8911)

The row drain and the export page reader advanced with
`order_key IS NULL OR (order_key, id) > (anchor)`. The OR keeps the unkeyed
tail reachable, but the planner cannot turn it into a range on the
`(table_id, order_key, id)` index, so every page scanned the table from its
first key and filtered out every row before the anchor: page cost grew with
page depth.

Split the seek into its two disjoint halves, the keyed rows past the anchor
and the unkeyed tail, each a limited index range, and UNION ALL them under
the same ORDER BY/LIMIT/OFFSET. Postgres merges the two ordered streams and
reads only the rows the page can reach. Rows, order, and cursor semantics are
unchanged; the inner limits cover ask + offset so a compound cursor resuming
inside the unkeyed tail still lands on the same row.

The db chain mock gains `unionAll`, resolving to the left chain's rows. The
mock test that asserted the seek SQL text is replaced by an integration case
that pages a table with keyed and unkeyed rows through both readers at
several page sizes.
@vercel

vercel Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 11, 2026 6:35pm UTC

Request Review

…-healthcare-legal-procurement (#8898)

Co-authored-by: Sim Pi Agent <pi@sim.ai>
icecrasher321 and others added 3 commits October 11, 2026 11:34
…8899)

* docs(library): update best-ai-agents-for-customer-support-automation

* Pi Babysit: address PR #8899 feedback

---------

Co-authored-by: Sim Pi Agent <pi@sim.ai>
* docs(library): update best-ai-agents-sales-crm-automation

* Pi Babysit: address PR #8901 feedback

---------

Co-authored-by: Sim Pi Agent <pi@sim.ai>
Co-authored-by: Sim Pi Agent <pi@sim.ai>
…Ups (#8903)

* feat(library): How to Build a Pipedrive CRM AI Agent for Deal Follow-Ups

* Pi Babysit: address PR #8903 feedback

* Pi Babysit: address PR #8903 feedback

---------

Co-authored-by: Sim Pi Agent <pi@sim.ai>
icecrasher321 and others added 2 commits October 11, 2026 11:35
* docs(library): update ai-agent-observability

* Pi Babysit: address PR #8900 feedback

---------

Co-authored-by: Sim Pi Agent <pi@sim.ai>
…t and Follow-Up? (#8905)

Co-authored-by: Sim Pi Agent <pi@sim.ai>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found and verified against the latest diff

Confidence score: 3/5

  • In reconcile-project-membership.ts, a URL-level search_path can make the replayed 0406 SQL resolve unqualified tables and indexes outside public, while the guard only checks public.*. Ensure the replay uses the intended schema or make the guard check the schema it will affect.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/db/scripts/reconcile-project-membership.ts">

<violation number="1" location="packages/db/scripts/reconcile-project-membership.ts:9">
P2: This connection inherits URL-level `search_path`, but the replayed 0406 SQL resolves `workspace`, `project`, and index names unqualified while the guard checks `public.*`; a custom path can make `db:push` alter shadow objects or fail. Pin the reconciliation connection to `public, pg_temp` before replay.</violation>
</file>

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/lib/projects/membership.ts
Comment thread packages/db/scripts/reconcile-project-membership.ts
@greptile-apps

greptile-apps Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High impact] The PR appears safe to merge; no actionable issues were established.

Summary

This release prepares Project membership to move onto workspace.projectId and improves database and live-search work.

  • Workspaces get their Project membership from the workspace row.
  • Live search checks more accounts and reads Lucid pages in parallel.
  • Live document reads tell callers whether to retry or reconnect.
  • Confluence search returns only results its read path can open.
  • Table pages seek past their cursor, including rows without keys.
  • Workspace file lookups use an index for displayed names.
  • Inbox replies stay with the workspace that received them.
  • Best-effort last-used and last-seen writes run off the request path.
  • Spreadsheet display formats stay bounded and time checks run linearly.
  • Knowledge document writes avoid blocking embedding inserts.
  • Realtime batch-add updates stay inside the caller’s workflow.
  • Loading screens use a wordmark instead of a square logo.

No actionable issues were established. Review checks were based on code and tests; no runtime tests were run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Project operation starts] --> B[Hold workspace table read lock]
  B --> C{Membership phase}
  C -->|connector| D[Read and write project_workspace]
  C -->|column| E{workspace.project_id assigned?}
  E -->|yes| F[Read workspace.project_id]
  E -->|no, connector remains| G[Read legacy membership]
  C -->|column writes| H[Write workspace.project_id]
  D --> I[Commit transaction]
  F --> I
  G --> I
  H --> I
  B -. prevents overlap .-> J[Authority switch or connector retirement]
Loading

Reviews (1) · Last reviewed commit: "perf(tables): seek keyset row pages inst..." · Reviewed by Greptile

@waleedlatif1
waleedlatif1 merged commit 22c3de1 into main Oct 11, 2026
88 checks passed

This branch was previously deployed

1 inactive deployment
Preview — 36849203 Deployed Oct 11, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants