Skip to content

Gate task-level model routing behind experimental opt-in - #9033

Merged
lpcox merged 2 commits into
mainfrom
copilot/gate-task-level-model-routing
Sep 26, 2026
Merged

lpcox merged 2 commits into
mainfrom
copilot/gate-task-level-model-routing

Conversation

Copilot AI commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Task-level model routing previously activated from apiProxy.routing alone. This change requires an explicit experimental opt-in and rejects routing requests that omit it.

  • Configuration: Add strict experimental.modelRouting validation and carry the gate into runtime config. The gate alone leaves routing inactive.
  • Runtime: Require both settings before staging a conversation or adding router services, networks, mounts, and proxy routing environment.
  • Documentation and coverage: Update the config specification and generated schemas; add focused default-off and enabled-path tests.
experimental:
  modelRouting: true
apiProxy:
  routing:
    objective:
      goal: cost
      mode: balanced
    task:
      conversationFile: /path/to/conversation.json

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Add experimental config for task-level model routing Gate task-level model routing behind experimental opt-in Sep 26, 2026
Copilot AI requested a review from lpcox September 26, 2026 01:32
@lpcox
lpcox marked this pull request as ready for review September 26, 2026 03:01
Copilot AI balanced review requested due to automatic review settings September 26, 2026 03:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The gate is consistently enforced across validation, mapping, staging, Compose generation, and runtime orchestration with focused coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Adds an explicit experimental gate for task-level model routing, keeping routing disabled unless both configuration fields are present.

Changes:

  • Validates and maps experimental.modelRouting.
  • Gates all routing staging, services, mounts, networks, and environment variables.
  • Updates schemas, documentation, and focused tests.
File Description
src/​types/​api-proxy-routing-options.ts Adds runtime gate option.
src/​services/​optional-services.ts Gates router service creation.
src/​services/​api-proxy-service-config.ts Gates routing mounts and networking.
src/​services/​api-proxy-env-config.ts Gates proxy routing environment.
src/​services/​api-proxy-env-config.test.ts Tests routing environment gating.
src/​schema-validator.ts Adds actionable gate validation error.
src/​routing/​bootstrap.ts Gates conversation staging.
src/​routing/​bootstrap.test.ts Tests default-off staging behavior.
src/​config-mapper.ts Maps the experimental setting.
src/​config-file.ts Adds the user-facing config type.
src/​config-file-validation.test.ts Tests strict gate validation.
src/​config-file-mapping.test.ts Tests gate mapping.
src/​compose-generator.ts Gates routing network creation.
src/​compose-generator.test.ts Tests routing infrastructure omission.
src/​commands/​build-config.ts Carries the gate into runtime config.
src/​commands/​build-config.test.ts Tests runtime config assembly.
src/​cli-workflow.ts Gates routing lifecycle hooks.
src/​cli-workflow.test.ts Tests disabled workflow behavior.
src/​awf-config-schema.json Updates the runtime schema.
docs/​awf-config.schema.json Updates the canonical public schema.
docs/​awf-config-spec.md Documents opt-in and migration syntax.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

  1. Download the docs-preview-pr-9033 artifact from the workflow run
  2. Unzip and open index.html in your browser

Built from commit 7b79687

@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 92.64% 92.66% 📈 +0.02%
Statements 91.12% 91.14% 📈 +0.02%
Functions 89.17% 89.17% ➡️ +0.00%
Branches 84.21% 84.26% 📈 +0.05%
📁 Per-file Coverage Changes (2 files)
File Lines (Before → After) Statements (Before → After)
src/routing/bootstrap.ts 85.3% → 85.3% (+0.00%) 79.3% → 79.8% (+0.48%)
src/log-directory-setup.ts 96.2% → 100.0% (+3.78%) 96.3% → 100.0% (+3.71%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • api.github.com
  • example.com

[!TIP]
api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.github.com"
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Gemini reports failed. Facets need polishing...

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • play.googleapis.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "play.googleapis.com"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Claude passed

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by Smoke Claude for #9033

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • o205451.ingest.us.sentry.io

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "o205451.ingest.us.sentry.io"

See Network Configuration for more information.

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

✅ Build Test Suite completed successfully!

Warning

Firewall blocked 8 domains

The following domains were blocked by the firewall during workflow execution:

  • api.nuget.org
  • bun.sh
  • dc.services.visualstudio.com
  • deno.land
  • dl.deno.land
  • github.com
  • releaseassets.githubusercontent.com
  • repo.maven.apache.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.nuget.org"
    - "bun.sh"
    - "dc.services.visualstudio.com"
    - "deno.land"
    - "dl.deno.land"
    - "github.com"
    - "releaseassets.githubusercontent.com"
    - "repo.maven.apache.org"

See Network Configuration for more information.

Generated by Build Test Suite for #9033

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Security Guard has started processing this pull request

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • example.com
  • github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"
    - "github.com"

See Network Configuration for more information.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Warning

Firewall blocked 13 domains

The following domains were blocked by the firewall during workflow execution:

  • ab.chatgpt.com
  • accounts.google.com
  • android.clients.google.com
  • api.github.com
  • clients2.google.com
  • collector.github.com
  • contentautofill.googleapis.com
  • github.com
  • github.githubassets.com
  • msfeed25.pkgs.visualstudio.com
  • update.googleapis.com
  • www.google.com
  • www.gstatic.com

[!TIP]
api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "accounts.google.com"
    - "android.clients.google.com"
    - "api.github.com"
    - "clients2.google.com"
    - "collector.github.com"
    - "contentautofill.googleapis.com"
    - "github.com"
    - "github.githubassets.com"
    - "msfeed25.pkgs.visualstudio.com"
    - "update.googleapis.com"
    - "www.google.com"
    - "www.gstatic.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@lpcox
lpcox enabled auto-merge (squash) September 26, 2026 16:18
@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Cloud Hypervisor + Copilot

  1. github-list_pull_requests (closed, limit 1): PASS — returned PR Upgrade gh-aw and recompile workflows #9041
  2. curl https://github.com: PASS — HTTP 200
  3. Write/read /tmp/gh-aw/agent/smoke-cloud-hypervisor-${GITHUB_RUN_ID}.txt: PASS — wrote and read back smoke-36254806431
  4. curl (example.com/redacted) (expect blocked): PASS — HTTP 000 (connection blocked as expected)

Result: All checks PASSED ✅

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • example.com
  • github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"
    - "github.com"

See Network Configuration for more information.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions github-actions Bot added smoke-cloud-hypervisor smoke-copilot-network-isolation Copilot network-isolation egress smoke test labels Sep 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com) reachable — HTTP 403
✅ Blocked domain (example.com) — blocked (SSL/connection error)

Overall: PASS

@lpcox

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • api.github.com
  • example.com

[!TIP]
api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.github.com"
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode ✅

  1. GitHub MCP ✅ — Retrieved 2 merged PRs (Upgrade gh-aw and recompile workflows #9041, fix: tolerate benign VMM thread churn in confinement re-verification #9017)
  2. GitHub.com Connectivity ✅ — HTTP 200
  3. File Write/Read ✅ — Smoke test file verified
  4. BYOK Inference ✅ — Running in direct BYOK mode via api-proxy → api.githubcopilot.com

Overall Status: PASS

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) with api-proxy sidecar forwarding to api.githubcopilot.com.

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Result
API status ✅ PASS
gh check ✅ PASS
File status ✅ PASS

Overall result: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by Smoke Claude for #9033 · claude · haiku45 · 30 AIC · ⊞ 4.7K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot Engine — PASS ✅

@Copilot @lpcox

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Services Connectivity

  • Redis PING: ✅ (PONG)
  • Postgres pg_isready: ✅ (accepting connections)
  • Postgres SELECT 1: ✅ (1)

Overall: PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison Results

Runtime Host Version Chroot Version Match?
Python Python 3.12.14 Python 3.12.14 ✅ YES
Node.js v24.21.0 v22.23.2 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Overall: FAILED — Node.js version mismatch between host and chroot environment. smoke-chroot label not applied since not all tests passed.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ✅ 1/1 passed ✅ PASS
Bun hono ✅ 1/1 passed ✅ PASS
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world ✅ N/A ✅ PASS
.NET json-parse ✅ N/A ✅ PASS
Go color ✅ 1/1 passed ✅ PASS
Go env ✅ 1/1 passed ✅ PASS
Go uuid ✅ 1/1 passed ✅ PASS
Java gson ✅ 1/1 passed ✅ PASS
Java caffeine ✅ 1/1 passed ✅ PASS
Node.js clsx ✅ passed ✅ PASS
Node.js execa ✅ passed ✅ PASS
Node.js p-limit ✅ passed ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — PASS

Notes
  • All repositories cloned successfully.
  • Java (mvn) runs required -Dmaven.repo.local=/tmp/m2-repo because ~/.m2 was pre-created as root-owned and read-only in this sandbox, blocking Maven's default local repository creation. This is a sandbox filesystem permission issue unrelated to the firewall/proxy — the proxy settings (squid-proxy:3128) applied correctly (confirmed via JAVA_TOOL_OPTIONS proxy properties visible in Maven output), and once the local repo path was redirected, both mvn compile and mvn test succeeded normally through the proxy.

Warning

Firewall blocked 8 domains

The following domains were blocked by the firewall during workflow execution:

  • api.nuget.org
  • bun.sh
  • dc.services.visualstudio.com
  • deno.land
  • dl.deno.land
  • github.com
  • releaseassets.githubusercontent.com
  • repo.maven.apache.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.nuget.org"
    - "bun.sh"
    - "dc.services.visualstudio.com"
    - "deno.land"
    - "dl.deno.land"
    - "github.com"
    - "releaseassets.githubusercontent.com"
    - "repo.maven.apache.org"

See Network Configuration for more information.

Generated by Build Test Suite for #9033 · copilot · auto · 25.7 AIC · ⊞ 11.8K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Upgrade gh-aw and recompile workflows
fix: tolerate benign VMM thread churn in confinement re-verification
GitHub PR checks ✅
Playwright title check ✅
File write/read ✅
Discussion lookup/comment ✅
AWF build (npm ci && npm run build) ✅
Overall: PASS

Warning

Firewall blocked 13 domains

The following domains were blocked by the firewall during workflow execution:

  • ab.chatgpt.com
  • accounts.google.com
  • android.clients.google.com
  • api.github.com
  • clients2.google.com
  • collector.github.com
  • contentautofill.googleapis.com
  • github.com
  • github.githubassets.com
  • msfeed25.pkgs.visualstudio.com
  • update.googleapis.com
  • www.google.com
  • www.gstatic.com

[!TIP]
api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "accounts.google.com"
    - "android.clients.google.com"
    - "api.github.com"
    - "clients2.google.com"
    - "collector.github.com"
    - "contentautofill.googleapis.com"
    - "github.com"
    - "github.githubassets.com"
    - "msfeed25.pkgs.visualstudio.com"
    - "update.googleapis.com"
    - "www.google.com"
    - "www.gstatic.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: API Proxy OpenTelemetry Tracing — All Scenarios ✅

# Scenario Result
1 Module Loading ✅ otel.js loaded; isEnabled(): true; exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled + internals
2 Test Suite ✅ 68/68 tests passed (3 suites: otel.test.js, otel-fanout.test.js, otel-workload-identity.test.js)
3 Env Var Forwarding ✅ env-passthrough.ts forwards GITHUB_AW_OTEL_TRACE_ID/GITHUB_AW_OTEL_PARENT_SPAN_ID; api-proxy-env-config.ts forwards GH_AW_OTLP_ENDPOINTS, OTEL_EXPORTER_OTLP_ENDPOINT, and trace context
4 Token Tracker Integration ✅ onUsage callback present in token-tracker-http.js (OTEL hook point)
5 OTEL Diagnostics ✅ No local otel.jsonl fallback written (expected — GH_AW_OTLP_ENDPOINTS was configured with a live Sentry OTLP endpoint). Squid access log confirms ~19 successful TCP_TUNNEL:HIER_DIRECT CONNECTs from the api-proxy container (172.30.0.30) to o205451.ingest.us.sentry.io:443, confirming spans were exported through Squid to the configured OTLP collector during the run

Overall: PASS. OTEL tracing pipeline (module init → span creation → env propagation → OTLP export through Squid) is functioning end-to-end with the live Sentry endpoint.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • o205451.ingest.us.sentry.io

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "o205451.ingest.us.sentry.io"

See Network Configuration for more information.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@lpcox
lpcox merged commit 6189e18 into main Sep 26, 2026
163 of 170 checks passed
@lpcox
lpcox deleted the copilot/gate-task-level-model-routing branch September 26, 2026 16:28

This branch was successfully deployed

1 active deployment
aoai-model — f539d15b Deployed Sep 26, 2026 by lpcox via conclusion #1780
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gate task-level model routing behind experimental config

3 participants