Skip to content

fix(agentex): only bracket IPv6 literals when building the OTLP metrics URL#323

Closed
parveshsaini wants to merge 2 commits into
scaleapi:mainfrom
parveshsaini:parveshsaini/fix/otlp-metrics-ipv6-brackets
Closed

fix(agentex): only bracket IPv6 literals when building the OTLP metrics URL#323
parveshsaini wants to merge 2 commits into
scaleapi:mainfrom
parveshsaini:parveshsaini/fix/otlp-metrics-ipv6-brackets

Conversation

@parveshsaini

@parveshsaini parveshsaini commented Jun 17, 2026

Copy link
Copy Markdown

What

run_worker built the worker's OTLP metrics endpoint by unconditionally wrapping DD_AGENT_HOST in IPv6 literal brackets:

metrics_url = f"http://[{host_url}]:4317" if host_url else None

This change extracts a small build_metrics_url helper that brackets the host only when it is an IPv6 literal (i.e. it contains a colon), and uses the host as-is otherwise.

Why

Per RFC 3986, brackets are only valid around an IPv6 literal. For the common cases, a hostname (localhost), a Kubernetes service name (datadog-agent), or an IPv4 literal (10.0.0.5) - the old code produced a malformed URL like http://[localhost]:4317.

That metrics_url flows into TemporalClientFactory.create_client_from_env(...)create_client(...)OpenTelemetryConfig(url=metrics_url). The Temporal SDK validates the URL eagerly when building the Runtime, raising Invalid OTel URL: invalid IPv6 address. Because that construction is inside the try block in create_client, it is re-raised as TemporalConnectionError, so the worker fails to start entirely whenever DD_AGENT_HOST is set to anything other than a bare IPv6 literal.

This also matches how DD_AGENT_HOST is already used elsewhere in the repo without brackets (e.g. app.py's statsd_host=os.getenv("DD_AGENT_HOST", "localhost")).

How it was verified

Reproduced on Linux (WSL Ubuntu, Python 3.12, temporalio 1.23.0 per uv.lock): feeding http://[localhost]:4317 / http://[datadog-agent]:4317 / http://[10.0.0.5]:4317 to OpenTelemetryConfig raises Invalid OTel URL: invalid IPv6 address; the bracket-less forms are accepted.

Added tests/unit/temporal/test_run_worker_metrics_url.py covering hostname / IPv4 (not bracketed), IPv6 literal (bracketed), and the unset case (None). The hostname/IPv4 assertions fail against the previous always-bracket behavior and pass with this change.

uv run ruff check src/temporal/run_worker.py tests/unit/temporal/test_run_worker_metrics_url.py   # All checks passed!
uv run ruff format --check ...                                                                     # 2 files already formatted
uv run pytest tests/unit/temporal/test_run_worker_metrics_url.py -m unit -q                        # 6 passed

Closes #313

Greptile Summary

  • Adds a build_metrics_url helper for constructing the worker OTLP metrics endpoint from DD_AGENT_HOST.
  • Brackets IPv6 literals while leaving hostnames and IPv4 addresses unbracketed.
  • Adds unit tests covering unset hosts, hostnames, IPv4, IPv6, explicit ports, and already-bracketed IPv6 inputs.

Confidence Score: 5/5

The change is narrowly scoped to OTLP metrics URL construction and is covered by focused unit tests for the relevant host forms.

The helper behavior matches the described URL requirements, and the tests exercise unset, hostname, IPv4, IPv6, explicit port, and already-bracketed IPv6 cases.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex captured the baseline metrics URL state as documented in the before-state log.
  • T-Rex captured the post-change metrics URL state as documented in the after-state log.
  • T-Rex compared the before and after outputs to confirm the updated host formatting and URL expansion.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (2): Last reviewed commit: "fix(agentex): handle host:port and brack..." | Re-trigger Greptile

@parveshsaini parveshsaini requested a review from a team as a code owner June 17, 2026 19:03
Comment thread agentex/src/temporal/run_worker.py Outdated
@parveshsaini

Copy link
Copy Markdown
Author

Hey Team! If the proposed changes looks right, can we have a quick review here please?

Thanks! @danielmillerp @RoxyFarhad @smoreinis @MichaelSun48 @declan-scale

@smoreinis

Copy link
Copy Markdown
Collaborator

thanks for flagging this! should get addressed as part of #346

@smoreinis

Copy link
Copy Markdown
Collaborator

closing in favor of 346

@smoreinis smoreinis closed this Jul 7, 2026
scale-sf added a commit that referenced this pull request Jul 7, 2026
## Summary
- fix worker OTLP metrics URL construction for normal `DD_AGENT_HOST`
values like `localhost`, service names, and IPv4 literals
- keep IPv6 literals bracketed and preserve explicit ports
- add focused unit coverage for metrics URL construction

Closes #313. Supersedes stale external PR #323 with the same underlying
fix on a fresh branch.

## Tests
- `uv run ruff check agentex/src/temporal/run_worker.py
agentex/tests/unit/temporal/test_run_worker_metrics_url.py`
- `uv run --package agentex-backend --group test pytest
agentex/tests/unit/temporal/test_run_worker_metrics_url.py -m unit -q`
- `uv run --package agentex-backend --group test pytest
agentex/tests/unit/temporal -m unit -q`
- Runtime repro with `temporalio 1.23.0`: old
`http://[{DD_AGENT_HOST}]:4317` URLs reject `localhost`,
`datadog-agent`, and `10.0.0.5` with `Invalid OTel URL: invalid IPv6
address`; `build_metrics_url(...)` outputs are accepted by
`Runtime(telemetry=TelemetryConfig(metrics=OpenTelemetryConfig(url=...)))`.

<!-- greptile_comment -->

<h3>Greptile Summary</h3>

Fixes the OTLP metrics URL construction for Temporal workers by
replacing the broken `http://[{DD_AGENT_HOST}]:4317` string
interpolation with a dedicated `build_metrics_url` function that
correctly handles all common `DD_AGENT_HOST` shapes and adds focused
unit tests for each case.

- `build_metrics_url` correctly handles plain hostnames, IPv4 literals,
bare IPv6 addresses (brackets them), pre-bracketed IPv6 (including the
previously-identified empty-trailing-port edge case `"[::1]:"`, now
covered by a test), and explicit ports for all host types.
- One unguarded edge case remains: a malformed input that begins with
`"["` but has no closing `"]"` bypasses the inner parsing block and is
then double-bracketed by the final colon-presence check, producing an
invalid URL like `http://[[::1]:4317`.

<details><summary><h3>Confidence Score: 5/5</h3></summary>

Safe to merge — the fix correctly resolves the invalid IPv6-bracket
wrapping for all realistic DD_AGENT_HOST values and is well-covered by
the new unit tests.

The core bug (wrapping every host in IPv6 brackets) is definitively
fixed and tested across all meaningful host shapes. The one unguarded
path — a malformed input that begins with an unclosed bracket — is an
unrealistic operator configuration and does not affect production
behaviour.

No files require special attention; the logic in run_worker.py is
straightforward and the test file covers all realistic cases.
</details>

<details><summary><h3>Important Files Changed</h3></summary>

| Filename | Overview |
|----------|----------|
| agentex/src/temporal/run_worker.py | Replaces the broken
`http://[{DD_AGENT_HOST}]:4317` template with `build_metrics_url`,
correctly handling plain hostnames, IPv4, bare IPv6 (brackets them),
pre-bracketed IPv6 (strips and re-brackets), and explicit ports. One
edge case remains: a malformed input like `"[::1"` (missing closing
bracket) bypasses the inner block and then gets double-bracketed by the
trailing guard, producing an invalid URL. |
| agentex/tests/unit/temporal/test_run_worker_metrics_url.py | New unit
tests covering None/empty input, plain hostnames, IPv4, bare IPv6,
pre-bracketed IPv6 (including the previously-reported `"[::1]:"`
empty-port case), and explicit-port variants. No tests for the malformed
`"[::1"` (missing closing bracket) case. |

</details>

<details><summary><h3>Flowchart</h3></summary>

<a href="#gh-light-mode-only">

```mermaid
%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[build_metrics_url] --> B{host_url falsy?}
    B -- Yes --> C[return None]
    B -- No --> D[host = host_url.strip]
    D --> E{host empty?}
    E -- Yes --> C
    E -- No --> F[port = 4317 default]
    F --> G{host starts with bracket?}
    G -- Yes --> H[find closing bracket]
    H --> I{bracket found?}
    I -- Yes --> J[parse rest after bracket]
    J --> K{rest starts with colon?}
    K -- Yes --> L[port = rest after colon or default]
    K -- No --> M[port stays default]
    L --> N[host = inner IPv6 address]
    M --> N
    I -- No --> O[host unchanged with leading bracket - BUG PATH]
    G -- No --> P{exactly one colon in host?}
    P -- Yes --> Q[split host and port]
    P -- No --> R[host unchanged]
    Q --> S{colon in host?}
    N --> S
    O --> S
    R --> S
    S -- Yes --> T[wrap host in brackets]
    S -- No --> U[return http://host:port]
    T --> U
```

</a>
<a href="#gh-dark-mode-only">

```mermaid
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[build_metrics_url] --> B{host_url falsy?}
    B -- Yes --> C[return None]
    B -- No --> D[host = host_url.strip]
    D --> E{host empty?}
    E -- Yes --> C
    E -- No --> F[port = 4317 default]
    F --> G{host starts with bracket?}
    G -- Yes --> H[find closing bracket]
    H --> I{bracket found?}
    I -- Yes --> J[parse rest after bracket]
    J --> K{rest starts with colon?}
    K -- Yes --> L[port = rest after colon or default]
    K -- No --> M[port stays default]
    L --> N[host = inner IPv6 address]
    M --> N
    I -- No --> O[host unchanged with leading bracket - BUG PATH]
    G -- No --> P{exactly one colon in host?}
    P -- Yes --> Q[split host and port]
    P -- No --> R[host unchanged]
    Q --> S{colon in host?}
    N --> S
    O --> S
    R --> S
    S -- Yes --> T[wrap host in brackets]
    S -- No --> U[return http://host:port]
    T --> U
```

</a>
</details>

<sub>Reviews (2): Last reviewed commit: ["fix(agentex): default empty
OTLP
metrics..."](24de57e)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=42166768)</sub>

<!-- /greptile_comment -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: run_worker wraps DD_AGENT_HOST in IPv6 brackets, producing an invalid OTLP metrics URL that crashes the worker

2 participants