vercel

otel

OTEL tracing for Vercel

by vercel

TypeScript

TypeScript96.5%
JavaScript3%
CSS0.4%
HTML0.1%

Commit activity

Last 12 weeks · 0 commits

Full graph →

Community health

1 of 6 standards met

Community profile →
37
✓README○License○Contributing○Code of Conduct○Issue Template○PR Template

Recent PRs & issues

Active · Last activity 6d ago
See all on GitHub →
bryan-hunter

Summary Fixes #220. tracks a process-local root span per trace ID. When concurrent requests on one instance continue the same trace (all have the same incoming ), the first request's root claims the trace. When that root ends, the processor force-ends the other requests' in-flight spans, roots included. Any attributes, events and calls those requests make afterwards are dropped. This shows up on Fluid compute, where one instance serves concurrent requests: queue- or workflow-driven steps that continue a parent trace (for example, eve subagents running as concurrent workflow steps) export spans that are cut short and missing their final attributes. In the affected traces, 26 of 28 truncated spans ended within 5 ms of the trace's local root span. None of 450 healthy spans did. Repro : The new tests also cover the other two symptoms in #220: each request now gets its own and its own flush. They also cover the late-child case in vercel/eve#4256, where a span started under an already-ended request became a "root" and force-ended its sibling, which then ended again ("You can only call on a span once", as in #189). All three fail on . Fix The processor now tracks the local root per request. A span whose parent is remote or absent starts a new local root, and a child inherits its parent's root. Roots are stored in a keyed by span, so entries need no cleanup. Ending a root now ends only the spans opened under that request. The flush wait is keyed by the root's span ID instead of the trace ID. Behavior preserved When a request's root ends, any of its spans still open are ended, so they are exported before the flush. That was why the force-end was added in #45. Covered by the new single-request test, which includes a grandchild. Request-context attributes and the flush still apply to each request's root span. A span started under a local parent after that request's root has ended still becomes a new root, as before. Validation in : 42 passed. The three new tests (two requests sharing a trace, per-request attributes and , late children) fail against 's processor and pass with the fix. , , () and on the changed files all pass. 🤖 Generated with Claude Code

bryan-hunter · 6d ago
nabesuke5098

Version: 2.1.3 (same on as of 2026-10-14), 2.12.0 What we need We run a Nuxt (Nitro) app on Vercel with and ship spans through a Trace Drain. Two things must happen before a span reaches the runtime exporter: 1. Sanitize values that are set after the span starts. is enough for URL, client IP and user agent (HTTP instrumentations set them at start). It is not enough for response headers, span names updated later, exception events, and the status message of a failed span, which contains the PostgreSQL error text (for example ). 2. Drop spans whose export decision needs end-time data. We suppress control SQL (, , , …) unless the span failed or was slow. is set after start and the duration is known only at end, so a Sampler cannot make this decision. Why the public API does not allow it The OpenTelemetry SDK 2.x interface has (experimental), which is the hook for exactly this: the span is still writable, it is called right before . only forwards and to the configured processors (composite-span-processor.ts), so never reaches them. receives a . The spec says a processor must not modify it, so sanitizing there means mutating an ended span and relying on being the span's own object. always adds (sdk.ts). is not exported from the package, and adds another instead of replacing the runtime one. So there is no public way to wrap the drain path with a filtering processor or exporter. What we do today (works, but depends on SDK internals) A processor placed first in rewrites , , and inside , before the runtime sees the span. Another processor clears on inside for control SQL, so skips the span. Both work with the current SDK because returns the live objects. The spec gives no guarantee, so we pin the behaviour with tests against the real SDK and re-check on every upgrade. Proposal Any one of these would let us stop mutating ended spans: 1. Forward from to the configured processors (the SDK's own already does this). This covers sanitizing. It does not cover dropping a span, because is immutable by spec. 2. Expose the runtime exporter, for example by exporting , or by letting take a wrapper (). Then an app can filter or sanitize spans on the drain path with ordinary exporter composition, which is spec-compliant. 3. A hook** in the configuration, , applied by the runtime path before . Option 2 is the most general. Options 1 + 2 together cover both needs. Minimal reproduction of the gap Related #220 (concurrent requests sharing a trace id). We hit that one too in the same app. #213 / #214 / #215 (export path changes). A public filtering point on the drain path would be independent of those.

nabesuke5098 · 6d ago
mehmetaliabbasoglu

Version: 2.1.3 (node and edge bundles) What happens tracks open requests in and , keyed by trace id. A span counts as a root only if it has no parent or its trace id is not tracked yet. Two requests with the same can overlap in one process, for example parallel fetches from one browser page that Fluid Compute serves on one instance. Then the second request's entry span is treated as a child of the first request: 1. It gets no request context attributes (no , , ...). 2. No is registered for it. 3. When the first request's root span ends, the processor calls on the second request's span. So the status code that is set later is lost, and the duration is wrong. The third point can explain #189 ("You can only call end() on a span once"): the framework ends the span a second time. Reproduction (plain Node, no Next.js). On 2.1.3 it prints , , . Expected: each incoming request is its own root, with its own attributes, its own and its own end. Suggested fix: decide request membership by span, not by trace id. A span joins a request when its parent span belongs to that request. A span with no parent, or with a remote parent that is not part of an open request, starts a new request. We run a local patch with this rule. With the patch, the reproduction prints , , . Note: open PRs #214 and #215 also key their new per-trace state by trace id, so they would have the same problem with concurrent same-trace requests.

mehmetaliabbasoglu · 1w ago
mjarosie

The issue is quite similar to https://github.com/vercel/otel/issues/188, but it is reproducible locally and only with Bun. I've got a llama server running locally, example model: Versions: Calling the LLM like this: Running it with Node, I'm getting a successful response: : Running it with Bun, I'm getting a "Invalid JSON response" error: : I did a bit of digging, and it turns out that when you remove this statement that wraps the call to which makes a clone of the to calculate its size: https://github.com/vercel/otel/blob/4745daa7559cd41d2fceedeadb56aeec7724735a/packages/otel/src/instrumentations/fetch.ts#L547-L568 the error doesn't appear anymore. It seems like referring to in the if statement somehow causes the within not really resulting in the being cloned.

mjarosie · 1mo ago
snehakhobragade90

After Vercel’s legacy OTel collector was removed (org-wide ~2026-08-06) and Trace Drain became the export path, Datadog APM no longer receives our application spans under the passed to . What we expect (pre–Aug 6 / still true on stage·preview): (from ) What we get in production Datadog today: Only platform spans: / (etc.) present — app name string is not Zero nested app spans with or in the same traces Bumping 1.11.0 → 1.14.2 (per support) did not restore this. Stage/preview still emit + ; prod Drain path does not. This looks like a Drain / runtime pipeline issue after , not misconfigured in the app. Asking for a platform fix (and optional client hardening). Related open work: #216, #214, #215, #213. Environment Same pattern across other NerdWallet apps using + Drain (not unique to this repo). Evidence 1) Prod Drain trace — platform spans only (no app ) Trace ID: Time: 2026-08-21 19:11:02 UTC Datadog: https://app.datadoghq.com/apm/trace/69d759ece5ffe32790394ee738a9977a Span tree (simplified): Absent from the entire tree: from app instrumentation (we only see / ) attributes (representative): Post– deploy, same shape continues (example trace , 2026-08-25). 2) Datadog queries Stage/preview spans still show and (non-Drain path). Prod is exclusively . 3) Client already sets what Datadog docs ask for Datadog’s Vercel integration says should be set for correct APM correlation. We do that. Drain still emits . Vercel Trace Drain docs example payload uses , which matches the platform-owned naming we observe (not the app’s ). Analysis vs source What the open-source client already does correctly: 1. Resource from / (~L117–124) () 2. Datadog for SERVER + + (~L173–183) 3. When Trace Drain is active, skip local OTLP/collector and rely on runtime export () Wired in (~L417–469) Introduced in #182 / #181 (1.14.0); hardened in #205 (1.14.2) 4. App spans enter Drain via → Why / did not fix us: The rename to and the absence of app spans happen after** (Drain / serverless-runtime pipeline), which is outside what a client bump can fix. References Datadog Vercel integration (serviceName note): https://docs.datadoghq.com/integrations/vercel/ Vercel Trace Drain reference: https://vercel.com/docs/drains/reference/traces Drain skip local export: #182, #205 Attach to platform traces (open): #216

snehakhobragade90 · 1mo ago
Structured data for AI agents

Repository: vercel/otel. Description: OTEL tracing for Vercel Stars: 97, Forks: 22. Primary language: TypeScript. Languages: TypeScript (96.5%), JavaScript (3%), CSS (0.4%), HTML (0.1%). Homepage: https://vercel.com/docs/observability/otel-overview Latest release: @vercel/otel@1.14.2 (4mo ago). Open PRs: 17, open issues: 38. Last activity: 6d ago. Community health: 37%. Top contributors: jankaifer, dvoytenko, github-actions[bot], kakadiadarpan, gaspar09, agadzik, bripkens, bengigone, ethshea, huozhi and others.