Instrument Go Services for AI Cost Attribution (2026 Guide)

I was on call at 02:17 am, staring at a flame‑graph that showed **99 %** of our CPU time burning inside the OpenAI SDK. The feature flag we’d just launched was “smart‑search” – a RAG endpoint that stitched together a user’s query, fetched a vector embedding, ran a few LLM calls, and returned a result. The cost report from the finance team came back an hour later: **$12 K** spent in the last 24 h, and we had no idea which tenant or which request was the culprit. Turns out we never tied the LLM token usage back to the request context, so the billing data was a single aggregate number. The whole night turned into a sprint to surface per‑feature cost data before the next sprint planning.

How to instrument Go services for AI cost attribution in 2026

To instrument Go services for AI cost attribution in 2026, wrap vendor SDKs with OpenTelemetry tracing and Prometheus metrics. Propagate a context containing user, session, and feature IDs. Create detailed spans for each LLM call, recording token usage and cost estimates as span attributes, then export this telemetry for dashboards and alerting.

Why fine‑grained cost attribution is a 2026 engineering imperative

The shift from observability to business observability

Observability used to be “Did the service crash?” Today the question is “Did the AI spend exceed the budget for this feature?” Finance and product teams need to see the same telemetry that SREs use for latency and error rates. The rise of **GenAI** workloads—LLM inference, embeddings, vector indexing—means every request can cost anywhere from a few cents to several dollars. When you run a multi‑tenant SaaS, that variance blows up the bottom line.

Financial leaks in LLM & embedding workloads

A recent Segment case study showed a **32 %** reduction in monthly LLM spend after they built per‑tenant cost attribution for their RAG pipelines. The leak was simple: background jobs kept generating embeddings for stale documents, and nobody could tell which tenant triggered them. Without telemetry, the leak was invisible until the finance alarm went off.

“Datadog’s 2025 State of Serverless report indicates that 67 % of teams using serverless functions for AI workflows lack the telemetry to attribute costs below the service level, leading to significant budgeting blind spots.”

In my experience, the moment you start surfacing **token‑level** cost data, you instantly get a new class of alerts—*budget‑drift* alerts—that are far more actionable than generic CPU or latency alarms.

Building your Go context wrapper for AI cost propagation

Standardizing the request‑scoped metadata bag

The first step is to agree on a **metadata struct** that travels with `context.Context`. I call it `CostContext`. It carries the identifiers we need for chargeback:

// go.mod: go 1.24
// costcontext.go
package telemetry

import (
	"context"
)

// CostContext holds business‑level identifiers.
type CostContext struct {
	UserID    string // end‑user or API key owner
	SessionID string // optional, for UI tracing
	FeatureID string // logical feature flag, e.g. "smart-search"
	TenantID  string // SaaS tenant
}

// ctxKey is an unexported type to avoid collisions.
type ctxKey struct{}

var costKey = ctxKey{}

// WithCost returns a new context carrying CostContext.
func WithCost(parent context.Context, cc CostContext) context.Context {
	return context.WithValue(parent, costKey, cc)
}

// FromCost extracts CostContext; panics if missing (helps catch bugs early).
func FromCost(ctx context.Context) CostContext {
	if v := ctx.Value(costKey); v != nil {
		return v.(CostContext)
	}
	panic("CostContext missing from context")
}

**Why panic?** In production you want the service to fail fast if you forget to propagate the bag; the panic will be caught by your global recover middleware and turned into a 500 with a clear log entry.

Integrating with popular frameworks (Gin, Fiber, Echo)

All three major Go HTTP frameworks let you inject middleware that enriches the request context.

// gin_cost_mw.go
package middleware

import (
	"net/http"

	"github.com/gin-gonic/gin"
	"myapp/telemetry"
)

// CostMiddleware extracts headers and stores a CostContext.
func CostMiddleware() gin.HandlerFunc {
	return func(c *gin.Context) {
		cc := telemetry.CostContext{
			UserID:    c.Request.Header.Get("X-User-ID"),
			SessionID: c.Request.Header.Get("X-Session-ID"),
			FeatureID: c.Request.Header.Get("X-Feature-ID"),
			TenantID:  c.Request.Header.Get("X-Tenant-ID"),
		}
		// Fallback defaults to avoid missing data.
		if cc.UserID == "" {
			cc.UserID = "anonymous"
		}
		ctx := telemetry.WithCost(c.Request.Context(), cc)
		c.Request = c.Request.WithContext(ctx)
		c.Next()
	}
}

For **Fiber** and **Echo** the code looks the same—just replace the middleware signature. I keep the middleware in a shared repo so each service can import it and stay consistent. If you need a quick refresher on context propagation basics, check out my internal tutorial on the topic.

Instrumenting OpenAI & Anthropic SDK calls with metrics & tracing

Wrapping the official SDKs for token‑level telemetry

OpenAI’s Go SDK (v5) returns a `ChatCompletionResponse` that contains a `Usage` field with `PromptTokens`, `CompletionTokens`, and `TotalTokens`. We can intercept that at the wrapper level.

// openai_wrapper.go
package ai

import (
	"context"
	"fmt"

	oteln "go.opentelemetry.io/otel"
	oteltrace "go.opentelemetry.io/otel/trace"

	openai "github.com/openai/openai-go/v5"
	"myapp/telemetry"
	"github.com/prometheus/client_golang/prometheus"
)

var (
	llmTokens = prometheus.NewCounterVec(
		prometheus.CounterOpts{
			Name: "llm_tokens_total",
			Help: "Total LLM tokens consumed, labeled by tenant and feature.",
		},
		[]string{"tenant_id", "feature_id", "type"},
	)
	// Estimated cost in micros (to avoid float errors)
	llmCostMicros = prometheus.NewCounterVec(
		prometheus.CounterOpts{
			Name: "llm_cost_micros_total",
			Help: "Estimated LLM cost in micro‑dollars, labeled by tenant and feature.",
		},
		[]string{"tenant_id", "feature_id"},
	)
)

func init() {
	// Register metrics; ignore error because duplicate registration panics only in tests.
	_ = prometheus.Register(llmTokens)
	_ = prometheus.Register(llmCostMicros)
}

// OpenAIClient abstracts the SDK; could be swapped for Anthropic.
type OpenAIClient struct {
	client *openai.Client
	tracer oteltrace.Tracer
}

// NewOpenAIClient creates a wrapper with OpenTelemetry tracer.
func NewOpenAIClient(apiKey string) *OpenAIClient {
	cfg := openai.ClientConfig{
		APIKey: apiKey,
	}
	return &OpenAIClient{
		client: openai.NewClient(cfg),
		tracer: oteln.Tracer("ai.openai"),
	}
}

// Chat wraps the ChatCompletion call, records tokens & cost.
func (c *OpenAIClient) Chat(ctx context.Context, req openai.ChatCompletionRequest) (openai.ChatCompletionResponse, error) {
	cc := telemetry.FromCost(ctx)

	// Start a span for the LLM call.
	_, span := c.tracer.Start(ctx, "OpenAI.ChatCompletion",
		oteltrace.WithAttributes(
			oteltrace.String("tenant.id", cc.TenantID),
			oteltrace.String("feature.id", cc.FeatureID),
			oteltrace.String("user.id", cc.UserID),
			oteltrace.String("model", req.Model),
		))
	defer span.End()

	resp, err := c.client.ChatCompletions.Create(ctx, req)
	if err != nil {
		span.RecordError(err)
		span.SetStatus(oteltrace.StatusCodeError, err.Error())
		return resp, fmt.Errorf("openai chat call failed: %w", err)
	}

	// Token telemetry.
	usage := resp.Usage
	llmTokens.WithLabelValues(cc.TenantID, cc.FeatureID, "prompt").Add(float64(usage.PromptTokens))
	llmTokens.WithLabelValues(cc.TenantID, cc.FeatureID, "completion").Add(float64(usage.CompletionTokens))
	llmTokens.WithLabelValues(cc.TenantID, cc.FeatureID, "total").Add(float64(usage.TotalTokens))

	// Rough cost model: $0.002 per 1k tokens for gpt‑3.5‑turbo.
	const pricePerK = 0.002
	cost := float64(usage.TotalTokens) / 1000 * pricePerK
	// Convert to micro‑dollars to stay integer.
	llmCostMicros.WithLabelValues(cc.TenantID, cc.FeatureID).Add(cost * 1_000_000)

	// Attach telemetry as span attributes.
	span.SetAttributes(
		oteltrace.Int64("llm.prompt_tokens", int64(usage.PromptTokens)),
		oteltrace.Int64("llm.completion_tokens", int64(usage.CompletionTokens)),
		oteltrace.Int64("llm.total_tokens", int64(usage.TotalTokens)),
		oteltrace.Float64("llm.estimated_cost_usd", cost),
	)

	return resp, nil
}

The wrapper does three things:

1. Starts an OpenTelemetry span that carries the request‑level identifiers.
2. Emits Prometheus counters for prompt/completion/total tokens and an estimated cost counter.
3. Records any SDK error as a span error, guaranteeing that failures surface in both tracing and metric dashboards.

You can create a similar wrapper for **Anthropic** using `github.com/anthropic/anthropic-go`—the token fields differ, but the pattern stays identical.

Attaching user, session, and feature IDs to every call

Because the wrapper extracts the `CostContext` from `context.Context`, any downstream job (e.g., a background worker that re‑processes failed messages) only needs to call `telemetry.WithCost(parentCtx, cc)` before making the LLM call. This guarantees the attribution chain remains unbroken.

Mapping costs to business logic with structured logging

Correlating Prometheus metrics with application context

Most teams already ship JSON logs to Loki or Elasticsearch. By adding the same `tenant_id`/`feature_id` fields to logs, you can join logs with metrics in Grafana. Here’s a pattern using Go 1.21’s **SLOG**:

// logger.go
package log

import (
	"context"
	"log/slog"
	"myapp/telemetry"
)

var logger = slog.New(slog.NewJSONHandler(os.Stdout, nil))

// WithCost adds cost metadata to the logger.
func WithCost(ctx context.Context) slog.Logger {
	cc := telemetry.FromCost(ctx)
	return logger.With(
		slog.String("tenant_id", cc.TenantID),
		slog.String("feature_id", cc.FeatureID),
		slog.String("user_id", cc.UserID),
		slog.String("session_id", cc.SessionID),
	)
}

Now any log entry automatically carries the cost identifiers:

func handler(c *gin.Context) {
	ctx := c.Request.Context()
	l := log.WithCost(ctx)
	l.Info("starting smart-search")
	// …
}

Creating golden signals for AI feature performance

We already have latency histograms for HTTP. Add a **cost‑per‑request** histogram:

var costPerReq = prometheus.NewHistogramVec(
	prometheus.HistogramOpts{
		Name:    "ai_feature_cost_usd",
		Help:    "Histogram of estimated AI cost per request.",
		Buckets: []float64{0.001, 0.005, 0.01, 0.05, 0.1, 0.5},
	},
	[]string{"tenant_id", "feature_id"},
)

func init() {
	_ = prometheus.Register(costPerReq)
}

// In the wrapper after cost calc:
costPerReq.WithLabelValues(cc.TenantID, cc.FeatureID).Observe(cost)

When you put latency, error rate, and cost together, you get a **triple‑golden‑signal** dashboard that product managers love. The cost histogram instantly tells you “Feature X is hitting the $0.10 bucket far more often than we budgeted.”

Cost attribution in asynchronous & batch processing workflows

Propagating context through queues (SQS, NATS)

Most AI pipelines use a message broker for decoupling. The trick is to serialize the `CostContext` into the message header or payload.

type QueueMessage struct {
	Body        []byte
	Metadata    map[string]string // includes cost fields
}

// Producer side (e.g., after HTTP request)
func enqueue(ctx context.Context, payload []byte) error {
	cc := telemetry.FromCost(ctx)
	msg := &QueueMessage{
		Body: payload,
		Metadata: map[string]string{
			"tenant_id":  cc.TenantID,
			"feature_id": cc.FeatureID,
			"user_id":    cc.UserID,
			"session_id": cc.SessionID,
		},
	}
	// Serialize & push to NATS or SQS (omitted for brevity)
	return nil
}

// Consumer side
func consume(msg *QueueMessage) error {
	cc := telemetry.CostContext{
		TenantID:  msg.Metadata["tenant_id"],
		FeatureID: msg.Metadata["feature_id"],
		UserID:    msg.Metadata["user_id"],
		SessionID: msg.Metadata["session_id"],
	}
	ctx := telemetry.WithCost(context.Background(), cc)
	// Now any LLM call inside this goroutine inherits the same IDs.
	return processAIJob(ctx, msg.Body)
}

Because the context travels across process boundaries, you preserve attribution even for jobs that run minutes later.

Handling cost attribution for offline vector indexing

Embedding generation is often done in batch jobs that read from a database, compute an embedding, and write to a vector DB. The cost model here is similar: the number of tokens *embedding size* times the provider’s per‑token price. Add a small helper:

func recordEmbeddingCost(ctx context.Context, tokenCount int, model string) {
	cc := telemetry.FromCost(ctx)
	// Example: $0.0001 per token for ada embeddings.
	const pricePerToken = 0.0001
	cost := float64(tokenCount) * pricePerToken

	llmTokens.WithLabelValues(cc.TenantID, cc.FeatureID, "embedding").Add(float64(tokenCount))
	llmCostMicros.WithLabelValues(cc.TenantID, cc.FeatureID).Add(cost * 1_000_000)
}

Call this right after receiving the embedding from the provider. If the embedding is cached, emit a `cache_hit=”true”` label (see FAQ below).

Architectural trade‑offs: performance vs. fidelity

Sampling strategies for high‑volume inference

If you serve **10 k RPS** LLM calls, emitting a span for every request can overwhelm the collector. OpenTelemetry’s **head‑based sampling** lets you keep, say, 1 % of spans while still collecting full metrics.

sampler := oteltrace.TraceIDRatioBased(0.01) // 1 % keep rate
otel.SetTracerProvider(
	oteltrace.NewTracerProvider(
		oteltrace.WithSampler(sampler),
	))

You still record the token counters because they’re cheap; only the trace data is sampled.

Choosing cardinality for your labels in 2026

Prometheus loves low cardinality. In 2026, **VictoriaMetrics** and **Grafana Alloy** can aggressively down‑sample high‑cardinality series before they hit long‑term storage. My rule of thumb:

Label	Recommended Cardinality	Reason