I was on call last Thursday when a newly‑released AI‑assistant feature hit production. A user hit “Upgrade” twice in rapid succession, the webhook fired, and—boom—our service posted two successful charges to Stripe. The support inbox erupted, churn spiked, and the ops team spent an hour scouring logs for a duplicate‑payment that didn’t even have a traceable request ID. The problem? Our “idempotency” check lived in an in‑memory map that vanished when the pod restarted.
- Generate a request‑scoped UUID v7 and store it as the idempotency key.
- Lock the key *before* you talk to Stripe; use PostgreSQL advisory locks or a distributed lock service.
- Wrap every external call in a `context.Context` with deadline, circuit‑breaker, and jittered back‑off.
- Persist the final payment status together with the key for at least 72 hours.
- Prefer a lightweight Go‑only path for low‑volume services; reach for Temporal or Step Functions when you need saga‑style retries.
Before you start: Go 1.24+, PostgreSQL 17, Stripe API v2025‑03‑31, `github.com/jackc/pgx/v5`, `github.com/stripe/stripe-go/v76`, UUID v7 generator (`github.com/google/uuid` v1.5+), and a basic understanding of Go contexts and SQL transactions.
What is Idempotent Billing and Why It’s Critical for AI Agents
To implement idempotent billing for AI agents in Go, use Stripe‑like idempotency keys. Generate a unique key per transaction request and check your database first. Lock the key, process payment via Stripe’s `PaymentIntent`, and store the result. This ensures duplicate requests yield the same successful charge, preventing double billing at scale.
Why Idempotency is Non‑negotiable for Subscription Systems
A subscription platform gets millions of tiny “click‑to‑upgrade” events per day. AI‑driven UI layers often retry automatically when they see a 5xx response, so the same user can fire the same request three times in under a second. If your back‑end treats each request as brand new, you’ll see a surge of duplicate charges that instantly erodes trust.
The High Cost of Double‑Charging and User Churn
Stripe’s 2025 case study shows a 99.5 % drop in double‑charge tickets after they hardened their idempotency pipeline. The financial hit is two‑fold: you pay chargeback fees, and you lose customers who walk away the next time they see a $‑20 surprise on their statement.
Core Principles for Idempotent Payment Architecture
Idempotency Keys: The Foundation of Reliable Transactions
Think of a key as a “once‑only ticket”. It must be:
- **Globally unique** – UUID v7 gives you time‑ordered monotonicity, which is handy for debugging.
- **Request‑scoped** – Include the source (webhook, frontend, internal job) so you can differentiate retries from separate purchases.
- **Persisted** – Store it in a table with a status column (`pending`, `succeeded`, `failed`) and a TTL column.
// go:generate go run github.com/google/uuid/v7/cmd/uuidgen
type IdempotencyRecord struct {
Key uuid.UUID `pg:"id"` // primary key
UserID uuid.UUID `pg:"user_id"` // foreign key
Status string `pg:"status"` // pending|succeeded|failed
CreatedAt time.Time `pg:"created_at,default:now()"`
ExpiresAt time.Time `pg:"expires_at"` // 72h after CreatedAt
// Stripe's PaymentIntent ID for audit
IntentID string `pg:"intent_id"`
}
Transactional Guarantees and State Management in Go
You need **atomicity** between the idempotency check and the Stripe call. The usual pattern is:
- Begin a DB transaction.
- Acquire a **row‑level lock** (`SELECT … FOR UPDATE`) or an **advisory lock** keyed by the UUID.
- If a record exists and is `succeeded`, return the stored result immediately.
- Otherwise, call Stripe **inside** the same transaction context (so you can roll back on failure).
- Commit the transaction only after Stripe confirms the charge.
// filename: billing.go
// go:1.24
package billing
import (
"context"
"database/sql"
"fmt"
"time"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"github.com/stripe/stripe-go/v76"
"github.com/stripe/stripe-go/v76/paymentintent"
)
type Service struct {
db *pgx.Conn // pooled via pgxpool in production
}
// CreateCharge attempts an idempotent charge.
func (s *Service) CreateCharge(ctx context.Context, userID uuid.UUID, amount int64, currency string) (string, error) {
// 1️⃣ generate key – caller should have already done this, but we guard anyway
key := uuid.New() // UUID v7 by default in 1.24
tx, err := s.db.BeginTx(ctx, pgx.TxOptions{})
if err != nil {
return "", fmt.Errorf("begin tx: %w", err)
}
defer tx.Rollback(ctx) // safe when already committed
// 2️⃣ lock / upsert the idempotency record
var rec IdempotencyRecord
err = tx.QueryRow(ctx,
`SELECT * FROM idempotency_keys WHERE id=$1 FOR UPDATE`,
key,
).Scan(&rec.Key, &rec.UserID, &rec.Status, &rec.CreatedAt, &rec.ExpiresAt, &rec.IntentID)
if err != nil && err != pgx.ErrNoRows {
return "", fmt.Errorf("fetch idempotency record: %w", err)
}
// If we already have a succeeded record, return it early.
if rec.Status == "succeeded" {
return rec.IntentID, nil
}
// 3️⃣ Prepare Stripe request with the same key
piParams := &stripe.PaymentIntentParams{
Amount: stripe.Int64(amount),
Currency: stripe.String(currency),
// The idempotency key we generated
IdempotencyKey: stripe.String(key.String()),
// Attach a description for audit
Description: stripe.String(fmt.Sprintf("User %s upgrade", userID)),
// We want to capture immediately for subscription upgrades
CaptureMethod: stripe.String(string(stripe.PaymentIntentCaptureMethodAutomatic)),
}
// Propagate context with a 10‑second deadline
piParams.SetContext(ctx)
// 4️⃣ Call Stripe with retry+backoff (see later section)
pi, err := paymentintent.New(piParams)
if err != nil {
// Store the failure for later analysis
_, insErr := tx.Exec(ctx,
`INSERT INTO idempotency_keys (id, user_id, status, expires_at, intent_id)
VALUES ($1,$2,'failed', now() + interval '72 hour', $3)
ON CONFLICT (id) DO UPDATE SET status='failed', intent_id=$3`,
key, userID, "", // intent_id empty on failure
)
if insErr != nil {
return "", fmt.Errorf("store failed key: %w (original: %v)", insErr, err)
}
return "", fmt.Errorf("stripe payment failed: %w", err)
}
// 5️⃣ Persist successful intent
_, err = tx.Exec(ctx,
`INSERT INTO idempotency_keys (id, user_id, status, expires_at, intent_id)
VALUES ($1,$2,'succeeded', now() + interval '72 hour', $3)
ON CONFLICT (id) DO UPDATE SET status='succeeded', intent_id=$3`,
key, userID, pi.ID,
)
if err != nil {
return "", fmt.Errorf("store success record: %w", err)
}
// 6️⃣ Commit transaction
if err = tx.Commit(ctx); err != nil {
return "", fmt.Errorf("commit tx: %w", err)
}
return pi.ID, nil
}
**Tip:** Use `pgxpool` instead of a raw `pgx.Conn` when scaling; the connection‑pooling article on my blog walks through configuring the pool for 2026‑level workloads.
Step‑by‑Step Implementation in Go (2026) with Code Examples
Designing the Idempotent Struct and Database Schema
Below is the minimal DDL that satisfies the requirements. The `expires_at` column is enforced by a daily `DELETE` job.
-- go:1.24
CREATE TABLE idempotency_keys (
id UUID PRIMARY KEY,
user_id UUID NOT NULL,
status TEXT NOT NULL CHECK (status IN ('pending','succeeded','failed')),
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
expires_at TIMESTAMPTZ NOT NULL,
intent_id TEXT
);
-- Index for fast cleanup
CREATE INDEX idx_idempotency_expires ON idempotency_keys (expires_at);
Integrating with Stripe’s `PaymentIntent` and Idempotency Keys
Stripe’s SDK already accepts an `IdempotencyKey` field. Most tutorials skip the **context** part, but 2026 best practice is to pass a deadline **and** wrap the call in a circuit‑breaker.
// go:1.24
func chargeWithRetry(ctx context.Context, params *stripe.PaymentIntentParams) (*stripe.PaymentIntent, error) {
// 2‑second base backoff, jitter ±30%
backoff := backoff.NewExponentialBackOff()
backoff.InitialInterval = 2 * time.Second
backoff.RandomizationFactor = 0.3
backoff.MaxElapsedTime = 30 * time.Second
// Simple circuit breaker (state stored in memory; replace with Redis for distributed)
cb := circuitbreaker.NewBreaker(circuitbreaker.Options{
MaxFailures: 5,
ResetTimeout: 1 * time.Minute,
})
var pi *stripe.PaymentIntent
operation := func() error {
if !cb.Allow() {
return fmt.Errorf("circuit open")
}
var err error
pi, err = paymentintent.New(params)
if err != nil {
// only count network‑level errors for the breaker
if stripeErr, ok := err.(*stripe.Error); ok && stripeErr.HTTPStatusCode >= 500 {
cb.Fail()
} else {
cb.Success()
}
return err
}
cb.Success()
return nil
}
err := backoff.Retry(operation, backoff)
if err != nil {
return nil, fmt.Errorf("payment intent failed after retries: %w", err)
}
return pi, nil
}
Implementing Atomic Operations and Conflict Resolution Logic
Two common patterns exist for preventing two pods from processing the same key at the same time:
| Approach | Pros | Cons |
|---|---|---|
| **Row‑level `SELECT … FOR UPDATE`** | Simple, works with any RDBMS | Holds a lock for the whole transaction (can become a bottleneck under heavy load) |
| **PostgreSQL advisory lock** | Very cheap, can lock on arbitrary 64‑bit key | Requires you to map a UUID to `bigint`, adds a tiny conversion step |
| **Distributed lock service (e.g., Redis RedLock)** | Works across multiple clusters | Adds another moving part, latency penalty |
Below is an advisory‑lock implementation that fits nicely into the earlier `CreateCharge` function.
func advisoryLockKey(id uuid.UUID) int64 {
// Use the first 8 bytes of UUID as int64. Collisions are astronomically unlikely.
b := id[:8]
return int64(binary.BigEndian.Uint64(b))
}
// Acquire lock, blocked until timeout or context cancellation.
func acquireLock(ctx context.Context, tx pgx.Tx, key uuid.UUID) error {
lockID := advisoryLockKey(key)
_, err := tx.Exec(ctx, "SELECT pg_advisory_xact_lock($1)", lockID)
return err
}
Replace the earlier row‑level lock with:
if err := acquireLock(ctx, tx, key); err != nil {
return "", fmt.Errorf("advisory lock failed: %w", err)
}
Production‑Grade Error Handling and Retry Loops
Go `Context`: Handling Network Timeouts Gracefully
Never fire a Stripe request without a deadline. In production we use a **15‑second** timeout for the whole payment flow, and we propagate that context downstream.
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
If the timeout fires, the retry wrapper (shown earlier) automatically backs off and respects the context cancellation.
Implementing Circuit Breakers and Jittered Backoffs
The `github.com/sony/gobreaker` library offers a battle‑tested breaker. Combine it with `github.com/cenkalti/backoff/v4` for jitter.
// go:1.24
import (
"github.com/cenkalti/backoff/v4"
"github.com/sony/gobreaker"
)
var (
breaker = gobreaker.NewCircuitBreaker(gobreaker.Settings{
Name: "stripe-payment",
MaxRequests: 3,
Interval: 2 * time.Minute,
Timeout: 30 * time.Second,
ReadyToTrip: func(counts gobreaker.Counts) bool {
// Trip after 5 consecutive failures
return counts.ConsecutiveFailures > 5
},
})
)
Apply it inside `chargeWithRetry` as shown in the previous section.
Logging and Alerting Strategies for Failed Transactions
Structured logging came native in Go 1.24 via the `log/slog` package. Use a consistent key set:
- `event=payment_attempt`
- `user_id`
- `key`
- `status=failed|succeeded`
- `error` (if any)
- `duration_ms`
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
logger.Info("payment_attempt",
"user_id", userID,
"key", key,
"status", "failed",
"error", err.Error(),
"duration_ms", time.Since(start).Milliseconds(),
)
Alert on the `status=failed` metric combined with a spike in `duration_ms > 5s`. A simple Prometheus rule:
- alert: HighPaymentFailureRate
expr: rate(payment_attempt_total{status="failed"}[5m]) > 0.01
for: 2m
labels:
severity: critical
annotations:
summary: "Payment failures > 1% in last 5 min"
description: "Investigate Stripe connectivity or DB lock contention."
Architectural Trade‑offs for Scale: Golang vs. Niche Solutions
Choosing Between Goroutines with Mutex Locks vs. PostgreSQL Advisory Locks
| Aspect | Goroutine + Mutex (in‑process) | PostgreSQL Advisory Lock |
|---|---|---|
| **Scope** | Limited to a single pod | Works across all pods |
| **Latency** | Near‑zero (in‑memory) | ~1 ms round‑trip |
| **Failure mode** | Pod crash loses lock state | Locks survive pod restarts |
| **Complexity** | Simple, but requires careful lock ordering | Slightly more SQL, but safe under churn |
If your traffic is under **10 k rps** and you run a single autoscaling group, the mutex approach can be acceptable. Push beyond that, and the advisory lock wins because it prevents “split‑brain” scenarios when a pod dies while holding a lock.
Pure Go Solution vs. Dedicated Orchestration (Temporal, AWS Step Functions)
| Feature | Pure Go + PostgreSQL | Temporal SDK | AWS Step Functions |
|---|---|---|---|
| **State durability** | Relies on DB transaction | Built‑in history store | Managed DynamoDB |
| **Saga‑style compensation** | Manual code | First‑class `Workflow` |