Pulse (Observability)

What You Get

After starting your API, the Pulse dashboard is available at /pulse/ui/ (default credentials: admin / pulse).

Every feature, in detail

What each part measures, and what it will not tell you. Everything below runs in your process: no Redis, no collector, no agent.

Request Tracing

Every request gets latency, throughput and a trace ID, propagated as the W3C traceparent header. If a request arrives already carrying a valid one, Pulse adopts the upstream trace rather than starting a new one, so a trace survives crossing a service boundary. Outbound calls made through pulse.WrapHTTPClient carry it onward.

A request slower than the threshold (1s by default) is marked slow. Sampling thins what is stored, never what is counted: at 50% sampling the request counts, error rates and SLO compliance still reflect every request, and errors and slow requests are stored whatever the rate. That is the difference between sampling you can trust at 3am and sampling that quietly halves your error count.

Database Monitoring

A GORM plugin captures every query with its duration and the file and line that issued it, which is the difference between knowing a query is slow and knowing where it comes from. Queries are aggregated by pattern, so a thousand lookups with different ids read as one row.

N+1s are detected and ranked by total impact rather than by count, because twenty queries in a hot endpoint matter more than two hundred in a nightly job. Connection pool stats sit beside them, since an exhausted pool and a slow query look identical from the outside. Slow SELECTs can optionally be explained: Pulse asks the database why, reads the sequential scan or the filesort off the plan, and does it at most once per pattern per hour, off the request path.

Runtime Metrics

Heap, goroutine count and GC pauses, sampled continuously, with goroutine leak detection on top: a count that only ever rises is the shape of a leak, and it is reported as one rather than left for you to notice on a graph.

A flame graph is available when profiling is enabled, defaulting to real CPU time rather than goroutine stacks. The distinction matters: with goroutine stacks, a parked database connection is counted on every tick and the code actually burning CPU shows up as a sliver.

Error Tracking

Panics are recovered and recorded with a stack trace and the request body that caused them. Errors are fingerprinted, so the same failure from ten thousand requests is one entry with a count, and classified automatically (validation, database, timeout, auth and so on) so the shape of an outage is visible before you read a single trace.

Captured context is redacted before it is stored: passwords, tokens, card numbers, JWTs and keys are stripped from request bodies, error messages and outbound URLs, whatever the field is called. The capture is the point, and a capture that stores a password is a liability rather than a tool.

Health Checks

/live and /ready, shaped for Kubernetes: liveness says the process is not wedged, readiness says it can serve. Your own checks plug in, and a Grit project registers its Redis connection automatically. The composite status is the worst of its parts, and flapping is detected, so a dependency that fails every other probe reads as unstable instead of alternating between fine and down.

Alerting

Threshold rules fire in two phases, which is the part that makes them usable: a condition has to hold, not merely occur, so one slow second does not page anybody. Cooldowns stop a firing rule repeating. Notifications go to Slack, Discord, email or a webhook, and webhooks are HMAC signed so the receiver can tell they came from you.

Above simple thresholds there are two more useful things. SLOs with multiwindow burn-rate alerting, following the Google SRE workbook, computed from exact per-minute counts so sampling never skews compliance. And per-route baselines: each route's hourly p95 is kept for 30 days, so Pulse can say a route got slower rather than only that it is slow, and a sustained regression against last week's median pages once, with the numbers you would want before deciding to roll back.

Prometheus

Standard exposition format at /pulse/metrics, for Grafana or anything else that scrapes. Pulse counts its own failures and exports those too: dropped writes, and the case where buffer capacity rather than your retention setting is what limits how far back you can see. A monitoring tool that quietly stops recording is worse than one that is plainly off.

Dashboard

Twelve pages at /pulse/ui/, embedded in the Go binary with go:embed, so there is no second thing to deploy: Overview, Routes, Database, Errors, Runtime, Health, Alerts, SLOs, USE, Test Runs, Flame Graph and Settings. It updates over a WebSocket, and the API behind it is protected with HS256 JWTs.

Two pages are worth knowing about before you need them. USE is the utilisation / saturation / errors grid for the host. Test Runs overlays a load test onto the timelines: run k6 against the app and the run appears as a band, with a before, during and after comparison per route, query and N+1 deltas, time to recovery, and a CPU profile captured over that window.

Configuration

Pulse is mounted in routes.go and configured via environment variables:

# Observability — Pulse
PULSE_ENABLED=true
PULSE_USERNAME=admin
PULSE_PASSWORD=pulse
PULSE_STORAGE=memory                 # "memory" (default) | "sqlite"
PULSE_STORAGE_DSN=pulse.db           # path used when PULSE_STORAGE=sqlite

# Runtime profiles: heap, allocations, goroutines, mutex, block and CPU.
# Needs this AND PULSE_PROFILE_ENABLED in the server's environment, because a
# profile describes your program and one forgotten setting should not expose it.
PULSE_PROFILING=false
PULSE_MUTEX_PROFILE_FRACTION=0       # 0 is off; 100 is a common starting point
PULSE_BLOCK_PROFILE_RATE=0           # 0 is off; 10000 is a common starting point

# The database's own plan for a slow SELECT, on the Database page. EXPLAIN only,
# never EXPLAIN ANALYZE, so the slow query is not run a second time.
PULSE_EXPLAIN_SLOW_QUERIES=false

Pulse v1.0 is mounted with functional options and a context.Context (the context drives clean shutdown of the dashboard WebSocket and background samplers). The scaffolded mount call:

// Mount Pulse observability
if cfg.PulseEnabled {
    pulseOpts := []pulse.Option{
        pulse.WithAppName(cfg.AppName),
        pulse.WithCredentials(cfg.PulseUsername, cfg.PulsePassword),
        pulse.WithExcludePaths("/studio/*", "/sentinel/*", "/docs/*", "/pulse/*"),
        pulse.WithPrometheus(),
        // Pulse's error middleware snapshots a request-body snippet for context,
        // but restores ONLY the first 4096 bytes — silently truncating larger
        // uploads/JSON. Disable capture so the full body reaches the handler.
        pulse.WithRequestBodyCaptureDisabled(),
    }
    if cfg.IsDevelopment() {
        pulseOpts = append(pulseOpts, pulse.WithDevMode())
    }
    if cfg.PulseStorage == "sqlite" && cfg.PulseStorageDSN != "" {
        pulseOpts = append(pulseOpts, pulse.WithSQLite(cfg.PulseStorageDSN))
    }

    p := pulse.Mount(context.Background(), r, db, pulseOpts...)

    // Register health checks for connected services
    if svc.Cache != nil {
        p.AddHealthCheck(pulse.HealthCheck{
            Name:     "redis",
            Type:     "redis",
            Critical: false,
            CheckFunc: func(ctx context.Context) error {
                return svc.Cache.Client().Ping(ctx).Err()
            },
        })
    }
}

Endpoints

Custom Health Checks

The Mount() function returns a Pulse instance. Use it to register health checks for any dependency:

p := pulse.Mount(context.Background(), r, db, pulseOpts...)

// Redis health check
p.AddHealthCheck(pulse.HealthCheck{
    Name:     "redis",
    Type:     "redis",
    Critical: false,  // non-critical = won't fail /ready
    CheckFunc: func(ctx context.Context) error {
        return redisClient.Ping(ctx).Err()
    },
})

// External API health check
p.AddHealthCheck(pulse.HealthCheck{
    Name:     "payment-api",
    Type:     "http",
    Critical: true,  // critical = will fail /ready if down
    CheckFunc: func(ctx context.Context) error {
        resp, err := http.Get("https://api.stripe.com/v1/health")
        if err != nil {
            return err
        }
        defer resp.Body.Close()
        if resp.StatusCode != 200 {
            return fmt.Errorf("status %d", resp.StatusCode)
        }
        return nil
    },
})

The database health check is registered automatically when you pass a *gorm.DB to Mount().

Alerting

Pulse ships with 5 default alert rules (high latency, high error rate, high memory, goroutine leak, health check failure) that run out of the box — no configuration needed. They fire against the metrics Pulse already collects and surface in the dashboard.

Built-in Default Rules

Data Storage

By default, Pulse uses in-memory ring buffers — fastest, but data resets on restart. For persistence across restarts, set PULSE_STORAGE=sqlite in your .env; the mount call then adds the pulse.WithSQLite() option automatically:

PULSE_STORAGE=sqlite       # switch from in-memory to persistent
PULSE_STORAGE_DSN=pulse.db # SQLite file (or ":memory:")

if cfg.PulseStorage == "sqlite" && cfg.PulseStorageDSN != "" {
    pulseOpts = append(pulseOpts, pulse.WithSQLite(cfg.PulseStorageDSN))
}

Prometheus Integration

Pulse exposes 18+ metrics in Prometheus exposition format at /pulse/metrics. Connect it to Grafana or any Prometheus-compatible tool.

Key metrics include:

• pulse_http_requests_total — Total HTTP requests by method, path, status
• pulse_http_request_duration_seconds — Request duration histogram
• pulse_http_error_rate — Current error rate percentage
• pulse_runtime_goroutines — Active goroutine count
• pulse_runtime_heap_bytes — Heap allocation in bytes
• pulse_db_query_duration_seconds — Database query duration
• pulse_health_check_status — Health check status (1=healthy, 0=unhealthy)

Disabling Pulse

Set PULSE_ENABLED=false in your .env to disable Pulse entirely. This skips the mount and adds zero overhead to your application.