Observe

@fougere/observability gives every operation a span, carries the trace across processes, turns the spans into metrics and log lines into the same trace, and sends all three over OTLP. Core holds none of it: an app that does not install the package pays nothing.

npm install @fougere/observability

pnpm add @fougere/observability

bun add @fougere/observability

yarn add @fougere/observability

One line

import { observability } from '@fougere/observability';

createApp({
  extensions: [observability({ service: 'blog', otlp: 'http://localhost:4318' })],
});

A frond that runs in its own process carries its half of the trace with it: put the extension in the frond's extensions/ directory, and it rises on whichever process serves that frond — fougere serve blog included.

// fronds/blog/extensions/observe.ts
import { observability } from '@fougere/observability';

export default observability({ service: 'blog', otlp: 'http://localhost:4318' });

Under a host that writes its boot file itself, the extension is named with its options, since a function cannot be written into a file:

// nuxt.config.ts
fougere: {
  observability: { service: 'blog', otlp: 'http://localhost:4318' },
  calls: { panel: true },        // `false` turns a key off; absent is the same
}

Each key is the name of an extension package and of its export — calls is import { calls } from '@fougere/calls' — so a misspelled key is refused rather than ignored.

Without otlp, nothing leaves the process; the spans, the metrics and the topology still answer.

What a span holds

startedAt comes from the wall clock, so two processes land on one timeline; ms from the monotonic clock, so a clock correction during a call does not move it.

The trace crosses the wire

The parent span travels in the invocation, not in an HTTP header, so it crosses whatever transport carries the call. The format is W3C Trace Context, which any collector reads:

app                     post.list        traceId a3f… spanId 01
  └─ blog (split)       post.list        traceId a3f… spanId 02, parent 01

Both sides of a split record the call — where it leaves and where it arrives — so the gap between the two spans is what the wire cost. A handler that calls another frond starts a new invocation; its parent is found in the context the first call still runs in (currentSpan()).

Metrics

Rate, errors and duration are one metric: a histogram of durations, split by operation and by whether it refused. Its count is the rate, its split is the error rate, its buckets are the latency. The fourth signal, saturation, counts the calls that have not finished yet (activeCalls()).

Cardinality is bounded by construction: frond × entity × operation × {ok, error}, plus the error code — all declared in the code. No id, user or trace ever becomes a dimension.

Working or waiting

A span's selfMs is its duration minus what its observed children took. An operation that calls two other fronds reports milliseconds and almost no self time: it is not slow, it is waiting — a column of durations cannot tell the two apart.

For that to be true, what runs under an operation is observed too: each SQL statement is charged to the operation that issued it. Its time comes out of selfMs, and statements counts it.

{ kind: 'operation', address: 'cart', operation: 'checkout', ms: 31.0, selfMs: 0.2, statements: 40 }

One statement per row is what an N+1 looks like, and that line says it without opening a trace. Divided by the call count, statements is statements per call.

Which statements ran

The count is always on. The detail — one span per statement — is off by default:

observability({ service: 'shop', otlp: '…', spanPerStatement: true })

item.listOneByOne   [0.54ms, self 0.47ms, 4 statements]     ← by default, one span

item.listOneByOne   [0.54ms, self 0.47ms, 4 statements]     ← with it, five
  items.select      [0.01ms]
  items.select      [0.01ms]
  items.select      [0.01ms]
  items.select      [0.01ms]

The count finds the slow operation; the detail explains it, and is turned on over that operation only. What decides is not CPU — a statement span costs well under a microsecond, against about 14 µs for the fastest SQLite select — but volume: a backend charges per span, and a page of forty rows would export forty.

Where an operation's work goes

placement says where an operation answers; reach says where its work goes:

fougere explain CommandeHandler.servable --root .
# Placement: commande / local
# Reach:     stock — 1 hop(s)

Comment the fronds: line out and the same operation reports 0 hop(s). Reach is read from the handler's constructor, so every operation of a handler reaches the same fronds. It is what keeps every crossing trace sampled, and what fougere load sets each threshold from: base + hops × perHop.

Logs on the trace

A log line is an announced fact, LogLine, and a destination is a handler that accepts it — no registration. observability() brings an OTLP destination with it, so otlp: is the whole wiring. Your own destination is a handler in one of your fronds:

import type { Fact, LogLine } from '@fougere/core';

export default class KeepHandler {
  async record(line: Fact<LogLine>): Promise<void> { … }
}

A line takes the trace id of the call it was written in, read when it is written. A line written outside any call — at boot, at shutdown — has no trace id, rather than a zero that would gather every process's startup into one trace. The console keeps its line, a destination that throws costs nobody theirs, and the log level filters first, so a line that is not printed is not shipped. Each level uses its own console method (debug, info, warn, error).

Sending it

OTLP over HTTP, batched every second, in JSON — no protobuf, no dependency. The three signals go to /v1/traces, /v1/metrics and /v1/logs of the address given; name metricsUrl when metrics go to another engine.

• A metric with no data points makes Prometheus reject the whole batch, so a process with no edges publishes no edge metric rather than an empty one.
• Metrics go out on every beat, even when no span finished: a gauge that stops being published reads as gone, not idle.
• An export that fails is reported to onError, silent by default: a trace never breaks a call.
• Before a process exits, flush() sends what is buffered.

The shape of the system

An app can be asked what shape it is in:

await call({ address: 'rpc', op: 'topology' });
// { fronds:   [{ frond: 'shop', placement: 'local', entities: 2, facades: 2 },
//              { frond: 'catalog', placement: 'remote', entities: 0, facades: 0 }],
//   edges:    [{ from: 'shop', to: 'catalog', count: 12, errors: 1 }],
//   declared: { fronds: [{ frond: 'shop', placement: 'local' },
//                        { frond: 'catalog',  placement: 'remote', at: 'http://127.0.0.1:4100' },
//                        { frond: 'shipping', placement: 'remote', at: 'http://127.0.0.1:4300' }],
//               edges:  [{ from: 'shop', to: 'catalog' }, { from: 'shop', to: 'shipping' }] },
//   active: 0, since: 1755861234567 }

fronds and edges are what this process saw: a frond is local because it is here, remote because it answered a call. declared is what the config says. Kept apart, they disagree exactly when something is wrong: shipping above is declared and has never answered — a dead edge, visible, instead of a smaller system that looks healthy. An address is cut to its host and port, since a declared one may carry credentials.

rpc.topology comes with this package; an app without it refuses the operation by name. The admin draws it on its Topology page.

When a receiver verifies the caller of a split call, invocation.caller names the frond that signed (Secure calls). Spans do not read it: callerFrond is the frond of the parent step in this process, not a verified caller.

On a dashboard

Anything that speaks OTLP reads this as it is. Moving between Jaeger with Prometheus and SigNoz changes two URLs and nothing else.

pnpm -C demos/observability dev      # three fronds, three processes
pnpm -C demos/observability load     # k6, in stages
pnpm -C demos/observability signoz   # a collector, OTLP on 4318, UI on 8080

import { entity, primary, text, number } from '@fougere/schema';

export default class Product extends entity({
  id: primary(),
  title: text({ min: 1 }),
  stock: number({ min: 0 }),
}) {}

The panels worth building, in the order you need them:

The two middle rows name a culprit rather than a symptom. Beyond them, a heatmap of the buckets shows a two-humped latency a p95 hides, and a table of operations sorted by p95 shows which one drags the global figure.

Wiring it by hand

The extension does this for you; without it, the parts are public:

import { tracing, metrics, otlp, type SpanSink } from '@fougere/observability';

const spans: SpanSink[] = [];             // who receives the spans — this app's list
const tracer = tracing(spans);
app.use(tracer.middleware);               // every operation
app.use('post', tracer.middleware);       // or one entity's

const measured = metrics(app);
spans.push(measured.sink);

const exporter = otlp({ service: 'blog', metrics: measured });
spans.push(exporter.sink);

The order matters: tracing() opens the span every log line of the call will carry. Nothing is opened while the list is empty. The list belongs to the app, so an app replaced by a reload stops feeding the metrics of the one that replaced it.