Client SDKs

The SDKs record an agent's runs, and the steps, model calls and tool calls inside each, as OpenTelemetry spans, and send them to ingest. Each one runs a tracer provider of its own beside whatever OpenTelemetry your application already has, sends nothing outside a run, and takes prompts, completions and tool payloads off every span before it leaves the process unless you ask and the project's content mode allows.

Their source is the platform's repository, telmoni/telmoni, which holds them beside the ingest they send to, and each installs with its language's own tool, below. Each is built on its language's OpenTelemetry SDK and sends with the language's own HTTP client, Rust with reqwest. Python, TypeScript and Go carry nothing else; Rust also carries serde, serde_json, tracing and uuid.

InstallationCopy Anchor Link

$ pip install telmoni

A runCopy Anchor Link

A run is one execution of an agent. Open one around the agent's work, and inside it open a step for each unit of work, a model call around each call to a model, and a tool call around each tool. The SDK sends each span when it ends. Set TELMONI_API_KEY to a project API key holding the ingest scope.

from telmoni import Telmoni, Usage

telmoni = Telmoni.from_env()


@telmoni.run("support-bot")
def answer(question: str) -> str:
    with telmoni.step("plan"):
        with telmoni.model_call("gpt-4o-mini", provider="openai") as call:
            reply = ask_the_model(question)  # your provider's call
            call.answered(
                response_model=reply.model,
                usage=Usage(input_tokens=reply.usage.input, output_tokens=reply.usage.output),
            )
    with telmoni.tool_call("search"):
        results = search(reply.query)  # an exception marks the call failed
    return reply.text


with telmoni.run(
    "support-bot",
    user_id="user_42",
    session_id="conv_7",
    tags={"env": "prod"},
) as run:
    ...

Flush before a short-lived process exits (flush() / flush() / Flush(ctx) / flush()), or shut the client down, which flushes first; a long-lived one needs neither.

A run recorded after it endedCopy Anchor Link

A job that keeps its own log, or a hook that runs once the work it reports is over, hands the finished run over whole, as data: its id, a UUID you give and keep with the log; its agent, user, session and tags; its start and end and how it ended; and its steps and calls, each with its own times. A step holds the calls made inside it. A record the export queue has no room for is refused whole, before anything is queued: flush, or wait, and record it again. Its times are read as the clock of the process that sends it: where that clock is a time zone or a date out, ingest moves the record by as much, so time a record by the clock that sends it.

from datetime import datetime, timezone

from telmoni import ModelCallRecord, RunRecord, ToolCallRecord, Usage

at = lambda s: datetime(2026, 10, 12, 9, 0, s, tzinfo=timezone.utc)

telmoni.record_run(
    RunRecord(
        run_id="3d6f8a1c-5b2e-4f7a-9c0d-8e1f2a3b4c5d",
        agent="nightly-triage",
        session_id="conv_7",
        start=at(0),
        end=at(9),
        inside=[
            ModelCallRecord(
                model="gpt-4o-mini", provider="openai", start=at(0), end=at(2),
                usage=Usage(input_tokens=1200, output_tokens=85),
            ),
            ToolCallRecord(name="search", start=at(2), end=at(3), error_type="TimeoutError"),
        ],
    )
)
telmoni.flush()

The run is sent as the live calls would have sent it, through the same exporter, content rules and sample rate, in a trace of its own named by the run's id, whatever trace is current. Each span's id is derived from the run's id and the span's place in the record, and a record travels in bodies of its own, so the same record sent again is the same bytes, whatever else the client is sending, which Telmoni, where it reaches its Redis, answers within the hour as it answered the first, storing nothing twice: give a run the same id each time you send it. A record holds no content. The coding-agent plugins are built on it.

A run across servicesCopy Anchor Link

An agent whose work spans more than one service — an orchestrator handing a task to a worker, a tool served over HTTP — is still one run. Before the request leaves, inject writes into its headers the W3C traceparent of the span current inside the run and the run's id and agent as the W3C baggage members telmoni.run_id and telmoni.agent, beside whatever members the headers hold already. In the service that answers, extract reads them back as the run continued there: the steps and calls it opens carry the caller's run and take the caller's span as their parent, so the trace shows the work where it was asked for, and the trace list shows one run, not one a service.

# The orchestrator, inside a run.
with telmoni.step("delegate"):
    headers = {"Content-Type": "application/json"}
    telmoni.inject(headers)
    requests.post("https://worker.internal/work", json=task, headers=headers)

# The worker.
@app.post("/work")
def work(request):
    run = telmoni.extract(request.headers)
    with telmoni.step("work", run=run):
        ...

In TypeScript, inject takes the handle to write from, or writes from the active context where your application runs OpenTelemetry's context manager; in Rust, inject reads the current context, which a handle's attach() or FutureExt::with_context sets, and inject_from takes a handle's context(). The worker never ends a run it continued: the orchestrator's own span does, and until it arrives the run is open with what the worker sent. Headers that name no run — no traceparent, no telmoni.run_id, a malformed one — continue nothing, and a step opened then is outside a run and sends nothing. A run sampled out hands nothing on, so no half of it reaches Telmoni. The headers carry the run's id and agent and nothing else; a service sending under another project's key records the spans as that project's own run, so a forged header reaches no other tenant. Telmoni's own helpers write and read the headers; no propagator is set globally, since your application's propagators are its own.

What a run sendsCopy Anchor Link

A run is a span of its own, named invoke_agent <agent>, inside your current trace when one is current and a trace of its own otherwise. It carries the run's id (telmoni.run_id, a UUID), its agent (telmoni.agent), the user and conversation you named (user.id, gen_ai.conversation.id), the first 32 of its tags Telmoni would keep, the most it keeps, as telmoni.tag.<key>, and the sample rate. Every span inside the run carries the run's id, agent, user and session, so Telmoni knows each span's run without waiting for the trace's tree. The resource every span of an export shares names the SDK (telemetry.distro.name telmoni, its version, and the OpenTelemetry SDK's name, language and version) and, where you set them, the deployment: deployment.environment.name from the environment option or TELMONI_ENVIRONMENT (production, staging), and service.version from release or TELMONI_RELEASE (a version or a commit), which Telmoni keeps as the environment and release of every span and the trace list filters by. A step is a span named as you named it; a model call is chat <model> (or the operation you gave) with the model asked for, the model that answered, the provider and the token counts under the GenAI conventions' names, each a whole number from 0, another left off the call with one warning; a tool call is execute_tool <name>. The exception's class alone marks a failure, as error.type; no message, stack trace or event is ever sent. The ingest page lists every attribute and what the server keeps of each.

ContentCopy Anchor Link

Prompts, completions, tool arguments and results — input_messages, output_messages, system_instructions, arguments, returned(...) — are recorded on the span and taken off by the exporter before anything leaves the process. They are sent only when the client is built with capture_content on and the project's content mode, which the SDK learns from the server's first answer, is on. No project can be switched to on today, so content never leaves the process. The server's own drop at the door is the second lock, not the first.

SamplingCopy Anchor Link

sample_rate, on the client or on one run, is the share of runs sent, a number above 0 and up to 1. A run sampled out is dropped whole in the SDK, never through the trace's sampled flag, so your application's own sampling is untouched and the volume Telmoni sees is a dial you hold. A step or call still open when its run ends is dropped too, however late it ends.

Beside your own OpenTelemetryCopy Anchor Link

The SDK builds every part of its tracer provider itself — resource, sampler, limits, batching — so none of your OTEL_* variables reaches it, and it neither replaces your provider nor sends your spans to Telmoni. A run joins the trace that is current where it begins, and your spans opened inside it nest under the run. The run's sampled flag follows the trace it joins; a run that starts a trace of its own takes the decision your own sampler gives it — given as the sampler option in every SDK, Python falling back to your provider's sampler when you give none — so your application keeps its own sampling rate inside runs as outside them. A run inside a trace your application does not sample is still sent.

Answers and warningsCopy Anchor Link

The SDK batches spans and sends them in the background, on a bounded queue, so a slow Telmoni never slows an agent. Each request times out after 10 seconds. It retries a 429 or a 503 after its Retry-After, waiting at most 60 seconds, and a request that failed or timed out before its answer's head was whole after 1 second, then 2; after three attempts in all it drops the batch with one warning. Any other refusal is logged once and never retried, and an answer whose body breaks after its head, or runs past the 10 seconds, is decided by its status, read as far as it came. A batch whose body would pass the 1 MiB ingest takes, or that holds more than its 1,000 spans, is sent in halves, each within both, and a span that alone passes the 1 MiB is sent without the attributes ingest drops for their length — a key past 128 characters, a value past 1,024 — with one warning. Once a part fails as the rest would, Telmoni out of reach or busy past the retries, or the key refused, the parts after it are dropped with it; past a part refused for what it holds, they are sent. A number attribute JSON cannot carry, NaN or an infinity, is sent as no value, and so dropped, and a whole number past 64 bits as its text or a double, never costing its batch. Each attempt carries Telmoni-Sent-At, the agent's clock as it leaves, so ingest can move a run from a clock a time zone or a date out as one. A flush waits up to 30 seconds. An attribute the server drops is logged once: content with the project's content mode, which decides the drop, and a link to what is dropped at the door; text the field inventory does not name with the telmoni.tag.<key> that would keep it.

The key on the wireCopy Anchor Link

The key travels as a bearer over HTTPS. Plain HTTP is accepted only to this machine, for a server you run beside the agent: localhost, a name under it, an address in 127.0.0.0/8, or ::1, an IPv4-mapped address such as [::ffff:127.0.0.1] counting as its IPv4. Python, TypeScript and Go refuse any other http:// endpoint when the client is built, and Rust when the first export would go, with one warning. An endpoint given without a scheme gets https://, or http:// for this machine, as the CLI reads one; any scheme but those two is refused the same way, as is an endpoint the four SDKs could read two ways: one that is not printable ASCII once the whitespace around it is trimmed, or holds credentials, a percent-escape, a query, a fragment, a . or .. segment or a character a URL parser would escape, such as a space or |, or a port of 0 or past 65535, or whose host is neither a name of letters, digits, dots, hyphens and underscores, in labels of 1 to 63 characters with no hyphen at either end and the last no number, nor an IPv4 address in four decimal parts, nor an IPv6 address in brackets. Write an address in full, 127.0.0.1 rather than 127.1, and a name in another script in its xn-- form. No SDK follows a redirect, which could carry the key to another host, nor sends to this machine through a proxy, whatever the proxy variables name, since the proxy would read it; and the key never appears in a log, a warning or an error.

ConfigurationCopy Anchor Link

from telmoni import Config, Telmoni

telmoni = Telmoni(
    Config(
        endpoint="https://api.telmoni.com",
        api_key="telmoni_...",
        organization_id="org_019a0b8c3f2e7d41",
        capture_content=False,
        sample_rate=1.0,
        environment="production",
        release="2026.10.18",
    )
)

environment and release name the deployment every span is sent from, as the resource's deployment.environment.name and service.version; a blank one is unset, and nothing is sent for one unset.

Each SDK keeps the API key out of the configuration's printed form: Rust's Debug shows <redacted> in its place and serialization skips it; Python holds it as a Secret that repr, str and dataclasses.asdict show redacted; Go's String and GoString redact it and JSON skips it; TypeScript holds apiKey as a non-enumerable property, which console.log and JSON.stringify pass over.

Environment variablesCopy Anchor Link

from_env / fromEnv / NewFromEnv / Config::from_env read:

Every SDK reads these five and no others. OpenTelemetry's own OTEL_* variables are your application's, for its own exporters, and change nothing of the SDK's: OTEL_RESOURCE_ATTRIBUTES in particular reaches nothing the SDK sends.