Skip to content

Trace Chronicle .NET client operations with OpenTelemetry

Register AddCratisChronicleInstrumentation() to trace Chronicle .NET client operations inside your application with OpenTelemetry. The client creates spans, timed records of operations, for event appends, unit-of-work commits and event handling in reactors and reducers.

These spans come from the activity source Cratis.Chronicle.Client. OpenTelemetry only records sources registered with its tracer provider; omitting this source produces no warning, just missing client spans.

Configure this instrumentation in your application host. To collect the kernel’s telemetry as well, configure server OpenTelemetry separately.

AddCratisChronicleInstrumentation() is an extension on TracerProviderBuilder in the Cratis.Chronicle.AspNetCore package, in the Cratis.Chronicle.AspNetCore.OpenTelemetry namespace. Add it next to the other instrumentation when you configure tracing:

using Cratis.Chronicle.AspNetCore.OpenTelemetry;
using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.DependencyInjection;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
public static class ClientTracingRegistration
{
public static void Configure(string[] args)
{
var builder = WebApplication.CreateBuilder(args);
builder.AddCratisChronicle();
builder.Services
.AddOpenTelemetry()
.ConfigureResource(resource => resource.AddService("orders-api"))
.WithTracing(tracing => tracing
.AddCratisChronicleInstrumentation()
.AddOtlpExporter());
var app = builder.Build();
}
}

View C# snippet source on GitHub

The snippet also needs the OpenTelemetry.Extensions.Hosting and OpenTelemetry.Exporter.OpenTelemetryProtocol packages, which provide AddOpenTelemetry() and AddOtlpExporter().

If your host does not reference Cratis.Chronicle.AspNetCore, such as a worker service, register the source by name instead. ClientActivity.SourceName is the constant Cratis.Chronicle.Client:

using Cratis.Chronicle.Diagnostics.OpenTelemetry.Tracing;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using OpenTelemetry.Trace;
public static class ClientTracingAddSourceRegistration
{
public static void Configure(string[] args)
{
var builder = Host.CreateApplicationBuilder(args);
builder.AddCratisChronicle();
builder.Services
.AddOpenTelemetry()
.WithTracing(tracing => tracing
.AddSource(ClientActivity.SourceName)
.AddOtlpExporter());
}
}

View C# snippet source on GitHub

By default, span operation and display names start with client. exactly as before. Set ChronicleTelemetryOptions.SpanNames to ClientSpanNames.Convention through WithTelemetry, or configure Telemetry:SpanNames as Convention, to opt in to cratis.chronicle.client.* names. Each operation still emits one span. The telemetry name reference lists all mappings and the legacy keys deprecated for removal in the next major version.

The tags below hold identifiers, not event content. Both naming modes also carry shared keys such as cratis.event_store.name, cratis.event_sequence.id, and cratis.correlation_id alongside these legacy keys. Both copies of an event source id follow the same identifier exposure setting.

Span Kind Tags
client.event_sequence.append Client event_store_name, namespace_name, event_sequence_id, event_source_type, event_source_id
client.event_sequence.append_many Client event_store_name, namespace_name, event_sequence_id
client.unit_of_work.commit Client correlation_id
client.unit_of_work.rollback Client correlation_id
client.reactor.handle Consumer event_store_name, namespace_name, event_sequence_id, reactor_id
client.reducer.handle Consumer event_store_name, namespace_name, event_sequence_id, reducer_id

A reactor or reducer span covers one delivery of events to that observer. Replay notifications do not produce one.

Failed append and handle operations set Error status and error.type to an exception type or a low-cardinality failure code, without recording exception messages. Append spans also carry cratis.chronicle.append.outcome: success, error, concurrency_violation, or constraint_violation.

AddOtlpExporter() follows the standard OpenTelemetry environment variables, so the same code works against any backend that accepts OTLP. Set the endpoint, and the headers when your backend needs them, in the environment of the application:

Terminal window
OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.example.com
OTEL_EXPORTER_OTLP_HEADERS=x-api-key=<your-api-key>
OTEL_SERVICE_NAME=orders-api

Read the key from your secret store or deployment configuration. Do not commit it, and do not put it in a tag or a log line.

For a local check, run the .NET Aspire Dashboard. This container receives OTLP traces and lets you inspect them in a browser:

Terminal window
docker run --rm -d --name aspire-dashboard \
-p 18888:18888 -p 4317:18889 \
-e DOTNET_DASHBOARD_UNSECURED_ALLOW_ANONYMOUS=true \
mcr.microsoft.com/dotnet/aspire-dashboard:latest

Start your application with the dashboard as the endpoint, then append an event:

Terminal window
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 dotnet run

Open http://localhost:18888, go to Traces, and look for your service. You should see client.event_sequence.append for the append, and, if a reactor observes that event, a client.reactor.handle span. Stop the dashboard with docker stop aspire-dashboard.

The dashboard allows anonymous access in this setup. Use it on your own machine only.

If client spans are missing, check the path from registration to export in this order:

  1. Confirm that AddCratisChronicleInstrumentation() or AddSource("Cratis.Chronicle.Client") is in the same WithTracing configuration as the exporter. The server’s Cratis.Chronicle source does not enable client spans.
  2. Append an event so there is an operation to record, then look again.
  3. Check the exporter endpoint. Its default is http://localhost:4317, which only works when a collector or backend is listening there.
  4. Check sampling. A sampler that records nothing, or a parent span that was not sampled, can exclude these spans along with the rest.

The spans Chronicle creates identify what happened and where, not what the event said. Two tags can still hold values you care about:

  • event_source_id is the client event source id exactly as you appended it by default (Raw). Kernel spans use cratis.eventsource.id. If you use an email address, a national id or another personal value, that value reaches your tracing backend unless you opt into Hashed or Omitted.
  • correlation_id is the correlation id of the unit of work. It is meant to be searched for, so do not derive it from anything private.

Apply the same checks when adding your own tags: include only identifiers and counts that your tracing backend is allowed to hold. Keep credentials, tokens, event content and read model content out of traces.

The client defaults to EventSourceIdTracing.Raw. Opt into Hashed to send a keyed pseudonym, or Omitted to leave out event_source_id. Both modes keep the original id in append requests and storage. Client spans use event_source_id; kernel spans use cratis.eventsource.id.

Supply a key of at least 32 characters from your secret store before starting the host. This worker example reads it from deployment configuration; the same WithTelemetry callback works with the ASP.NET Core Chronicle builder:

using Cratis.Chronicle;
using Cratis.Chronicle.Diagnostics;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
var builder = Host.CreateApplicationBuilder(args);
var hashKey = builder.Configuration["ChronicleSecrets:EventSourceIdHashKey"];
builder.AddCratisChronicle(configure: chronicle => chronicle.WithTelemetry(options =>
{
options.EventSourceIdTracing = EventSourceIdTracing.Hashed;
options.EventSourceIdHashKey = hashKey;
}));
await builder.Build().RunAsync();

View C# snippet source on GitHub

Hashed fails validation without a sufficiently long key. Raw and Omitted do not require one. The hash is HMAC-SHA256 truncated to 128 bits, encoded as 32 lowercase hexadecimal characters. Use the same key in clients and the kernel to correlate ids across processes. Rotating the key changes the hashes and breaks correlation with earlier traces.

For local development you can explicitly select EventSourceIdTracing.Raw. It exposes the id to exporters; do not use it with personal ids unless your tracing backend is allowed to hold that data. These options do not redact logs. See event source ids in telemetry for kernel configuration, metric exclusions, and log levels.

client.unit_of_work.commit and client.unit_of_work.rollback carry the unit of work’s correlation_id. The append, reactor and reducer spans do not carry it, so searching on a correlation id finds the commit or rollback span only.