Observability

Tracing spans and events, optional metrics, and the audit records of every exchange.

On this page

Three observability seams; none can change what happens to a message:

  • tracing: every send, receive, signature, decryption, receipt, MDN, SMP lookup, revocation check and audit write runs in a documented span.
  • metrics (feature metrics): seven counters and histograms through the metrics facade.
  • AuditSink: one evidence record per exchange leg.

The library never installs a subscriber or recorder and reads no environment variable. With nothing installed, spans cost a disabled-callsite check and metric macros are no-ops. Only an audit sink declared Durable can fail an exchange.

The tables below are a public contract, checked both ways against what the test suite emits (tests/telemetry_contract.rs, tests/metrics_contract.rs); changes are recorded in CHANGELOG.md.

Wiring it up

The application installs a subscriber and a recorder, for example:

[dependencies]
asx-rs = { version = "0.16", features = ["metrics"] }
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
metrics-exporter-prometheus = "0.17"
// in the application's main, once:
tracing_subscriber::fmt().json().with_env_filter("info,asx=info").init(); // asx=debug for the child spans
metrics_exporter_prometheus::PrometheusBuilder::new().install()?;

For OpenTelemetry, add tracing-opentelemetry as a layer: asx.* spans become OpenTelemetry spans under the caller's span (for example tower-http's TraceLayer). The library propagates no traceparent header.

Tracing

Target

Every span and event uses the target asx: filter with RUST_LOG=asx=info (asx=debug for the child spans).

Spans

Values learned during the operation (outcome, error.kind, ebms.code, duplicate, duration_ms, …) are recorded before the span closes; a field that does not apply stays empty.

SpanLevelFields
asx.sendINFOprotocol, profile, partner, message_id, outcome, error.kind, ebms.code, duration_ms
asx.receiveINFOprotocol, profile, partner, message_id, outcome, error.kind, ebms.code, duplicate, duration_ms
asx.signDEBUGalgorithm, reference_count
asx.verifyDEBUGalgorithm, signer.fingerprint, outcome
asx.encryptDEBUGalgorithm, key_transport
asx.decryptDEBUGalgorithm, outcome
asx.receiptDEBUGmessage_id, signed, nrr, outcome
asx.mdnDEBUGmessage_id, disposition, signed, outcome
asx.smp_lookupDEBUGparticipant_scheme, document_type, transport_profile, outcome
asx.revocationDEBUGmethod, source, signer.fingerprint, outcome, reason
asx.auditDEBUGrecord_id, durability, outcome
  • asx.send covers one As4::send, As2::send, As2::send_async_mdn or As2::verify_async_mdn call, including the verification of the receipt or MDN.
  • asx.receive covers one request handled by an Inbound, from the first byte read to the response, the handler call included.
  • Child spans nest under whichever of the two is current, also on the crypto pool.
  • asx.mdn covers the verification of one MDN, synchronous or asynchronous, inside asx.send; disposition is the disposition type and modifier class (processed, processed/warning, processed/error, failed), never the partner's text; signed whether it was signed.
  • asx.revocation covers the check of one non-anchor certificate of a verified signer's path, top-down, after the signature and the sender rule. method is the source that decided (crl when the certificate names no source), source whether the data came from the cache or the network.

Field values: identifiers verbatim; signer.fingerprint is the lowercase hex SHA-256 of the DER certificate, the only certificate datum recorded; participant_scheme and document_type carry identifier schemes, never a participant value; duration_ms is an integer.

Never recorded: payload or wire bytes, filenames from a message, private keys, secrets, HTTP header values other than Content-Type, certificate subject or issuer names.

Values

FieldSpanValues
protocolasx.send, asx.receiveas2, as4
outcomeasx.senddelivered, sent, partner_error, indeterminate, config, transport, storage
outcomeasx.receiveaccepted, duplicate, in_flight, pending, rejected, storage
outcomeasx.verify, asx.decrypt, asx.receipt, asx.mdn, asx.smp_lookup, asx.auditok, failed
outcomeasx.revocationgood, revoked, unknown, error
error.kindasx.send, asx.receiveconfig, rejected, partner_error, transport, indeterminate, storage
nrrasx.receiptverified, absent, mismatch, n/a
methodasx.revocationocsp, crl
sourceasx.revocationcache, network
reasonasx.revocationca_distrusted, not_obtainable, no_source, refresh_failing
durabilityasx.auditdurable, best_effort

ebms.code is an ebMS 3.0 §6.7 error code (EBMS:0001 … EBMS:0303). error.kind names an ErrorKind (storage is ErrorKind::Backend) and is also an audit record's error_kind. The receive outcome pending and the closing event fragment received are reserved and not emitted.

Closing events

Every asx.send and asx.receive span ends with exactly one event inside it.

SpanConditionLevelMessage
asx.senddelivered, or sent awaiting an asynchronous MDNINFOmessage sent
asx.sendthe partner answered with an eb:Error or a negative MDNWARNpartner returned an error
asx.sendtransmitted, no verifiable receiptERRORdelivery indeterminate
asx.sendfailed before or during transmission (config, transport, storage)WARNsend failed
asx.receiveacceptedINFOmessage received
asx.receivea fragment of an incomplete group was storedINFOfragment received
asx.receiveduplicate or in flightINFOduplicate message
asx.receiverejected, before or after authenticationWARNmessage rejected

Other events:

  • an audit sink's failure: ERROR audit record not written (fields record_id, message_id, signal.sha256, durability) inside asx.audit;
  • revocation status revoked or unknown: WARN certificate revocation status (field outcome) inside asx.revocation;
  • a CA distrusted under the BDEW three-day rule: WARN issuing CA distrusted, and a CRL whose refresh keeps failing: WARN CRL refresh failing, both with fields ca.fingerprint (SHA-256 of the CA certificate), since (the last successful fetch, the passed nextUpdate or the first failed refresh, RFC 3339) and reason;
  • everything else is DEBUG.

Metrics

Feature metrics. Descriptions and units are registered on first emission. Label values come from the closed sets above; no label carries a partner, participant, message or certificate identifier.

NameTypeLabels
asx_messages_sent_totalcounterprotocol, outcome
asx_messages_received_totalcounterprotocol, outcome
asx_receipt_verifications_totalcounterprotocol, outcome, nrr
asx_revocation_checks_totalcountermethod, outcome
asx_audit_failures_totalcounterdurability
asx_send_duration_secondshistogramprotocol, outcome
asx_receive_duration_secondshistogramprotocol, outcome
  • asx_messages_* and the duration histograms: when asx.send / asx.receive close, with the span's outcome.
  • asx_receipt_verifications_total: when the verification of an AS4 receipt or an AS2 MDN ends (outcome ok/failed).
  • asx_revocation_checks_total: when asx.revocation closes.
  • asx_audit_failures_total: when a sink returns Err.

There is no profile label; the preset name is the span field profile.

Audit records

Install an AuditSink per client with Builder::audit; without one nothing is stored, but the record is still returned by Verified::evidence() and Error::evidence(). Implementing a sink, and what a sink failure means, is on the persistence page.

When a record is written

LegDirection / outcomesignal
AS4 send, receipt verifiedoutbound / deliveredthe receipt as received
AS4 send, eb:Error receivedoutbound / partner_errorthe error signal
AS4 send, response unverifiableoutbound / indeterminatethe response bytes, if any
AS2 send (or verify_async_mdn) answered by an MDNoutbound / delivered, partner_error or indeterminatethe MDN
AS4 or AS2 receive, handler returned Okinbound / acceptedthe receipt or MDN issued for it
Duplicate, in flight, or refused (before or after authentication)—no record

A record is written before an inbound dedup claim is accepted and before a verified receipt or MDN is returned to you.

The stored form

AuditRecord is opaque; a sink stores its serde form. Bytes are standard base64 with padding, digests lowercase hex, recorded_at RFC 3339 UTC with three fractional digits, absent values null, enums snake_case; unknown fields are ignored on read, so newer records still load. Index accessors: record_id() (UUIDv7, the idempotency key), message_id(), partner(), signal().

{
  "record_id": "01929d3e-6a1b-7c3e-9f00-3b5c2e4d1a77",
  "recorded_at": "2026-10-16T09:30:12.345Z",
  "protocol": "as4", "direction": "outbound", "outcome": "delivered",
  "message_id": "c2a1f6e0-…@sender.example", "ref_to_message_id": null,
  "conversation_id": "conv-1", "local_party": "9900000000001", "partner": "9900000000002",
  "profile": "bdew", "service": "…", "action": "…",
  "message_timestamp": "2026-10-16T09:30:11.902Z",
  "signer": { "certificate_der": "MIIB…", "fingerprint_sha256": "3f1c…" },
  "references": [ { "uri": "#body", "digest_algorithm": "http://www.w3.org/2001/04/xmlenc#sha256", "digest": "q1w2…" } ],
  "mic": null,
  "signal": {
    "bytes": "PD94bWwg…", "sha256": "9a8b…", "message_id": "receipt-7@partner.example",
    "ref_to_message_id": "c2a1f6e0-…@sender.example", "timestamp": "2026-10-16T09:30:12.201Z",
    "signed": true, "nrr": [ { "uri": "#body", "digest_algorithm": "…", "digest": "q1w2…" } ]
  },
  "disposition": "receipt", "error_kind": null, "ebms_code": null, "quirks": []
}

tests/fixtures/audit/record-as4-delivered.json and record-as2-mdn.json pin this form. quirks lists every partner quirk the leg relied on.

Check your sink before production

asx::testkit::audit::conformance checks the durability() claim through fresh handles to the same store.

use asx::{AuditRecord, BoxFuture};
use asx::testkit::audit::{conformance, AuditSinkHarness, MemoryAuditSink};

/// Opens handles to the store under test. `MemoryAuditSink` stands in for
/// yours: its clones share one store, as connections to a database do.
struct Harness(MemoryAuditSink);

impl AuditSinkHarness for Harness {
    type Sink = MemoryAuditSink;
    fn open(&self) -> BoxFuture<'_, MemoryAuditSink> {
        Box::pin(async move { self.0.clone() })
    }
    fn read_all<'a>(&'a self, sink: &'a MemoryAuditSink) -> BoxFuture<'a, Vec<AuditRecord>> {
        Box::pin(async move { sink.records() })
    }
}

#[tokio::main]
async fn main() {
    let failures = conformance(&Harness(MemoryAuditSink::new())).await;
    assert!(failures.is_empty(), "{failures:#?}");
}
CheckFails a sink that …
audit.declares_durabledeclares BestEffort (nothing else runs)
audit.recorded_is_readableacknowledges a record that its own handle cannot read
audit.records_survive_a_fresh_handlekeeps records per handle
audit.every_field_round_tripsloses or alters a field of sample_records()
audit.bytes_round_trip_exactlyalters non-UTF-8 bytes (a 1 MiB signal included)
audit.record_is_idempotent_by_idstores a retried record twice
audit.distinct_ids_are_distinct_recordsmerges records that share a message_id
audit.concurrent_records_all_landloses records under 64 concurrent writes over 8 handles