Conformance

Two separate things, often conflated:

They are complementary, and the second is a prerequisite for passing the first without burning support cycles. Do the self-hosted loop first.

Is the official service free?

Yes.

• The Interoperability Test Bed (ITB/GITB) that the service runs on is open-source, EUPL-licensed, and the EC states "All Test Bed software is free to use (also when used as a service)". It is explicitly not restricted to public administrations — "used by Public Administrations, International Organisations, Industry and Academia across the world".
• The eDelivery Conformance Testing Service exists to verify "a software package either commercial or Open Source". DIGIT owns, hosts and operates the infrastructure.
• Neither the service pages, the Service Offering Description, nor the 46-page AS4 User Guide mention a fee, price, or charge anywhere.

The cost is engineering time, not money — and the engineering cost is the part worth planning for (see What ASX still needs).

Caveat worth stating plainly: "no fee is documented anywhere" is strong evidence but is not the same as a published "this service is free of charge" sentence, which we did not find. Confirm in the registration email; it costs one line.

The official process

Source: eDelivery Conformance Testing Service — User Guide for AS4 testing v1.20, © European Union 2024.

Actors

Steps

1. Registration — email EC-EDELIVERY-SUPPORT@ec.europa.eu with: name and version of the product, specifications to test (AS4 Profile and/or SMP Profile), optional Profile Enhancements, external public URL of the SUT, planned start date, single point of contact. The SO creates your test-bed account.
2. Preparation — integrate the SUT with the platform, implement the SUT Controller, run basic connectivity tests.
3. Execution — run the suites from the GITB UI, resolve failures (with SO support).
4. Closing — receive the signed test report; the SO publishes it and adds your solution to the conformant list.

The SUT Controller

This is the part people underestimate. It is not part of AS4 — it is a test-harness protocol carried over AS4, using ordinary user messages whose eb:CollaborationInfo/Action names a command and whose parameters travel as eb:MessageProperties.

Submit/Deliver properties: MessageId, ConversationId, RefToMessageId, FromPartyId, FromPartyRole, ToPartyId, ToPartyRole, Service, ServiceType, Action, and originalSender/finalRecipient for the Four-Corner Model enhancement.

PMode profiles under test

All of them require compression, signing and encryption.

Payload profiles

Four parts, exercising multi-payload messages:

Caveat on this table. Rows marked ✅ reflect implemented and unit-tested. For the one-way push + NRR flow there is additionally third-party evidence: the Holodeck B2B counterparty test exchanges a signed message for a signed, digest-echoing receipt with an independent WSS4J-based implementation on every run. The caution still applies to everything that exchange does not cover.

What ASX still needs

ASX is a library; the conformance service tests an access point. The gap is deployment scaffolding, not protocol work.

The two ⬜ deployment blockers are the access point binary and the SUT Controller. Everything else is either done or a small increment.

✅ The protocol blocker found here is fixed. The oracle initially showed xmlsec1 rejecting ASX signatures (0/3 reference digests). Root cause: the canonicalizer stripped whitespace-only text nodes, which no W3C canonicalization does. Removed, plus the missing per-ds:Reference exc-c14n ds:Transform. Now 3/3.

Live counterparty — shipped

tests/interop_holodeck_counterparty.rs exchanges a signed AS4 push with Non-Repudiation of Receipt against Holodeck B2B 6.1.1 — an independent, WSS4J-based ebMS3/AS4 implementation — in a container:

cargo test --all-features --test interop_holodeck_counterparty -- --ignored --nocapture

Per run, the harness generates fresh RSA identities, converts party B's to PKCS#12 with the local openssl, assembles the three JKS keystores with the container's own keytool, installs a One-Way/Push NRR P-Mode by bind mount, and then:

1. asx signs and pushes a UserMessage;
2. Holodeck verifies the signature (WSS4J), resolves its P-Mode, delivers the payload, and answers with a signed receipt carrying NonRepudiationInformation;
3. asx verifies the receipt — signature, X509v3 BST signer resolution, and the NRR digest echo against the sent message.

This exchange is the strongest interop evidence in the suite: it exercises P-Mode resolution, WSS4J signature verification of asx output, and asx verification of WSS4J output — failure classes that neither self-tests nor a bare XMLDSig oracle can reach.

AS2 S/MIME oracle — shipped

tests/interop_openssl_smime_oracle.rs validates AS2 in both directions against the openssl smime CLI — no container required:

cargo test --all-features --test interop_openssl_smime_oracle -- --ignored

openssl verifies and decrypts asx-signed/encrypted messages; asx verifies and decrypts openssl-produced ones, which differ from asx output in ways that matter (MIME preambles, legacy application/x-pkcs7-* media types, media type carried only in the HTTP header).

Independent oracle — shipped

tests/interop_xmlsec_oracle.rs verifies ASX-generated WS-Security signatures with xmlsec1, the W3C XMLDSig reference implementation, in a container:

cargo test --all-features --test interop_xmlsec_oracle -- --ignored --nocapture

Two tests: a positive one, and a negative one that flips a payload byte and requires the oracle to reject it — without that, a harness bug that verified nothing would look like a pass.

Managed with testcontainers (blocking runner, WaitFor::Exit), so the container is removed even if an assertion panics.

This is the cheapest possible independent check (no gateway, no PModes, no keystores, ~2 s) and it targets the exact defect class this codebase keeps producing: canonicalization and digest disagreements that our own verifier cannot see because it shares the signer's assumptions. It found a real defect on its first run — a whitespace-stripping bug that made every ASX signature non-interoperable, invisible to 998 other tests.

Three harness transformations are applied, none changing any digest input: the cid: reference is retargeted to a local file with identical octets (xmlsec1 cannot resolve cid:), the WSS SwA Attachment-Content-Signature-Transform is stripped from that reference (xmlsec1 has no SwA support; for binary content the transform's digest input is the raw octets), and --id-attr is given per referenced element (ebms:Messaging, soap:Body) because wsu:Id is not a schema-declared ID. The first two edit ds:SignedInfo, so the tests assert the per-reference digest tally rather than the overall SignatureValue verdict — recomputing every reference digest under an independent C14N implementation is the interoperability question the oracle exists to answer.

Self-hosted end-to-end testing (do this first)

Since ITB is EUPL and Docker-distributed, and Domibus — the very access point DIGIT uses as Test Driver — is also freely available, the whole loop can run locally and in CI without registering for anything.

Option A — real counterparty, via testcontainers

Exchange real AS4 traffic with an independent implementation. This is the third-party interop evidence the project still lacks, and unlike the official run it executes on every commit.

On testcontainers-rs. The crate is testcontainers (0.27.x as of 2026-04), the official Rust fork of testcontainers.org. GenericImage plus a WaitFor strategy covers a gateway container:

let container = GenericImage::new("holodeck", "7")
    .with_exposed_port(8080.tcp())
    .with_wait_for(WaitFor::message_on_stdout("Server startup"))
    .start()
    .await?;
let port = container.get_host_port_ipv4(8080).await?;

WaitFor::Exit(ExitWaitStrategy::new()) also covers run-to-completion containers, which is how the xmlsec1 oracle uses it — no ports, just a bind-mount, a command, and the exit code plus captured stdout/stderr. That is why the oracle uses testcontainers rather than shelling out to docker run: the container is torn down when the handle drops, including when an assertion panics, and a missing daemon produces a typed error instead of a silently skipped test.

testcontainers is a dev-dependency only (features = ["blocking"]), and every test that uses it is #[ignore]d, so a normal cargo test never starts a container.

Candidate counterparties, in rough order of value:

Shape of the harness:

#[tokio::test]
#[ignore = "requires docker"]           // opt-in: `cargo test -- --ignored`
async fn as4_push_is_accepted_by_domibus() {
    let gateway = Domibus::default().start().await?;   // testcontainers
    // ASX sends a signed+encrypted push to the container's AS4 endpoint
    // and verifies the returned receipt with `verify_sync_response`.
}

Both directions matter: ASX → counterparty proves our send path produces conformant wire bytes; counterparty → ASX proves our receive path accepts what real implementations emit. The second direction is where the defects found in this codebase have clustered (attachment coverage, single-quoted href, ds:CanonicalizationMethod) — precisely because in-crate fixtures are written by the same hands as the parser.

Gate it behind #[ignore] plus a docker CI job so the default cargo test stays hermetic and fast.

Option B — self-hosted ITB with the real test suite

Stand up GITB from the ISAITB Docker images and run the actual eDelivery test cases locally. This gets you the same assertions the official run uses, without registration or a public URL.

The catch: the published messaging adapter, isaitb/asx-messaging-v4, drives Domibus via its Default WebService Plugin. To point the suite at ASX you implement the GITB messaging-service API (a small SOAP service: send, check acknowledgement, receive by message id) on top of ASX — which is most of the SUT Controller work anyway, so it is not wasted effort.

Recommended order

1. Option A with one counterparty, wired into CI. Cheapest, immediate, and catches the class of bug that internal fixtures structurally cannot.
2. Option B once the messaging adapter exists — the official assertions, run locally.
3. Official registration for the certificate, once 1 and 2 are green. By then the SUT Controller exists and the run is a formality.

References

• eDelivery conformance testing
• Using the conformance testing service
• AS4 User Guide v1.20 (PDF)
• Interoperability Test Bed · ISAITB on GitHub
• phase4
• Contact: EC-EDELIVERY-SUPPORT@ec.europa.eu