feat(bbmri-miabis): add MIABIS-on-FHIR + BBMRI.de mixed-node E2E test by a-tuerk · Pull Request #4 · samply/headlights

a-tuerk · 2026-03-20T21:20:51Z

Ready for review pending samply/focus#342 merge. Both focus instances use
samply/focus:localbuild for now. The image tag will be updated to
samply/focus:main (or a pinned release) once #342 is merged and published.

Summary

Adds bbmri-miabis/ with a compose stack running a two-node federation:
one MIABIS-on-FHIR focus node (CQL_FLAVOUR=miabis) and one BBMRI.de focus node
Two FHIR transaction bundles (test-bundle.json, test-bundle-bbmri.json)
and a tester service running 8 bash scenarios
All 8 scenarios pass against rustyspot 0.2.2 (verified 2026-03-23)

What is tested

Mixed-node scenarios (totals across both sites):

Empty AST → 4 (2 MIABIS + 2 BBMRI.de)
storage_temperature = temperatureRoom → 2
storage_temperature = temperatureLN → 3
sample_kind = blood-plasma → 2
sample_kind = whole-blood → 2

Diagnosis scenarios (BBMRI.de node unaffected by MIABIS workarounds):

diagnosis = C34 → 1 (MIABIS only)
diagnosis = C50 → 1 (BBMRI.de only)
diagnosis = C61 → 1 (BBMRI.de only)

Dependency

Requires samply/focus#342 (Flavour::Miabis, CQL_FLAVOUR env var).
Once that PR merges and a new focus image is published, the localbuild
tags will be replaced with the real tag.

Adds a new `bbmri-miabis/` integration test folder that validates the MIABIS CQL flavour (focus PR #341) alongside a standard BBMRI.de node. Stack: - focus-miabis (CQL_FLAVOUR=miabis, localbuild) + blaze-miabis + MIABIS test bundle - focus-bbmri (localbuild with rustyspot 0.2.x compat fix) + blaze-bbmri + BBMRI.de test bundle - spot (rustyspot:latest 0.2.2) fans queries out to both nodes via Beam Test script (test.sh): - Uses rustyspot 0.2.x two-step API: POST /beam (submit) + GET /beam/{id} (SSE) - 8 scenarios: 5 mixed-node patient counts + 3 diagnosis scenarios - All 8 pass (verified 2026-03-20) Note: both focus instances use localbuild because rustyspot 0.2.x changed the Beam task body format (raw Operation JSON instead of base64-wrapped Language struct); the compat fix falls back to parsing raw Operation JSON when base64 decode fails. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Replaces the old 5-scenario single-node description with the current 8-scenario mixed-node (MIABIS-on-FHIR + BBMRI.de) setup, including both test bundles, the tester service, and the rustyspot 0.2.x note. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

timakro

Thanks for the PR. I hadn't had the chance to test it yet but the approach looks good. I have one comment that needs to be addressed.

timakro · 2026-03-23T10:17:35Z

bbmri-miabis/test.sh

+    query_str=$(printf '%s' "${ast_obj}" | python3 -c "import sys,json; print(json.dumps(sys.stdin.read()))")
+    # Step 1: submit the query
+    curl -sf -m 10 \
+        -X POST "${SPOT_URL}/beam" \
+        -H "Content-Type: application/json" \
+        -d "{\"query\":${query_str},\"id\":\"${id}\"}" \


Spot expects a double base64-encoded request, which is admittedly a bit confusing. See this example request:

{"id":"e015565c-af71-4567-900e-4aee16de8a21","query":"eyJsYW5nIjoiYXN0IiwicGF5bG9hZCI6ImV5SmhjM1FpT25zaWIzQmxjbUZ1WkNJNklrOVNJaXdpWTJocGJHUnlaVzRpT2x0ZGZTd2lhV1FpT2lJNE56YzJZelZoWkMxa05Ua3lMVFExWVRBdFlqZ3hNUzA1WWpZME5UazBNVGsyWmpFaWZRPT0ifQ=="}

The query field is base64-encoded, and if you decode that you get another JSON object with a payload field which contains the base64-encoded (again) AST.

Encode your request like that and you can remove the special request handling you added to Focus.

Also better use jq and base64 instead of python.

Ah, the python thing was in my understanding to format the json object as an escaped json string (hence my comment below). I missed the double b64, hence, support your suggestion of jq (which can do b64 encoding internally, if I recall correctly. So no need to use the external base64 command)

TKussel

Thank you for branching out the integration tests of the original Focus PR to Headlights. While I can't comment on Headlight's conventions, I found mainly "taste" things regarding the docker compose setup.

bbmri-miabis/compose.local.yaml

bbmri-miabis/test.sh

bbmri-miabis/test-bundle-bbmri.json

bbmri-miabis/test-bundle.json

bbmri-miabis/README.md

Spot expects query=base64({"lang":"ast","payload":base64(AST)}). The previous approach (Python JSON-string escaping) sent a raw JSON object through the Spot→Beam path, which focus could not decode. Switch to jq + base64 (no python dependency). Also replace the grep+sed pipeline with sed -n for SSE line filtering, and use bracket notation for test-data-loader entrypoints. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The inner payload for Spot must decode to {"ast":{...},"id":"..."} as expected by focus parse_blaze_query_payload_ast, not bare AST JSON. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Use http://spot:8055 (Docker DNS) in test.sh instead of localhost - Remove network_mode: host from tester (not needed with Docker DNS) - Consolidate local actual declaration Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

a-tuerk · 2026-03-23T14:23:20Z

Thank you both for the detailed reviews! Here's a summary of what was addressed:

@timakro

Double base64 encoding: implemented using jq and base64 as suggested; also corrected the inner payload structure to wrap the AST in {"ast":{...},"id":"..."} as required by focus's parse_blaze_query_payload_ast
The rustyspot compatibility fix in focus has been reverted — it was masking a test encoding bug, not a real incompatibility

@TKussel

SPOT_URL changed to http://spot:8055; network_mode: host removed from the tester service accordingly
Entrypoint changed to bracket notation
--data-binary changed to -d
grep | sed replaced with sed -n 's/^data: //p'
Unnecessary comments removed
local actual declaration consolidated
Port exposures kept per timakro's convention note

All 8 scenarios pass with the updated encoding.

Andreas Tuerk and others added 2 commits March 20, 2026 16:06

timakro requested changes Mar 23, 2026

View reviewed changes

TKussel requested changes Mar 23, 2026

View reviewed changes

Andreas Tuerk and others added 3 commits March 23, 2026 13:51

fix(bbmri-miabis): wrap AST in ast::Ast envelope before base64 encoding

1afec2a

The inner payload for Spot must decode to {"ast":{...},"id":"..."} as expected by focus parse_blaze_query_payload_ast, not bare AST JSON. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(bbmri-miabis): add MIABIS-on-FHIR + BBMRI.de mixed-node E2E test#4

feat(bbmri-miabis): add MIABIS-on-FHIR + BBMRI.de mixed-node E2E test#4
a-tuerk wants to merge 5 commits intosamply:mainfrom
a-tuerk:main

a-tuerk commented Mar 20, 2026 •

edited

Loading

Uh oh!

timakro left a comment

Uh oh!

timakro Mar 23, 2026

Uh oh!

TKussel Mar 23, 2026

Uh oh!

TKussel left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

a-tuerk commented Mar 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

a-tuerk commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What is tested

Dependency

Uh oh!

timakro left a comment

Choose a reason for hiding this comment

Uh oh!

timakro Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

TKussel Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

TKussel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

a-tuerk commented Mar 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

a-tuerk commented Mar 20, 2026 •

edited

Loading