From Eiger HDF5 to CrysAlis outputs

This tutorial explains the end-to-end conversion path used by ewoksscxrd: from Eiger/LImA HDF5 input to CrysAlis-compatible outputs (.esperanto, .ini, .set, .ccd, .par, .run) plus gallery/EDF products.

1. Choose a workflow

Use one of the bundled workflows:

• src/ewoksscxrd/workflows/demo_eiger2crysalis.json

• src/ewoksscxrd/workflows/demo_eiger2crysalis_lima2.json (dual-threshold with Lima2)

2. Inspect task inputs

ewoks show src/ewoksscxrd/workflows/demo_eiger2crysalis.json

This shows you the required workflow inputs (for example images, output, processed_output, distance, beam, and optional wavelength/polarization).

3. Provide the inputs from a script and execute the workflow

The bundled workflow is intentionally generic. You can execute it directly from Python with explicit runtime inputs:

import json
from pathlib import Path

from ewoks import execute_graph

workflow_path = Path("src/ewoksscxrd/workflows/demo_eiger2crysalis_lima2.json")
workflow = json.loads(workflow_path.read_text())

# User-editable values
images = ["/data/.../scan0001/eiger2_frame_0_00000.h5"]
lima2_output = "/data/.../scan0001/eiger2_frame"
esperanto_output = "/data/.../scan0001/frame_{index}.esperanto"
processed_output = "/data/.../scan0001/frame"
par_file = "/data/.../files/scan0001.par"
ini_file = "/data/.../files/CrysalisExpSettings.ini"
set_file = "/data/.../files/scan0001.set"
ccd_file = "/data/.../files/scan0001.ccd"

rotation = None
transpose = False
flip_ud = False
flip_lr = True
energy = None
wavelength = 0.2846
distance = 151.8
beam = [1052, 1102]
polarization = 0.99
kappa = 0
alpha = 50
theta = 0
phi = 0
omega = "-5.000000-index*-0.500000"
dummy = -1
offset = 1
dry_run = False
calc_mask = False
count = 144
omega_start = -35
omega_end = 35
domega = 1

# Currently the run file task expects a dict with all the processing parameters.
run_parameters = {
    "count": count,
    "omega_start": omega_start,
    "omega_end": omega_end,
    "domega": domega,
    "flip_ud": flip_ud,
    "flip_lr": flip_lr,
    "wavelength": wavelength,
    "distance": distance,
    "beam": beam,
    "polarization": polarization,
    "kappa": kappa,
    "alpha": alpha,
    "theta": theta,
    "phi": phi,
    "omega": omega,
    "rotation": rotation,
    "dummy": dummy,
    "offset": offset,
    "dry_run": dry_run,
    "calc_mask": calc_mask,
}

inputs = [
    {
        "name": "images",
        "value": images,
        "task_identifier": "ewoksscxrd.tasks.lima2thresholding.Lima2Thresholding",
    },
    {
        "name": "output",
        "value": lima2_output,
        "task_identifier": "ewoksscxrd.tasks.lima2thresholding.Lima2Thresholding",
    },
    {
        "name": "scale_factor",
        "value": 1,
        "task_identifier": "ewoksscxrd.tasks.lima2thresholding.Lima2Thresholding",
    },
    {
        "name": "output",
        "value": esperanto_output,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "processed_output",
        "value": processed_output,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "rotation",
        "value": rotation,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "transpose",
        "value": transpose,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "flip_ud",
        "value": flip_ud,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "flip_lr",
        "value": flip_lr,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "energy",
        "value": energy,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "wavelength",
        "value": wavelength,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "distance",
        "value": distance,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "beam",
        "value": beam,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "polarization",
        "value": polarization,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "kappa",
        "value": kappa,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "alpha",
        "value": alpha,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "theta",
        "value": theta,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "phi",
        "value": phi,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "omega",
        "value": omega,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "dummy",
        "value": dummy,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "offset",
        "value": offset,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "dry_run",
        "value": dry_run,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "calc_mask",
        "value": calc_mask,
        "task_identifier": "ewoksscxrd.tasks.eiger2crysalis.Eiger2Crysalis",
    },
    {
        "name": "par_file",
        "value": par_file,
        "task_identifier": "ewoksscxrd.tasks.createparfiles.CreateParFiles",
    },
    {
        "name": "ini_file",
        "value": ini_file,
        "task_identifier": "ewoksscxrd.tasks.createinifiles.CreateIniFiles",
    },
    {
        "name": "ccd_set_file",
        "value": set_file,
        "task_identifier": "ewoksscxrd.tasks.createsetccdfiles.CreateSetCcdFiles",
        "id": "3",
    },
    {
        "name": "ccd_set_file",
        "value": ccd_file,
        "task_identifier": "ewoksscxrd.tasks.createsetccdfiles.CreateSetCcdFiles",
        "id": "4",
    },
    {
        "name": "run_parameters",
        "value": run_parameters,
        "task_identifier": "ewoksscxrd.tasks.createrunfiles.CreateRunFiles",
    },
]

execute_graph(
    workflow,
    inputs=inputs,
    convert_destination="executed_demo_eiger2crysalis.json",
    merge_outputs=False,
)

4. Execute the workflow

python run_eiger2crysalis.py

Once the workflow is executed, it is converted with all the provided inputs into a json file. The name and path of the json file is defined in convert_destination (i.e. executed_demo_eiger2crysalis.json in this case).

5. Validate produced files

A typical processed output tree looks like:

PROCESSED_DATA/sample/sample_dataset/scan0001/
  CrysalisExpSettings.ini
  gallery/sample_dataset_0001_average.png
  sample_dataset_0001_1_1.esperanto
  ...
  sample_dataset_0001.ccd
  sample_dataset_0001.par
  sample_dataset_0001.run
  sample_dataset_0001.set

PROCESSED_DATA/sample/edf_PX/
  sample_dataset_0001.edf

The exact names depend on your scan and workflow inputs.

6. Data Portal visibility

If the DataPortal task runs with valid context/credentials, processed data can be published and browsed in the Data Portal with gallery preview and files.

Note

Data Portal publication depends on runtime environment and ICAT connectivity. This onlys works when executeing the workflow on ESRF premises.