ICALEPCS2023 - List of Authors (Homs Puron, A.)

Paper	Title	Page
MO2AO01	Facing the Challenges of Experiment Control and Data Management at ESRF-EBS	66
	J.M. Meyer, W. De Nolf, S. Debionne, S. Fisher, A. Götz, M. Guijarro, P. Guillou, A. Homs Puron, V. Valls ESRF, Grenoble, France
	In 2020 the new ESRF-EBS (Extremely Brilliant Source) took-up operation. With the much higher photon flux, experiments are faster and produce more data. To meet the challenges, a complete revision of data acquisition, management and analysis tools was undertaken. The result is a suite of advanced software tools, deployed today on more than 30 beamlines. The main packages are BLISS for experiment control and data acquisition, LIMA2 for high-speed detector control, EWOKS for data reduction and analysis workflows, and Daiquiri the web GUI framework. BLISS is programmed in Python, to allow easy sequence programming for scientists and easy integration of scientific software. BLISS offers: Configuration of hardware and experimental set-ups, a generic scanning engine for step-based and continuous data acquisition, live data display, frameworks to handle 1D and 2D detectors, spectrometers, monochromators, diffractometers (HKL) and regulation loops. For detectors producing very high data rates, data reduction at the source is important. LIMA2 allows parallel data processing to add the necessary computing power (CPU and GPU) for online data reduction in a flexible way. The EWOKS workflow system can use online or offline data to automate data reduction or analysis. Workflows can run locally or on a compute cluster, using CPUs or GPUs. Results are saved or fed back to the control system for display or to adapt the next data acquisition.
	Slides MO2AO01 [2.766 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2AO01
About •	Received ※ 03 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 12 October 2023 — Issued ※ 29 October 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

THMBCMO31	LImA2: Edge Distributed Acquisition and Processing Framework for High Performance 2D Detectors	1269
	S. Debionne, L. Claustre, P. Fajardo, A. Götz, A. Homs Puron, J. Kieffer, R. Ponsard ESRF, Grenoble, France
	LImA* is a framework born at the ESRF for 2D Data Acquisition (DAQ), basic Online Data Analysis (ODA) and processing with high-throughput detectors. While in production for 15 years in several synchrotron facilities, the ever-increasing detector frame rates make more and more difficult performing DAQ & ODA tasks on a single computer. LImA2 is designed to scale horizontally, using multiple hosts for DAQ & ODA. This enables more advanced strategies for data feature extraction while keeping a low latency. LImA2 separates three functional blocks: detector control, image acquisition, and data processing. A control process configures the detector, while one or more receiver processes perform the DAQ and ODA, like the generation of fast feedback signals. The detectors currently supported in LImA2 are the PSI/Jungfrau, the ESRF/Smartpix and the Dectris/Eiger2. The former performs pixel assembly and intensity correction in GPU; the second exploits RoCE capabilities; and the latter features dual threshold, multi-band images. Raw data rates up to 8 GByte/s can be handled by a single computer, scalable if necessary. In addition to a classic processing, advanced pipelines are also implemented. A Serial-MX/pyFAI* pipeline extracts diffraction peaks in GPU in order to filter low quality data. NVIDIA GPUDirect is used by a third pipeline providing 2D processing with remarkable low latency. IBM Power9 optimizations like the NX GZIP compression and the PCI-e multi-host extension are exploited. * LIMA - https://accelconf.web.cern.ch/ICALEPCS2013/papers/frcoaab08.pdf Jungfraujoch - https://doi.org/10.1107/S1600577522010268 * pyFAI - https://doi.org/10.1107/S1600576715004306
	Slides THMBCMO31 [0.572 MB]
	Poster THMBCMO31 [14.959 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THMBCMO31
About •	Received ※ 06 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 11 December 2023 — Issued ※ 13 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

Facing the Challenges of Experiment Control and Data Management at ESRF-EBS

66

J.M. Meyer, W. De Nolf, S. Debionne, S. Fisher, A. Götz, M. Guijarro, P. Guillou, A. Homs Puron, V. Valls
ESRF, Grenoble, France

In 2020 the new ESRF-EBS (Extremely Brilliant Source) took-up operation. With the much higher photon flux, experiments are faster and produce more data. To meet the challenges, a complete revision of data acquisition, management and analysis tools was undertaken. The result is a suite of advanced software tools, deployed today on more than 30 beamlines. The main packages are BLISS for experiment control and data acquisition, LIMA2 for high-speed detector control, EWOKS for data reduction and analysis workflows, and Daiquiri the web GUI framework. BLISS is programmed in Python, to allow easy sequence programming for scientists and easy integration of scientific software. BLISS offers: Configuration of hardware and experimental set-ups, a generic scanning engine for step-based and continuous data acquisition, live data display, frameworks to handle 1D and 2D detectors, spectrometers, monochromators, diffractometers (HKL) and regulation loops. For detectors producing very high data rates, data reduction at the source is important. LIMA2 allows parallel data processing to add the necessary computing power (CPU and GPU) for online data reduction in a flexible way. The EWOKS workflow system can use online or offline data to automate data reduction or analysis. Workflows can run locally or on a compute cluster, using CPUs or GPUs. Results are saved or fed back to the control system for display or to adapt the next data acquisition.

Slides MO2AO01 [2.766 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2AO01

About •

Received ※ 03 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 12 October 2023 — Issued ※ 29 October 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

THMBCMO31

LImA2: Edge Distributed Acquisition and Processing Framework for High Performance 2D Detectors

1269

S. Debionne, L. Claustre, P. Fajardo, A. Götz, A. Homs Puron, J. Kieffer, R. Ponsard
ESRF, Grenoble, France

LImA* is a framework born at the ESRF for 2D Data Acquisition (DAQ), basic Online Data Analysis (ODA) and processing with high-throughput detectors. While in production for 15 years in several synchrotron facilities, the ever-increasing detector frame rates make more and more difficult performing DAQ & ODA tasks on a single computer**. LImA2 is designed to scale horizontally, using multiple hosts for DAQ & ODA. This enables more advanced strategies for data feature extraction while keeping a low latency. LImA2 separates three functional blocks: detector control, image acquisition, and data processing. A control process configures the detector, while one or more receiver processes perform the DAQ and ODA, like the generation of fast feedback signals. The detectors currently supported in LImA2 are the PSI/Jungfrau, the ESRF/Smartpix and the Dectris/Eiger2. The former performs pixel assembly and intensity correction in GPU; the second exploits RoCE capabilities; and the latter features dual threshold, multi-band images. Raw data rates up to 8 GByte/s can be handled by a single computer, scalable if necessary. In addition to a classic processing, advanced pipelines are also implemented. A Serial-MX/pyFAI*** pipeline extracts diffraction peaks in GPU in order to filter low quality data. NVIDIA GPUDirect is used by a third pipeline providing 2D processing with remarkable low latency. IBM Power9 optimizations like the NX GZIP compression and the PCI-e multi-host extension are exploited.
* LIMA - https://accelconf.web.cern.ch/ICALEPCS2013/papers/frcoaab08.pdf
** Jungfraujoch - https://doi.org/10.1107/S1600577522010268
*** pyFAI - https://doi.org/10.1107/S1600576715004306

Slides THMBCMO31 [0.572 MB]

Poster THMBCMO31 [14.959 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THMBCMO31

About •

Received ※ 06 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 11 December 2023 — Issued ※ 13 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)