ICALEPCS2023 - List of Authors (Gotz, A.)

Paper	Title	Page
MO2AO01	Facing the Challenges of Experiment Control and Data Management at ESRF-EBS	66
	J.M. Meyer, W. De Nolf, S. Debionne, S. Fisher, A. Götz, M. Guijarro, P. Guillou, A. Homs Puron, V. Valls ESRF, Grenoble, France
	In 2020 the new ESRF-EBS (Extremely Brilliant Source) took-up operation. With the much higher photon flux, experiments are faster and produce more data. To meet the challenges, a complete revision of data acquisition, management and analysis tools was undertaken. The result is a suite of advanced software tools, deployed today on more than 30 beamlines. The main packages are BLISS for experiment control and data acquisition, LIMA2 for high-speed detector control, EWOKS for data reduction and analysis workflows, and Daiquiri the web GUI framework. BLISS is programmed in Python, to allow easy sequence programming for scientists and easy integration of scientific software. BLISS offers: Configuration of hardware and experimental set-ups, a generic scanning engine for step-based and continuous data acquisition, live data display, frameworks to handle 1D and 2D detectors, spectrometers, monochromators, diffractometers (HKL) and regulation loops. For detectors producing very high data rates, data reduction at the source is important. LIMA2 allows parallel data processing to add the necessary computing power (CPU and GPU) for online data reduction in a flexible way. The EWOKS workflow system can use online or offline data to automate data reduction or analysis. Workflows can run locally or on a compute cluster, using CPUs or GPUs. Results are saved or fed back to the control system for display or to adapt the next data acquisition.
	Slides MO2AO01 [2.766 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2AO01
About •	Received ※ 03 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 12 October 2023 — Issued ※ 29 October 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

TUMBCMO38	Towards the Zero Code Waste to Increase the Impact of Science	456
	P.P. Goryl, W. Soroka, Ł. Żytniak S2Innovation, Kraków, Poland A. Götz ESRF, Grenoble, France V. Hardion MAX IV Laboratory, Lund University, Lund, Sweden S. Hauf EuXFEL, Schenefeld, Germany K.S. White ORNL, Oak Ridge, Tennessee, USA
	Accelerators and other big science facilities rely heavily on internally developed technologies, including control system software. Much of it can and is shared between labs, like the Tango Controls and EPICS. Then, some of it finds broad application outside science, like the famous World Wide Web. However, there are still a lot of duplicating efforts in the labs, and a lot of software has the potential to be applied in other areas. Increasing collaboration and involving private companies can help avoid redundant work. It can decrease the overall costs of laboratory development and operation. Having private industry involved in technology development also increases the chances of new applications. This can positively impact society, which means effective spending of public funds. The talk will be based on the results of a survey looking at how much scientific institutes and companies focus on collaboration and dissemination in the field of software technologies. It will also include remarks based on the authors’ experiences in building an innovative ecosystem.
	Slides TUMBCMO38 [0.294 MB]
	Poster TUMBCMO38 [1.016 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUMBCMO38
About •	Received ※ 06 October 2023 — Revised ※ 12 October 2023 — Accepted ※ 28 November 2023 — Issued ※ 06 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

TUSDSC01	BLISS: ESRF All-In-One, Python-based Experiment Control System
	M. Guijarro, G. Berruyer, L. Claustre, W. De Nolf, L. Felix, A. Götz, P. Guillou, C. Guilloud, J.M. Meyer, E. Papillon, S. Petitdemange, L. Pithan, V. Valls ESRF, Grenoble, France
	BLISS is an all-in-one experiment control system designed to address the complex challenges of synchronized data acquisition and management, for synchrotrons and other labs. Written in Python, BLISS provides a comprehensive solution for hardware control (BLISS native, Tango and EPICS control systems are supported), experiment control sequences, data acquisition, and data visualization. Its modular design makes it easy to configure and customize for different setups. One of the key features of BLISS is its decoupling of data acquisition from data storage, which is achieved through the use of Redis as a temporary buffer. Thanks to a companion Python library called "blissdata" clients can access data without perturbing the acquisition, alleviating real-time constraints for display, saving or to perform online data analysis. On top of blissdata, BLISS is shipped with Flint, a powerful data visualization tool to display and interact with experimental data in real-time, providing an efficient solution for quality control and immediate feedback. BLISS comes with handy web applications, including a configuration tool and a web terminal ; users can easily configure the system and interact with it. It is designed to interface with Daiquiri, for more advanced web applications. Additionally, BLISS includes a full simulation environment, which can be used to learn about the system and to try it out. In summary, BLISS is a complete solution for laboratory data acquisition and management that provides a user-friendly interface and supports online data analysis and data display.
	Poster TUSDSC01 [2.538 MB]
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE3BCO07	Extending the ICAT Metadata Catalogue to New Scientific Use Cases	1033
	A. Götz, M. Bodin, A. De Maria Antolinos, M. Gaonach ESRF, Grenoble, France M. AlMohammad, S.A. Matalgah SESAME, Allan, Jordan P. Austin, V. Bozhinov, L.E. Davies, A. Gonzalez Beltran, K.S. Phipps STFC/RAL/SCD, Didcot, United Kingdom R. Cabezas Quirós ALBA-CELLS, Cerdanyola del Vallès, Spain R. Krahl HZB, Berlin, Germany A. Pinto LNLS, Campinas, Brazil K. Syder DLS, Oxfordshire, United Kingdom
	The ICAT metadata catalogue is a flexible solution for managing scientific metadata and data from a wide variety of domains following the FAIR data principles. This paper will present an update of recent developments of the ICAT metadata catalogue and the latest status of the ICAT collaboration. ICAT was originally developed by UK Science and Technology Facilities Council (STFC) to manage the scientific data of ISIS Neutron and Muon Source and Diamond Light Source. They have since been joined by a number of other institutes including ESRF, HZB, SESAME, and ALBA who together now form the ICAT Collaboration [1]. ICAT has been used to manage petabytes of scientific data for ISIS, DLS, ESRF, HZB, and in the future SESAME and ALBA and make these data FAIR. The latest version of the ICAT core as well as the new user interfaces, DataGateway and DataHub, and extensions to ICAT for implementing free text searching, a common search interface across Photon and Neutron catalogues, a protocol-based interface that allows making the metadata available for findability, electronic logbooks, sample tracking, and web-based data and domain specific viewers developed by the community will be presented. Finally recent developments to use ICAT to develop applications for processed data with rich metadata in the fields of small angle scattering, macromolecular crystallography and cryo-electron microscopy will be described. [1] https://icatproject.org
	Slides WE3BCO07 [7.888 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE3BCO07
About •	Received ※ 05 October 2023 — Revised ※ 23 October 2023 — Accepted ※ 14 December 2023 — Issued ※ 14 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

TH1BCO03	The Tango Controls Collaboration Status in 2023	1100
	T. Juerges SKAO, Macclesfield, United Kingdom G. Abeillé SOLEIL, Gif-sur-Yvette, France R.J. Auger-Williams OSL, St Ives, Cambridgeshire, United Kingdom B. Bertrand, V. Hardion, A.F. Joubert MAX IV Laboratory, Lund University, Lund, Sweden R. Bourtembourg, A. Götz, D. Lacoste, N. Leclercq ESRF, Grenoble, France T. Braun byte physics, Annaburg, Germany G. Cuní, C. Pascual-Izarra, S. Rubio-Manrique ALBA-CELLS, Cerdanyola del Vallès, Spain Yu. Matveev DESY, Hamburg, Germany M. Nabywaniec, T.R. Noga, Ł. Żytniak S2Innovation, Kraków, Poland L. Pivetta Elettra-Sincrotrone Trieste S.C.p.A., Basovizza, Italy
	Since 2021 the Tango Controls collaboration has improved and optimised its efforts in many areas. Not only have Special Interest Group meetings (SIGs) been introduced to speed up the adoption of new technologies or improvements, the kernel has switched to a fixed six-month release cycle for quicker adoption of stable kernel versions by the community. CI/CD provides now early feedback on test failures and compatibility issues. Major code refactoring allowed for a much more efficient use of developer resources. Relevant bug fixes, improvements and new features are now adopted at a much higher rate than ever before. The community participation has also noticeably improved. The kernel switched to C++14 and the logging system is undergoing a major refactoring. Among many new features and tools is jupyTango, Jupyter Notebooks on Tango Controls steroids. PyTango is now easy to install via binary wheels, old Python versions are no longer supported, the build-system is switching to CMake, and releases are now made much closer to stable cppTango releases.
	Slides TH1BCO03 [1.357 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TH1BCO03
About •	Received ※ 05 October 2023 — Revised ※ 24 October 2023 — Accepted ※ 21 November 2023 — Issued ※ 13 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

THMBCMO31	LImA2: Edge Distributed Acquisition and Processing Framework for High Performance 2D Detectors	1269
	S. Debionne, L. Claustre, P. Fajardo, A. Götz, A. Homs Puron, J. Kieffer, R. Ponsard ESRF, Grenoble, France
	LImA* is a framework born at the ESRF for 2D Data Acquisition (DAQ), basic Online Data Analysis (ODA) and processing with high-throughput detectors. While in production for 15 years in several synchrotron facilities, the ever-increasing detector frame rates make more and more difficult performing DAQ & ODA tasks on a single computer. LImA2 is designed to scale horizontally, using multiple hosts for DAQ & ODA. This enables more advanced strategies for data feature extraction while keeping a low latency. LImA2 separates three functional blocks: detector control, image acquisition, and data processing. A control process configures the detector, while one or more receiver processes perform the DAQ and ODA, like the generation of fast feedback signals. The detectors currently supported in LImA2 are the PSI/Jungfrau, the ESRF/Smartpix and the Dectris/Eiger2. The former performs pixel assembly and intensity correction in GPU; the second exploits RoCE capabilities; and the latter features dual threshold, multi-band images. Raw data rates up to 8 GByte/s can be handled by a single computer, scalable if necessary. In addition to a classic processing, advanced pipelines are also implemented. A Serial-MX/pyFAI* pipeline extracts diffraction peaks in GPU in order to filter low quality data. NVIDIA GPUDirect is used by a third pipeline providing 2D processing with remarkable low latency. IBM Power9 optimizations like the NX GZIP compression and the PCI-e multi-host extension are exploited. * LIMA - https://accelconf.web.cern.ch/ICALEPCS2013/papers/frcoaab08.pdf Jungfraujoch - https://doi.org/10.1107/S1600577522010268 * pyFAI - https://doi.org/10.1107/S1600576715004306
	Slides THMBCMO31 [0.572 MB]
	Poster THMBCMO31 [14.959 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THMBCMO31
About •	Received ※ 06 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 11 December 2023 — Issued ※ 13 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

Facing the Challenges of Experiment Control and Data Management at ESRF-EBS

66

J.M. Meyer, W. De Nolf, S. Debionne, S. Fisher, A. Götz, M. Guijarro, P. Guillou, A. Homs Puron, V. Valls
ESRF, Grenoble, France

In 2020 the new ESRF-EBS (Extremely Brilliant Source) took-up operation. With the much higher photon flux, experiments are faster and produce more data. To meet the challenges, a complete revision of data acquisition, management and analysis tools was undertaken. The result is a suite of advanced software tools, deployed today on more than 30 beamlines. The main packages are BLISS for experiment control and data acquisition, LIMA2 for high-speed detector control, EWOKS for data reduction and analysis workflows, and Daiquiri the web GUI framework. BLISS is programmed in Python, to allow easy sequence programming for scientists and easy integration of scientific software. BLISS offers: Configuration of hardware and experimental set-ups, a generic scanning engine for step-based and continuous data acquisition, live data display, frameworks to handle 1D and 2D detectors, spectrometers, monochromators, diffractometers (HKL) and regulation loops. For detectors producing very high data rates, data reduction at the source is important. LIMA2 allows parallel data processing to add the necessary computing power (CPU and GPU) for online data reduction in a flexible way. The EWOKS workflow system can use online or offline data to automate data reduction or analysis. Workflows can run locally or on a compute cluster, using CPUs or GPUs. Results are saved or fed back to the control system for display or to adapt the next data acquisition.

Slides MO2AO01 [2.766 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2AO01

About •

Received ※ 03 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 12 October 2023 — Issued ※ 29 October 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

TUMBCMO38

Towards the Zero Code Waste to Increase the Impact of Science

456

P.P. Goryl, W. Soroka, Ł. Żytniak
S2Innovation, Kraków, Poland
A. Götz
ESRF, Grenoble, France
V. Hardion
MAX IV Laboratory, Lund University, Lund, Sweden
S. Hauf
EuXFEL, Schenefeld, Germany
K.S. White
ORNL, Oak Ridge, Tennessee, USA

Accelerators and other big science facilities rely heavily on internally developed technologies, including control system software. Much of it can and is shared between labs, like the Tango Controls and EPICS. Then, some of it finds broad application outside science, like the famous World Wide Web. However, there are still a lot of duplicating efforts in the labs, and a lot of software has the potential to be applied in other areas. Increasing collaboration and involving private companies can help avoid redundant work. It can decrease the overall costs of laboratory development and operation. Having private industry involved in technology development also increases the chances of new applications. This can positively impact society, which means effective spending of public funds. The talk will be based on the results of a survey looking at how much scientific institutes and companies focus on collaboration and dissemination in the field of software technologies. It will also include remarks based on the authors’ experiences in building an innovative ecosystem.

Slides TUMBCMO38 [0.294 MB]

Poster TUMBCMO38 [1.016 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUMBCMO38

About •

Received ※ 06 October 2023 — Revised ※ 12 October 2023 — Accepted ※ 28 November 2023 — Issued ※ 06 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

TUSDSC01

BLISS: ESRF All-In-One, Python-based Experiment Control System

M. Guijarro, G. Berruyer, L. Claustre, W. De Nolf, L. Felix, A. Götz, P. Guillou, C. Guilloud, J.M. Meyer, E. Papillon, S. Petitdemange, L. Pithan, V. Valls
ESRF, Grenoble, France

BLISS is an all-in-one experiment control system designed to address the complex challenges of synchronized data acquisition and management, for synchrotrons and other labs. Written in Python, BLISS provides a comprehensive solution for hardware control (BLISS native, Tango and EPICS control systems are supported), experiment control sequences, data acquisition, and data visualization. Its modular design makes it easy to configure and customize for different setups. One of the key features of BLISS is its decoupling of data acquisition from data storage, which is achieved through the use of Redis as a temporary buffer. Thanks to a companion Python library called "blissdata" clients can access data without perturbing the acquisition, alleviating real-time constraints for display, saving or to perform online data analysis. On top of blissdata, BLISS is shipped with Flint, a powerful data visualization tool to display and interact with experimental data in real-time, providing an efficient solution for quality control and immediate feedback. BLISS comes with handy web applications, including a configuration tool and a web terminal ; users can easily configure the system and interact with it. It is designed to interface with Daiquiri, for more advanced web applications. Additionally, BLISS includes a full simulation environment, which can be used to learn about the system and to try it out. In summary, BLISS is a complete solution for laboratory data acquisition and management that provides a user-friendly interface and supports online data analysis and data display.

Poster TUSDSC01 [2.538 MB]

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE3BCO07

Extending the ICAT Metadata Catalogue to New Scientific Use Cases

1033

A. Götz, M. Bodin, A. De Maria Antolinos, M. Gaonach
ESRF, Grenoble, France
M. AlMohammad, S.A. Matalgah
SESAME, Allan, Jordan
P. Austin, V. Bozhinov, L.E. Davies, A. Gonzalez Beltran, K.S. Phipps
STFC/RAL/SCD, Didcot, United Kingdom
R. Cabezas Quirós
ALBA-CELLS, Cerdanyola del Vallès, Spain
R. Krahl
HZB, Berlin, Germany
A. Pinto
LNLS, Campinas, Brazil
K. Syder
DLS, Oxfordshire, United Kingdom

The ICAT metadata catalogue is a flexible solution for managing scientific metadata and data from a wide variety of domains following the FAIR data principles. This paper will present an update of recent developments of the ICAT metadata catalogue and the latest status of the ICAT collaboration. ICAT was originally developed by UK Science and Technology Facilities Council (STFC) to manage the scientific data of ISIS Neutron and Muon Source and Diamond Light Source. They have since been joined by a number of other institutes including ESRF, HZB, SESAME, and ALBA who together now form the ICAT Collaboration [1]. ICAT has been used to manage petabytes of scientific data for ISIS, DLS, ESRF, HZB, and in the future SESAME and ALBA and make these data FAIR. The latest version of the ICAT core as well as the new user interfaces, DataGateway and DataHub, and extensions to ICAT for implementing free text searching, a common search interface across Photon and Neutron catalogues, a protocol-based interface that allows making the metadata available for findability, electronic logbooks, sample tracking, and web-based data and domain specific viewers developed by the community will be presented. Finally recent developments to use ICAT to develop applications for processed data with rich metadata in the fields of small angle scattering, macromolecular crystallography and cryo-electron microscopy will be described. [1] https://icatproject.org

Slides WE3BCO07 [7.888 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE3BCO07

About •

Received ※ 05 October 2023 — Revised ※ 23 October 2023 — Accepted ※ 14 December 2023 — Issued ※ 14 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

TH1BCO03

The Tango Controls Collaboration Status in 2023

1100

T. Juerges
SKAO, Macclesfield, United Kingdom
G. Abeillé
SOLEIL, Gif-sur-Yvette, France
R.J. Auger-Williams
OSL, St Ives, Cambridgeshire, United Kingdom
B. Bertrand, V. Hardion, A.F. Joubert
MAX IV Laboratory, Lund University, Lund, Sweden
R. Bourtembourg, A. Götz, D. Lacoste, N. Leclercq
ESRF, Grenoble, France
T. Braun
byte physics, Annaburg, Germany
G. Cuní, C. Pascual-Izarra, S. Rubio-Manrique
ALBA-CELLS, Cerdanyola del Vallès, Spain
Yu. Matveev
DESY, Hamburg, Germany
M. Nabywaniec, T.R. Noga, Ł. Żytniak
S2Innovation, Kraków, Poland
L. Pivetta
Elettra-Sincrotrone Trieste S.C.p.A., Basovizza, Italy

Since 2021 the Tango Controls collaboration has improved and optimised its efforts in many areas. Not only have Special Interest Group meetings (SIGs) been introduced to speed up the adoption of new technologies or improvements, the kernel has switched to a fixed six-month release cycle for quicker adoption of stable kernel versions by the community. CI/CD provides now early feedback on test failures and compatibility issues. Major code refactoring allowed for a much more efficient use of developer resources. Relevant bug fixes, improvements and new features are now adopted at a much higher rate than ever before. The community participation has also noticeably improved. The kernel switched to C++14 and the logging system is undergoing a major refactoring. Among many new features and tools is jupyTango, Jupyter Notebooks on Tango Controls steroids. PyTango is now easy to install via binary wheels, old Python versions are no longer supported, the build-system is switching to CMake, and releases are now made much closer to stable cppTango releases.

Slides TH1BCO03 [1.357 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TH1BCO03

About •

Received ※ 05 October 2023 — Revised ※ 24 October 2023 — Accepted ※ 21 November 2023 — Issued ※ 13 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

THMBCMO31

LImA2: Edge Distributed Acquisition and Processing Framework for High Performance 2D Detectors

1269

S. Debionne, L. Claustre, P. Fajardo, A. Götz, A. Homs Puron, J. Kieffer, R. Ponsard
ESRF, Grenoble, France

LImA* is a framework born at the ESRF for 2D Data Acquisition (DAQ), basic Online Data Analysis (ODA) and processing with high-throughput detectors. While in production for 15 years in several synchrotron facilities, the ever-increasing detector frame rates make more and more difficult performing DAQ & ODA tasks on a single computer**. LImA2 is designed to scale horizontally, using multiple hosts for DAQ & ODA. This enables more advanced strategies for data feature extraction while keeping a low latency. LImA2 separates three functional blocks: detector control, image acquisition, and data processing. A control process configures the detector, while one or more receiver processes perform the DAQ and ODA, like the generation of fast feedback signals. The detectors currently supported in LImA2 are the PSI/Jungfrau, the ESRF/Smartpix and the Dectris/Eiger2. The former performs pixel assembly and intensity correction in GPU; the second exploits RoCE capabilities; and the latter features dual threshold, multi-band images. Raw data rates up to 8 GByte/s can be handled by a single computer, scalable if necessary. In addition to a classic processing, advanced pipelines are also implemented. A Serial-MX/pyFAI*** pipeline extracts diffraction peaks in GPU in order to filter low quality data. NVIDIA GPUDirect is used by a third pipeline providing 2D processing with remarkable low latency. IBM Power9 optimizations like the NX GZIP compression and the PCI-e multi-host extension are exploited.
* LIMA - https://accelconf.web.cern.ch/ICALEPCS2013/papers/frcoaab08.pdf
** Jungfraujoch - https://doi.org/10.1107/S1600577522010268
*** pyFAI - https://doi.org/10.1107/S1600576715004306

Slides THMBCMO31 [0.572 MB]

Poster THMBCMO31 [14.959 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THMBCMO31

About •

Received ※ 06 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 11 December 2023 — Issued ※ 13 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)