ICALEPCS2023 - Table of Session: WE1BC (Hardware

Paper	Title	Page
WE1BCO01	VME2E: VME to Ethernet - Common Hardware Platform for legacy VME Module Upgrade	949
	J.P. Jamilkowski Brookhaven National Laboratory (BNL), Electron-Ion Collider, Upton, New York, USA Y. Tian BNL, Upton, New York, USA
	Funding: DOE Office of Science VME architecture was developed in late 1970s. It has proved to be a rugged control system hardware platform for the last four decades. Today the VME hardware platform is facing four challenges from 1) backplane communication speed bottleneck; 2) computing power limits from centralized computing infrastructure; 3) obsolescence and cost issues to support a real-time operating system; 4) obsolescence issues of the legacy VME hardware. The next generation hardware platform such as ATCA and microTCA requires fundamental changes in hardware and software. It also needs large investment. For many legacy system upgrades, this approach is not applicable. We will discuss an open-source hardware platform, VME2E (VME to Ethernet), which allows the one-to-one replacement of legacy VME module without disassembling of the existing VME system. The VME2E has the VME form factor. It can be installed the existing VME chassis, but without use the VME backplane to communicate with the front-end computer and therefore solves the first three challenges listed above. The VME2E will only take advantage of two good benefits from a VME system: stable power supply which VME2E module will get from the backplane, and the cooling environment. The VME2E will have the most advanced 14nm Xilinx FPGA SOM with GigE for parallel computing and high speed communication. It has a high pin count (HPC) FPGA mezzanine connector (FMC) to benefit the IO daughter boards supply of the FMC ecosystem. The VME2E is designed as a low cost, open-source common platform for legacy VME upgrade.
	Slides WE1BCO01 [1.141 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO01
About •	Received ※ 06 October 2023 — Revised ※ 09 October 2023 — Accepted ※ 19 November 2023 — Issued ※ 22 November 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO02	Data Management Infrastructure for European XFEL	952
	J. Malka, S. Aplin, D. Boukhelef, K. Filippakopoulos, L.G. Maia, T. Piszczek, Mr. Previtali, J. Szuba, K. Wrona EuXFEL, Schenefeld, Germany S. Dietrich, MA. Gasthuber, J. Hannappel, M. Karimi, Y. Kemp, R. Lueken, T. Mkrtchyan, K. Ohrenberg, F. Schlünzen, P. Suchowski, C. Voss DESY, Hamburg, Germany
	Effective data management is crucial to ensure research data is easily accessible and usable. We will present design and implementation of the European XFEL data management infrastructure supporting high level data management services. The system architecture comprises four layers of storage systems, each designed to address specific challenges. The first layer, referred to as online, is designed as a fast cache to accommodate extreme high rates (up to 15GB/s) of data generated during experiment at single scientific instrument. The second layer, called high-performance storage, provides necessary capabilities for data processing both during and after experiments. The layers are incorporated into a single infiniband fabric and connected through a 4km long 1Tb/s link. This allows fast data transfer from the European XFEL experiment hall to the DESY computing center. The third layer, mass-storage, extends the capacity of data storage system to allow mid-term data access for detailed analysis. Finally, the tape archive, provides data safety and long-term archive (5-10years). The high performance and mass storage systems are connected to computing clusters. This allows users to perform near-online and offline data analysis or alternatively export data outside of the European XFEL facility. The data management infrastructure at the European XFEL has the capacity to accept and process up to 2PB of data per day, which demonstrates the remarkable capabilities of all the sub-services involved in this process.
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO02
About •	Received ※ 06 October 2023 — Revised ※ 23 October 2023 — Accepted ※ 08 December 2023 — Issued ※ 12 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO03	Design of the HALF Control System	958
	G. Liu, L.G. Chen, C. Li, X.K. Sun, K. Xuan, D.D. Zhang USTC/NSRL, Hefei, Anhui, People’s Republic of China
	The Hefei Advanced Light Facility (HALF) is a 2.2-GeV 4th synchrotron radiation light source, which is scheduled to start construction in Hefei, China in 2023. The HALF contains an injector and a 480-m diffraction limited storage ring, and 10 beamlines for phase one. The HALF control system is EPICS based with integrated application and data platforms for the entire facility including accelerator and beamlines. The unified infrastructure and network architecture are designed to build the control system. The infrastructure provides resources for the EPICS development and operation through virtualization technology, and provides resources for the storage and process of experimental data through distributed storage and computing clusters. The network is divided into the control network and the dedicated high-speed data network by physical separation, the control network is subdivided into multiple subnets by VLAN technology. Through estimating the scale of the control system, the 10Gbps control backbone network and the data network that can be expanded to 100Gbps can fully meet the communication requirements of the control system. This paper reports the control system architecture design and the development work of some key technologies in details.
	Slides WE1BCO03 [2.739 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO03
About •	Received ※ 02 October 2023 — Revised ※ 09 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 26 October 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO04	The LCLS-II Experiment System Vacuum Controls Architecture	962
	M. Ghaly, T.A. Wallace SLAC, Menlo Park, California, USA
	Funding: This work is supported by Department of Energy contract DE-AC02-76SF00515. The LCLS-II Experiment System Vacuum Controls Architecture is a collection of vacuum system design templates, interlock logics, supported components (eg. gauges, pumps, valves), interface I/O, and associated software libraries which implement a baseline functionality and simulation. The architecture also includes a complement of engineering and deployment tools including cable test boxes or hardware simulators, as well as some automatic configuration tools. Vacuum controls at LCLS spans from rough vacuum in complex pumping manifolds, protection of highly-sensitive x-ray optics using fast shutters, maintenance of ultra-high vacuum in experimental sample delivery setups, and beyond. Often, the vacuum standards for LCLS systems exceeds what most vendors are experienced with. The system must maintain high-availability, while also remaining flexible and handling ongoing modifications. This paper will review the comprehensive architecture, the requirements of the LCLS systems, and introduce how to use it for new vacuum system designs. The architecture is meant to influence all phases of a vacuum system lifecycle, and ideally could become a shared project for installations beyond LCLS-II.
	Slides WE1BCO04 [3.154 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO04
About •	Received ※ 31 October 2023 — Revised ※ 20 November 2023 — Accepted ※ 08 December 2023 — Issued ※ 12 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO05	High Accuracy and Cost-Efficient Ethernet-Based Timing System for the IFMIF-DONES Facility
	J. Díaz, I.C. Casero, C. Megías UGR, Granada, Spain
	Funding: This work has been carried out within EUROfusion, funded by the European Union via the Euratom Research and Training Programme (Grant Agreement No 101052200). This article presents the timing system design of the IFMIF-DONES facility, which aims to develop materials that can withstand the harsh conditions of a fusion reactor while maintaining their structural integrity and functional properties. A key goal is to achieve high availability, which requires strong resiliency and redundancy measures throughout the plant design. The timing system design starts with a master clock composed of a stable master oscillator combined with GNSS receiver and clock disciplining equipment. They generate a local time scale and reference frequency with high stability. Three different Ethernet-based protocols are then combined, including NTP, IEEE-1588-2008 & 2019 High Accuracy profile (White-Rabbit) for time transfer purposes. NTP is used for generic computers and industrial devices that lack significant timing constraints, while IEEE-1588-2008 is used for industrial devices that require 1us accuracy or better. Both techniques can be implemented using off-the-shelf equipment and operate well over networks with moderate bandwidth utilization. The White-Rabbit protocol is used for devices that require highly accurate timing and can achieve sub-ns accuracy. It is typically designed for small, dedicated networks for timing only. This contribution describes the design of this timing system, highlighting how the best trade-off between cost and performance can be achieved through Ethernet technologies and how resiliency methods are implemented. Department of Computer engineering, automation and robotics, University of Granada, Spain.
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO07	The LCLS-II Precision Timing Control System	966
	T.K. Johnson, M.C. Browne, C.B. Pino SLAC, Menlo Park, California, USA
	The LCLS-II precision timing system is responsible for the synchronization of optical lasers with the LCLS-II XFEL. The system uses both RF and optical references for synchronization. In contrast to previous systems used at LCLS the optical lasers are shared resources, and must be managed during operations. The timing system consists of three primary functionalities: RF reference distribution, optical reference distribution, and a phase-locked loop (PLL). This PLL may use either the RF or the optical reference as a feedback source. The RF allows for phase comparisons over a relatively wide range, albeit with limited resolution, while the optical reference enables very fine phase comparison (down to attoseconds), but with limited operational range. These systems must be managed using high levels of automation. Much of this automation is done via high-level applications developed in EPICS. The beamline users are presented with relatively simple interfaces that streamline operation and abstract much of the system complexity away. The system provides both PyDM GUIs as well as python interfaces to enable time delay scanning in the LCLS-II DAQ.
	Slides WE1BCO07 [3.734 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO07
About •	Received ※ 06 November 2023 — Revised ※ 09 November 2023 — Accepted ※ 14 December 2023 — Issued ※ 20 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

VME2E: VME to Ethernet - Common Hardware Platform for legacy VME Module Upgrade

949

J.P. Jamilkowski
Brookhaven National Laboratory (BNL), Electron-Ion Collider, Upton, New York, USA
Y. Tian
BNL, Upton, New York, USA

Funding: DOE Office of Science
VME architecture was developed in late 1970s. It has proved to be a rugged control system hardware platform for the last four decades. Today the VME hardware platform is facing four challenges from 1) backplane communication speed bottleneck; 2) computing power limits from centralized computing infrastructure; 3) obsolescence and cost issues to support a real-time operating system; 4) obsolescence issues of the legacy VME hardware. The next generation hardware platform such as ATCA and microTCA requires fundamental changes in hardware and software. It also needs large investment. For many legacy system upgrades, this approach is not applicable. We will discuss an open-source hardware platform, VME2E (VME to Ethernet), which allows the one-to-one replacement of legacy VME module without disassembling of the existing VME system. The VME2E has the VME form factor. It can be installed the existing VME chassis, but without use the VME backplane to communicate with the front-end computer and therefore solves the first three challenges listed above. The VME2E will only take advantage of two good benefits from a VME system: stable power supply which VME2E module will get from the backplane, and the cooling environment. The VME2E will have the most advanced 14nm Xilinx FPGA SOM with GigE for parallel computing and high speed communication. It has a high pin count (HPC) FPGA mezzanine connector (FMC) to benefit the IO daughter boards supply of the FMC ecosystem. The VME2E is designed as a low cost, open-source common platform for legacy VME upgrade.

Slides WE1BCO01 [1.141 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO01

About •

Received ※ 06 October 2023 — Revised ※ 09 October 2023 — Accepted ※ 19 November 2023 — Issued ※ 22 November 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO02

Data Management Infrastructure for European XFEL

952

J. Malka, S. Aplin, D. Boukhelef, K. Filippakopoulos, L.G. Maia, T. Piszczek, Mr. Previtali, J. Szuba, K. Wrona
EuXFEL, Schenefeld, Germany
S. Dietrich, MA. Gasthuber, J. Hannappel, M. Karimi, Y. Kemp, R. Lueken, T. Mkrtchyan, K. Ohrenberg, F. Schlünzen, P. Suchowski, C. Voss
DESY, Hamburg, Germany

Effective data management is crucial to ensure research data is easily accessible and usable. We will present design and implementation of the European XFEL data management infrastructure supporting high level data management services. The system architecture comprises four layers of storage systems, each designed to address specific challenges. The first layer, referred to as online, is designed as a fast cache to accommodate extreme high rates (up to 15GB/s) of data generated during experiment at single scientific instrument. The second layer, called high-performance storage, provides necessary capabilities for data processing both during and after experiments. The layers are incorporated into a single infiniband fabric and connected through a 4km long 1Tb/s link. This allows fast data transfer from the European XFEL experiment hall to the DESY computing center. The third layer, mass-storage, extends the capacity of data storage system to allow mid-term data access for detailed analysis. Finally, the tape archive, provides data safety and long-term archive (5-10years). The high performance and mass storage systems are connected to computing clusters. This allows users to perform near-online and offline data analysis or alternatively export data outside of the European XFEL facility. The data management infrastructure at the European XFEL has the capacity to accept and process up to 2PB of data per day, which demonstrates the remarkable capabilities of all the sub-services involved in this process.

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO02

About •

Received ※ 06 October 2023 — Revised ※ 23 October 2023 — Accepted ※ 08 December 2023 — Issued ※ 12 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO03

Design of the HALF Control System

958

G. Liu, L.G. Chen, C. Li, X.K. Sun, K. Xuan, D.D. Zhang
USTC/NSRL, Hefei, Anhui, People’s Republic of China

The Hefei Advanced Light Facility (HALF) is a 2.2-GeV 4th synchrotron radiation light source, which is scheduled to start construction in Hefei, China in 2023. The HALF contains an injector and a 480-m diffraction limited storage ring, and 10 beamlines for phase one. The HALF control system is EPICS based with integrated application and data platforms for the entire facility including accelerator and beamlines. The unified infrastructure and network architecture are designed to build the control system. The infrastructure provides resources for the EPICS development and operation through virtualization technology, and provides resources for the storage and process of experimental data through distributed storage and computing clusters. The network is divided into the control network and the dedicated high-speed data network by physical separation, the control network is subdivided into multiple subnets by VLAN technology. Through estimating the scale of the control system, the 10Gbps control backbone network and the data network that can be expanded to 100Gbps can fully meet the communication requirements of the control system. This paper reports the control system architecture design and the development work of some key technologies in details.

Slides WE1BCO03 [2.739 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO03

About •

Received ※ 02 October 2023 — Revised ※ 09 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 26 October 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO04

The LCLS-II Experiment System Vacuum Controls Architecture

962

M. Ghaly, T.A. Wallace
SLAC, Menlo Park, California, USA

Funding: This work is supported by Department of Energy contract DE-AC02-76SF00515.
The LCLS-II Experiment System Vacuum Controls Architecture is a collection of vacuum system design templates, interlock logics, supported components (eg. gauges, pumps, valves), interface I/O, and associated software libraries which implement a baseline functionality and simulation. The architecture also includes a complement of engineering and deployment tools including cable test boxes or hardware simulators, as well as some automatic configuration tools. Vacuum controls at LCLS spans from rough vacuum in complex pumping manifolds, protection of highly-sensitive x-ray optics using fast shutters, maintenance of ultra-high vacuum in experimental sample delivery setups, and beyond. Often, the vacuum standards for LCLS systems exceeds what most vendors are experienced with. The system must maintain high-availability, while also remaining flexible and handling ongoing modifications. This paper will review the comprehensive architecture, the requirements of the LCLS systems, and introduce how to use it for new vacuum system designs. The architecture is meant to influence all phases of a vacuum system lifecycle, and ideally could become a shared project for installations beyond LCLS-II.

Slides WE1BCO04 [3.154 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO04

About •

Received ※ 31 October 2023 — Revised ※ 20 November 2023 — Accepted ※ 08 December 2023 — Issued ※ 12 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO05

High Accuracy and Cost-Efficient Ethernet-Based Timing System for the IFMIF-DONES Facility

J. Díaz, I.C. Casero, C. Megías
UGR, Granada, Spain

Funding: This work has been carried out within EUROfusion, funded by the European Union via the Euratom Research and Training Programme (Grant Agreement No 101052200).
This article presents the timing system design of the IFMIF-DONES facility, which aims to develop materials that can withstand the harsh conditions of a fusion reactor while maintaining their structural integrity and functional properties. A key goal is to achieve high availability, which requires strong resiliency and redundancy measures throughout the plant design. The timing system design starts with a master clock composed of a stable master oscillator combined with GNSS receiver and clock disciplining equipment. They generate a local time scale and reference frequency with high stability. Three different Ethernet-based protocols are then combined, including NTP, IEEE-1588-2008 & 2019 High Accuracy profile (White-Rabbit) for time transfer purposes. NTP is used for generic computers and industrial devices that lack significant timing constraints, while IEEE-1588-2008 is used for industrial devices that require 1us accuracy or better. Both techniques can be implemented using off-the-shelf equipment and operate well over networks with moderate bandwidth utilization. The White-Rabbit protocol is used for devices that require highly accurate timing and can achieve sub-ns accuracy. It is typically designed for small, dedicated networks for timing only. This contribution describes the design of this timing system, highlighting how the best trade-off between cost and performance can be achieved through Ethernet technologies and how resiliency methods are implemented.
Department of Computer engineering, automation and robotics, University of Granada, Spain.

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

WE1BCO07

The LCLS-II Precision Timing Control System

966

T.K. Johnson, M.C. Browne, C.B. Pino
SLAC, Menlo Park, California, USA

The LCLS-II precision timing system is responsible for the synchronization of optical lasers with the LCLS-II XFEL. The system uses both RF and optical references for synchronization. In contrast to previous systems used at LCLS the optical lasers are shared resources, and must be managed during operations. The timing system consists of three primary functionalities: RF reference distribution, optical reference distribution, and a phase-locked loop (PLL). This PLL may use either the RF or the optical reference as a feedback source. The RF allows for phase comparisons over a relatively wide range, albeit with limited resolution, while the optical reference enables very fine phase comparison (down to attoseconds), but with limited operational range. These systems must be managed using high levels of automation. Much of this automation is done via high-level applications developed in EPICS. The beamline users are presented with relatively simple interfaces that streamline operation and abstract much of the system complexity away. The system provides both PyDM GUIs as well as python interfaces to enable time delay scanning in the LCLS-II DAQ.

Slides WE1BCO07 [3.734 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE1BCO07

About •

Received ※ 06 November 2023 — Revised ※ 09 November 2023 — Accepted ※ 14 December 2023 — Issued ※ 20 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)