ICALEPCS2023 - Table of Session: MO4A (FPGA)

Paper	Title	Page
MO4AO01	Xilinx Zync Ultrascale⁺ MPSoC Used as Embedded IOC for a Beam Position Monitor (BPM) System	210
	G.M. Marinkovic, D. Anicic, R. Ditter, B. Keil, J. Purtschert, M. Roggli PSI, Villigen PSI, Switzerland
	At PSI we are combining the hardware, firmware, operating system, control system, embedded event system, operation and supervision in a Beam Position Monitor (BPM) system for 24/7 accelerator operation, using a Multi-Processing-System-on-Chip (MPSoC) of type Xilinx Zynq UltraScale+. We presently use MPSoCs for our latest generic BPM electronics platform called "DBPM3" in the Athos soft X-ray branch, as well as for new BPMs and general controls hardware and devices for SLS 2.0, a major upgrade of the Swiss Light Source. We are also in the process of upgrading our previous "MBU" (modular BPM Unit) platform for the SwissFEL linac and hard X-ray "Aramis" from external VMEbus based IOCs to integrated add-on cards with MPSoC IOCs. On all these MPSoCs, we are integrating an EPICS IOC, event receiver, measurement and feedback data real-time processing on a single chip. In this contribution, we describe our experience with the tight integration and daily operation of the various firmware and software components and features on the MPSoC, using the BPM system also to discuss general aspects relevant for other systems and components discussed in other PSI contributions on this conference.
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO01
About •	Received ※ 06 October 2023 — Revised ※ 09 October 2023 — Accepted ※ 23 November 2023 — Issued ※ 11 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO02	HydRA: A System-on-Chip to Run Software in Radiation-Exposed Areas	217
	T. Gingold, G. Daniluk, J. Serranopresenter, T. Włostowski CERN, Meyrin, Switzerland M. Rizzi PSI, Villigen PSI, Switzerland
	In the context of the High-Luminosity LHC project at CERN, a platform has been developed to support groups needing to host electronics in radiation-exposed areas. This platform, called DI/OT, is based on a modular kit consisting of a System Board, Peripheral Boards and a radiation-tolerant power converter, all housed in a standard 3U crate. Groups customise their systems by designing Peripheral Boards and developing custom gateware and software for the System Board, featuring an IGLOO2 flash-based FPGA. It is compulsory for gateware designs to be radiation-tested in dedicated facilities before deployment. This process can be cumbersome and affects iteration time because access to radiation testing facilities is a scarce commodity. To make customisation more agile, we have developed a radiation-tolerant System-on-Chip (SoC), so that a single gateware design, extensively validated, can serve as a basis for different applications by just changing the software running in the processing unit of the SoC. HydRA (Hydra-like Resilient Architecture) features a triplicated RISC-V processor for safely running software in a radiation environment. This paper describes the overall context for the project, and then moves on to provide detailed explanations of all the design decisions for making HydRA radiation-tolerant, including the protection of programme and data memories. Test harnesses are also described, along with a summary of the test results so far. It concludes with ideas for further development and plans for deployment in the LHC. https://ohwr.org/project/hydra/wikis/home https://ohwr.org/project/diot/wikis/home
	Slides MO4AO02 [11.131 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO02
About •	Received ※ 06 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 27 October 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO03	The DESY Open Source FPGA Framework	222
	Ł. Butkowski, A. Bellandi, M. Büchler, B. Dursun, Ç. Gümüş, N. Omidsajedi, K. Schulz DESY, Hamburg, Germany
	Modern FPGA firmware development involves integrating various intellectual properties (IP), modules written in hardware description languages (HDL), high-level synthesis (HLS), and software/hardware CPUs with embedded Linux or bare-metal applications. This process may involve multiple tools from the same or different vendors, making it complex and challenging. Additionally, scientific institutions such as DESY require long-term maintenance and reproducibility for designs that may involve multiple developers, further complicating the process. To address these challenges, we have developed an open-source FPGA firmware framework (FWK) at DESY that streamlines development, facilitates collaboration, and reduces complexity. The FWK achieves this by providing an abstraction layer, a defined structure, and guidelines to create big FPGA designs with ease. FWK also generates documentation and address maps necessary for high-level software frameworks like ChimeraTK. This paper presents an overview and the idea of the FWK.
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO03
About •	Received ※ 05 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 13 October 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO05	Development of a Timing and Data Link for EIC Common Hardware Platform	228
	P. Bachek, T. Hayes, J. Mead, K. Mernick, G. Narayan, F. Severino BNL, Upton, New York, USA
	Funding: Contract Number DE-AC02-98CH10886 with the auspices of the US Department of Energy Modern timing distribution systems benefit from high configurability and the bidirectional transfer of timing data. The Electron Ion Collider (EIC) Common Hardware Platform (CHP) will integrate the functions of the existing RHIC Real Time Data Link (RTDL), Event Link, and Beam Sync Link, along with the Low-Level RF (LLRF) system Update Link (UL), into a common high speed serial link. One EIC CHP carrier board sup-ports up to eight external 8 Gbps high speed links via SFP+ modules, as well as up to six 8 Gbps high speed links to each of two daughterboards. A daughterboard will be designed for the purpose of timing data link distribution for use with the CHP. This daughterboard will have two high speed digital crosspoint switches and a Xilinx Artix Ultrascale⁺ FPGA onboard with GTY transceivers. One of these will be dedicated for a high-speed control and data link directly between the onboard FPGA and the carrier FPGA. The remaining GTY transceivers will be routed through the crosspoint switches. The daughterboard will support sixteen external SFP+ ports for timing distribution infrastructure with some ports dedicated for transmit only link fanout. The timing data link will support bidirectional data transfer including sending data or events from a downstream device back upstream. This flexibility will be achieved by routing the SFP+ ports through the crosspoint switches which allows the timing link datapaths to be forwarded directly through the daughterboard to the carrier and into the FPGA on the daughterboard in many different configurations.
	Slides MO4AO05 [1.236 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO05
About •	Received ※ 05 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 23 November 2023 — Issued ※ 07 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO06	Overview and Outlook of FPGA Based Hardware Solutions for Data Synchronization, Acquisition and Processing at the Euxfel	233
	B.J. Fernandes, F. Babies, T. Freyermuth, P. Gessler, I.S. Soekmen, H. Sotoudi Namin EuXFEL, Schenefeld, Germany
	The European X-Ray Free Electron Laser facility (EuXFEL) provides ultra short coherent X-Ray flashes, spaced by 220 nanoseconds and with a duration of less than 100 femtoseconds, in bursts of up to 2700 pulses every 100ms to several instruments. The facility has been using standardized Field-Programmable Gate Array (FPGA) based hardware platforms since the beginning of user operation in 2017. These are used for timing distribution, data processing from large 2D detectors, high speed digitizers for acquisition and processing of pulse signals, monitoring beam characteristics, and low latency communication protocol for pulse data vetoing and Machine Protection System (MPS). Our experience grows in tandem with user requests for more specific and challenging case studies, leading to tailor made hardware algorithms and setups. In some cases, these can be fulfilled with the integration of new hardware, where collaboration with companies for new and/or updated platforms is a key factor, or taking advantage of unused features in current setups. In this overview, we present the FPGA hardware based solutions used to fulfill EuXFEL’s requirements. We also present our efforts in integrating new solutions and possible development directions, including Machine Learning (ML) research, with the aim of bringing more accurate results and configurable setups to user experiments and facilitate communications with other platforms used in the facility, namely Programmable Logic Controllers (PLC).
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO06
About •	Received ※ 06 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 23 October 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO07	Status of the MicroTCA Based Beam Instrumentation DAQ Systems at GSI and FAIR	239
THPDP015	use link to see paper's listing under its alternate paper code
	T. Hoffmann, H. Bräuning, R.N. Geißler, T. Milosic GSI, Darmstadt, Germany
	While the first FAIR accelerator buildings are soon to be completed, MicroTCA-based data acquisition sys-tems for FAIR beam instrumentation are ready for use. By using commercial off-the-shelf components as well as open hardware with in-house expertise in FPGA programming, there are now DAQ solutions for almost all major detector systems in MicroTCA in operation at the existing GSI accelerators. Applications span a wide range of detector systems and hardware, often taking advantage of the high channel density and data trans-mission bandwidth available with MicroTCA. All DAQ systems are synchronised and triggered using a com-prehensive White Rabbit based timing system. This allows correlation of the data from the distributed acquisition systems on a nanosecond scale. In this paper, we present some examples of our DAQ implemented in MicroTCA covering the range of beam current, tune, position and profile measurements. While the latter uses GigE cameras in combination with scintillating screens, the other applications are based on ADCs with different sampling frequencies between 125 MSa/s up to 2.5 GSa/s or latching scalers with up to 10 MHz latching frequency.
	Slides MO4AO07 [3.497 MB]
	Poster MO4AO07 [3.790 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO07
About •	Received ※ 29 September 2023 — Revised ※ 07 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 07 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

Xilinx Zync Ultrascale⁺ MPSoC Used as Embedded IOC for a Beam Position Monitor (BPM) System

210

G.M. Marinkovic, D. Anicic, R. Ditter, B. Keil, J. Purtschert, M. Roggli
PSI, Villigen PSI, Switzerland

At PSI we are combining the hardware, firmware, operating system, control system, embedded event system, operation and supervision in a Beam Position Monitor (BPM) system for 24/7 accelerator operation, using a Multi-Processing-System-on-Chip (MPSoC) of type Xilinx Zynq UltraScale+. We presently use MPSoCs for our latest generic BPM electronics platform called "DBPM3" in the Athos soft X-ray branch, as well as for new BPMs and general controls hardware and devices for SLS 2.0, a major upgrade of the Swiss Light Source. We are also in the process of upgrading our previous "MBU" (modular BPM Unit) platform for the SwissFEL linac and hard X-ray "Aramis" from external VMEbus based IOCs to integrated add-on cards with MPSoC IOCs. On all these MPSoCs, we are integrating an EPICS IOC, event receiver, measurement and feedback data real-time processing on a single chip. In this contribution, we describe our experience with the tight integration and daily operation of the various firmware and software components and features on the MPSoC, using the BPM system also to discuss general aspects relevant for other systems and components discussed in other PSI contributions on this conference.

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO01

About •

Received ※ 06 October 2023 — Revised ※ 09 October 2023 — Accepted ※ 23 November 2023 — Issued ※ 11 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO02

HydRA: A System-on-Chip to Run Software in Radiation-Exposed Areas

217

T. Gingold, G. Daniluk, J. Serranopresenter, T. Włostowski
CERN, Meyrin, Switzerland
M. Rizzi
PSI, Villigen PSI, Switzerland

In the context of the High-Luminosity LHC project at CERN, a platform has been developed to support groups needing to host electronics in radiation-exposed areas. This platform, called DI/OT, is based on a modular kit consisting of a System Board, Peripheral Boards and a radiation-tolerant power converter, all housed in a standard 3U crate. Groups customise their systems by designing Peripheral Boards and developing custom gateware and software for the System Board, featuring an IGLOO2 flash-based FPGA. It is compulsory for gateware designs to be radiation-tested in dedicated facilities before deployment. This process can be cumbersome and affects iteration time because access to radiation testing facilities is a scarce commodity. To make customisation more agile, we have developed a radiation-tolerant System-on-Chip (SoC), so that a single gateware design, extensively validated, can serve as a basis for different applications by just changing the software running in the processing unit of the SoC. HydRA (Hydra-like Resilient Architecture) features a triplicated RISC-V processor for safely running software in a radiation environment. This paper describes the overall context for the project, and then moves on to provide detailed explanations of all the design decisions for making HydRA radiation-tolerant, including the protection of programme and data memories. Test harnesses are also described, along with a summary of the test results so far. It concludes with ideas for further development and plans for deployment in the LHC.
https://ohwr.org/project/hydra/wikis/home
https://ohwr.org/project/diot/wikis/home

Slides MO4AO02 [11.131 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO02

About •

Received ※ 06 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 27 October 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO03

The DESY Open Source FPGA Framework

222

Ł. Butkowski, A. Bellandi, M. Büchler, B. Dursun, Ç. Gümüş, N. Omidsajedi, K. Schulz
DESY, Hamburg, Germany

Modern FPGA firmware development involves integrating various intellectual properties (IP), modules written in hardware description languages (HDL), high-level synthesis (HLS), and software/hardware CPUs with embedded Linux or bare-metal applications. This process may involve multiple tools from the same or different vendors, making it complex and challenging. Additionally, scientific institutions such as DESY require long-term maintenance and reproducibility for designs that may involve multiple developers, further complicating the process. To address these challenges, we have developed an open-source FPGA firmware framework (FWK) at DESY that streamlines development, facilitates collaboration, and reduces complexity. The FWK achieves this by providing an abstraction layer, a defined structure, and guidelines to create big FPGA designs with ease. FWK also generates documentation and address maps necessary for high-level software frameworks like ChimeraTK. This paper presents an overview and the idea of the FWK.

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO03

About •

Received ※ 05 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 13 October 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO05

Development of a Timing and Data Link for EIC Common Hardware Platform

228

P. Bachek, T. Hayes, J. Mead, K. Mernick, G. Narayan, F. Severino
BNL, Upton, New York, USA

Funding: Contract Number DE-AC02-98CH10886 with the auspices of the US Department of Energy
Modern timing distribution systems benefit from high configurability and the bidirectional transfer of timing data. The Electron Ion Collider (EIC) Common Hardware Platform (CHP) will integrate the functions of the existing RHIC Real Time Data Link (RTDL), Event Link, and Beam Sync Link, along with the Low-Level RF (LLRF) system Update Link (UL), into a common high speed serial link. One EIC CHP carrier board sup-ports up to eight external 8 Gbps high speed links via SFP+ modules, as well as up to six 8 Gbps high speed links to each of two daughterboards. A daughterboard will be designed for the purpose of timing data link distribution for use with the CHP. This daughterboard will have two high speed digital crosspoint switches and a Xilinx Artix Ultrascale⁺ FPGA onboard with GTY transceivers. One of these will be dedicated for a high-speed control and data link directly between the onboard FPGA and the carrier FPGA. The remaining GTY transceivers will be routed through the crosspoint switches. The daughterboard will support sixteen external SFP+ ports for timing distribution infrastructure with some ports dedicated for transmit only link fanout. The timing data link will support bidirectional data transfer including sending data or events from a downstream device back upstream. This flexibility will be achieved by routing the SFP+ ports through the crosspoint switches which allows the timing link datapaths to be forwarded directly through the daughterboard to the carrier and into the FPGA on the daughterboard in many different configurations.

Slides MO4AO05 [1.236 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO05

About •

Received ※ 05 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 23 November 2023 — Issued ※ 07 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO06

Overview and Outlook of FPGA Based Hardware Solutions for Data Synchronization, Acquisition and Processing at the Euxfel

233

B.J. Fernandes, F. Babies, T. Freyermuth, P. Gessler, I.S. Soekmen, H. Sotoudi Namin
EuXFEL, Schenefeld, Germany

The European X-Ray Free Electron Laser facility (EuXFEL) provides ultra short coherent X-Ray flashes, spaced by 220 nanoseconds and with a duration of less than 100 femtoseconds, in bursts of up to 2700 pulses every 100ms to several instruments. The facility has been using standardized Field-Programmable Gate Array (FPGA) based hardware platforms since the beginning of user operation in 2017. These are used for timing distribution, data processing from large 2D detectors, high speed digitizers for acquisition and processing of pulse signals, monitoring beam characteristics, and low latency communication protocol for pulse data vetoing and Machine Protection System (MPS). Our experience grows in tandem with user requests for more specific and challenging case studies, leading to tailor made hardware algorithms and setups. In some cases, these can be fulfilled with the integration of new hardware, where collaboration with companies for new and/or updated platforms is a key factor, or taking advantage of unused features in current setups. In this overview, we present the FPGA hardware based solutions used to fulfill EuXFEL’s requirements. We also present our efforts in integrating new solutions and possible development directions, including Machine Learning (ML) research, with the aim of bringing more accurate results and configurable setups to user experiments and facilitate communications with other platforms used in the facility, namely Programmable Logic Controllers (PLC).

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO06

About •

Received ※ 06 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 23 October 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO4AO07

Status of the MicroTCA Based Beam Instrumentation DAQ Systems at GSI and FAIR

239

THPDP015

use link to see paper's listing under its alternate paper code

T. Hoffmann, H. Bräuning, R.N. Geißler, T. Milosic
GSI, Darmstadt, Germany

While the first FAIR accelerator buildings are soon to be completed, MicroTCA-based data acquisition sys-tems for FAIR beam instrumentation are ready for use. By using commercial off-the-shelf components as well as open hardware with in-house expertise in FPGA programming, there are now DAQ solutions for almost all major detector systems in MicroTCA in operation at the existing GSI accelerators. Applications span a wide range of detector systems and hardware, often taking advantage of the high channel density and data trans-mission bandwidth available with MicroTCA. All DAQ systems are synchronised and triggered using a com-prehensive White Rabbit based timing system. This allows correlation of the data from the distributed acquisition systems on a nanosecond scale. In this paper, we present some examples of our DAQ implemented in MicroTCA covering the range of beam current, tune, position and profile measurements. While the latter uses GigE cameras in combination with scintillating screens, the other applications are based on ADCs with different sampling frequencies between 125 MSa/s up to 2.5 GSa/s or latching scalers with up to 10 MHz latching frequency.

Slides MO4AO07 [3.497 MB]

Poster MO4AO07 [3.790 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4AO07

About •

Received ※ 29 September 2023 — Revised ※ 07 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 07 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)