ICALEPCS2023 - Table of Session: MO2BC (Software Best Practises)

Paper	Title	Page
MO2BCO01	Driving Behavioural Change of Software Developers in a Global Organisation Assisted by a Paranoid Android	25
	U.Y. Yilmaz, M.G.P.T. Android SKAO, Macclesfield, United Kingdom M.J.A. de Beerpresenter SARAO, Cape Town, South Africa
	Ensuring code quality standards at the Square Kilometre Array Observatory (SKAO) is of utmost importance, as the project spans multiple nations and encompasses a wide range of software products delivered by developers from around the world. To improve code quality and meet certain open-source software prerequisites for a wider collaboration, the SKAO employs the use of a chatbot that provides witty, direct and qualified comments with detailed documentation that guide developers in improving their coding practices. The bot is modelled after a famous character albeit a depressed one, creating a relatable personality for developers. This has resulted in an increase in code quality and faster turnaround times. The bot has not only helped developers adhere to code standards but also fostered a culture of continuous improvement with an engaging and enjoyable process. Here we present the success story of the bot and how a chatbot can drive behavioural change within a global organisation and help DevOps teams to improve developer performance and agility through an innovative and engaging approach to code reviews.
	Slides MO2BCO01 [8.171 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO01
About •	Received ※ 06 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 19 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO02	Concept and Design of an Extensible Middle-Layer Application Framework for Accelerator Operations and Development	30
	M. Schütte, J. Georgpresenter, A. Grünhagen, H. Schlarb DESY, Hamburg, Germany
	Data collection and analysis are becoming increasingly vital not only for the experiments conducted with particle accelerators but also for their operation, maintenance, and development. Due to lack of feasible alternatives, experts regularly resort to writing task-specific scripts to perform actions such as (event triggered or temporary) data collection, system failure detection and recovery, and even simple high-level feedbacks. Often, these scripts are not shared and are deemed to have little reuse value, giving them a short lifetime and causing redundant work. We report on a modular Python framework for constructing middle-layer applications from a library of parameterized functionality blocks (modules) by writing a simple configuration file in a human-oriented format. This encourages the creation of maintainable and reusable modules while offering an increasingly powerful and flexible platform that has few requirements to the user. A core engine instantiates the modules according to the configuration file, collects the required data from the control system and distributes it to the individual module instances for processing. Additionally, a publisher-subscriber messaging system is provided for inter-module communication. We discuss architecture & design choices, current state and future goals of the framework as well as real use-case examples from the European XFEL.
	Slides MO2BCO02 [1.915 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO02
About •	Received ※ 05 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 30 October 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO03	Strategy and Tools to Test Software in the SKA Project: The CSP. LMC Case	34
	G. Marotta, C. Baffa, E. Giani INAF - OA Arcetri, Firenze, Italy G. Brajnik IDS, Udine, Italy M. Colciago, I. Novak Cosylab Switzerland, Brugg, Switzerland
	The Square Kilometre Array (SKA) Telescope will be one of the largest and most complex scientific instruments ever built. The development of a reliable software for monitoring and controlling its operations is critical to the success of the entire SKA project. The Local Monitoring and Control of the Central Signal Processor (CSP. LMC) is a software responsible for controlling a key subsystem of the telescope, i.e. the Central Signal Processor (CSP). The software is implemented as a "device" within the TANGO framework, written in Python. In this paper we describe a testing strategy that addresses some typical problems of such a large and complex instrument. It is a multi-level strategy, based on a combination of automated tests (unit/component/integration), in the context of CI/CD practices. Software is also tested against errors and anomalous conditions that can occur while the CSP. LMC is interacting with external subsystems, which can be simulated. The paper also discusses needs and solutions based on data mining test results. This allows us to obtain statistics of unexpected failures and to investigate their causes. Furthermore, a database containing test results supports discovery of interesting and unexpected patterns of behaviors of the tests based on correlations about different test-related events and data. This helps us to develop a deeper understanding of the code’s functioning and to find suitable solutions to minimize unexpected behaviors. In addition it can be used also to support reliability testing.
	Slides MO2BCO03 [2.336 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO03
About •	Received ※ 06 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 13 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO04	Applying Standardised Software Architectural Concepts to Design Robust and Adaptable PLC Solutions	40
	S.T. Huynh, B. Baranasic, M. Bueno, L. Feltrin Zanellatto, T. Freyermuth, P. Gessler, N. Jardón Bueno, N. Mashayekh, J. Tolkiehn EuXFEL, Schenefeld, Germany
	Between evolving requirements, additional feature requests and urgent maintenance tasks, the Programmable Logic Controllers (PLC) at the European X-Ray Free Electron Laser Facility (EuXFEL) have become subjected to an array of demands. As the maintainability effort towards the existing systems peak, the requirement for a sustainable solution become an ever pressing concern. Ultimately, in order to provide a PLC code base which can easily be supported and adapted to, a reworking was required from the ground up in the form of a new suite of libraries and tools. Through this, it was possible to bring standardised software principals into PLC design and development, conjunctively offering an interface into the existing code base for ongoing support of legacy code. The set of libraries are developed by incorporating software engineering principles and design patterns in test driven development within a layered architecture. In defining clear interfaces across all the architectural layers - from hardware, to the software representation of hardware, and clusters of software devices, the complexity of PLC development decreases down into modular blocks of unit tested code. Regular tasks such as the addition of features, modifications or process control can easily be performed due to the adaptability, flexibility and modularity of the core PLC code base.
	Slides MO2BCO04 [0.910 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO04
About •	Received ※ 05 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 09 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO05	Enabling Transformational Science Through Global Collaboration and Innovation Using the Scaled Agile Framework	47
	L.R. Brederode, S. Ujjainkar, S. Valame SKAO, Macclesfield, United Kingdom J. Coles University of Cambridge, Cambridge, United Kingdom F. Graser VIVO, Somerset West, South Africa J.A. Kolatkar PSL, Pune, India
	Funding: Square Kilometre Array Observatory The SKAO is one observatory, with two telescopes on three continents. It will be the world’s largest radio telescope once constructed, and will be able to observe the sky with unprecedented sensitivity and resolution. The SKAO software and computing systems will largely be responsible for orchestrating the observatory and associated telescopes, and processing the science data, before data products are distributed to regional science centres. The Scaled Agile Framework (SAFe) is being leveraged to coordinate over thirty lean agile development teams that are distributed throughout the world. In this paper, we report on our experience in using the Scaled Agile Framework, the successes we have enjoyed, as well as the impediments and challenges that have stood in our way.
	Slides MO2BCO05 [6.064 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO05
About •	Received ※ 06 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 12 October 2023 — Issued ※ 15 October 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO06	Embedded Controller Software Development Best Practices at the National Ignition Facility	54
	V.K. Gopalan, A.I. Barnes, C.M. Estes, J.M. Fisher, V.J. Hernandez, P. Kale, A. Pao, P.K. Singh LLNL, Livermore, USA
	Funding: This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. Software development practices such as continuous integration and continuous delivery (CI/CD) are widely adopted by the National Ignition Facility (NIF) which helps to automate the software development, build, test, and deployment processes. However, using CI/CD in an embedded controller project poses several challenges due to the limited computing resources such as processing power, memory capacity and storage availability in such systems. This paper will present how CI/CD best practices were tailored and used to develop and deploy software for one of the NIF Master Oscillator Room (MOR) embedded controllers, which is based on custom designed hardware consisting of a microcontroller and a variety of laser sensors and drivers. The approach included the use of automated testing frameworks, customized build scripts, simulation environments, and an optimized build and deployment pipeline, leading to quicker release cycles, improved quality assurance and quicker defect correction. The paper will also detail the challenges faced during the development and deployment phases and the strategies used to overcome them. The experience gained with this methodology on a pilot project demonstrated that using CI/CD in embedded controller projects can be challenging, yet feasible with the right tools and strategies, and has the potential to be scaled and applied to the vast number of embedded controllers in the NIF control system. LLNL Release Number: LLNL-ABS-848418
	Slides MO2BCO06 [1.346 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO06
About •	Received ※ 29 September 2023 — Revised ※ 12 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 30 November 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO07	Continuous Integration and Debian Packaging for Rapidly Evolving Software	61
	A.W.C. Barker, J. Georgpresenter, M. Hierholzer, M. Killenberg, T. Kozak, D. Rothe, N. Shehzad, C. Willner DESY, Hamburg, Germany
	We describe our Jenkins-based continuous integration system and Debian packaging methods, and their application to the rapid development of the ChimeraTK framework. ChimeraTK is a C++ framework for control system applications and hardware access with a high level of abstraction and consists of more than 30 constantly changing interdependent libraries. Each component has its own release cycle for rapid development, yet API and ABI changes must be propagated to prevent problems in dependent libraries and over 60 applications. We present how we configured a Jenkins-based continuous integration system to detect problems quickly and systematically for the rapid development of ChimeraTK. The Debian packaging system is designed to ensure the compatibility of binary interfaces (ABI) and of development files (API). We present our approach using build scripts that allow the deployment of rapidly changing libraries and their dependent applications as Debian packages. These even permit applications to load runtime plugins that draw from the same core library, yet are compiled independently.
	Slides MO2BCO07 [0.805 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO07
About •	Received ※ 06 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 26 October 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

Driving Behavioural Change of Software Developers in a Global Organisation Assisted by a Paranoid Android

25

U.Y. Yilmaz, M.G.P.T. Android
SKAO, Macclesfield, United Kingdom
M.J.A. de Beerpresenter
SARAO, Cape Town, South Africa

Ensuring code quality standards at the Square Kilometre Array Observatory (SKAO) is of utmost importance, as the project spans multiple nations and encompasses a wide range of software products delivered by developers from around the world. To improve code quality and meet certain open-source software prerequisites for a wider collaboration, the SKAO employs the use of a chatbot that provides witty, direct and qualified comments with detailed documentation that guide developers in improving their coding practices. The bot is modelled after a famous character albeit a depressed one, creating a relatable personality for developers. This has resulted in an increase in code quality and faster turnaround times. The bot has not only helped developers adhere to code standards but also fostered a culture of continuous improvement with an engaging and enjoyable process. Here we present the success story of the bot and how a chatbot can drive behavioural change within a global organisation and help DevOps teams to improve developer performance and agility through an innovative and engaging approach to code reviews.

Slides MO2BCO01 [8.171 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO01

About •

Received ※ 06 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 19 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO02

Concept and Design of an Extensible Middle-Layer Application Framework for Accelerator Operations and Development

30

M. Schütte, J. Georgpresenter, A. Grünhagen, H. Schlarb
DESY, Hamburg, Germany

Data collection and analysis are becoming increasingly vital not only for the experiments conducted with particle accelerators but also for their operation, maintenance, and development. Due to lack of feasible alternatives, experts regularly resort to writing task-specific scripts to perform actions such as (event triggered or temporary) data collection, system failure detection and recovery, and even simple high-level feedbacks. Often, these scripts are not shared and are deemed to have little reuse value, giving them a short lifetime and causing redundant work. We report on a modular Python framework for constructing middle-layer applications from a library of parameterized functionality blocks (modules) by writing a simple configuration file in a human-oriented format. This encourages the creation of maintainable and reusable modules while offering an increasingly powerful and flexible platform that has few requirements to the user. A core engine instantiates the modules according to the configuration file, collects the required data from the control system and distributes it to the individual module instances for processing. Additionally, a publisher-subscriber messaging system is provided for inter-module communication. We discuss architecture & design choices, current state and future goals of the framework as well as real use-case examples from the European XFEL.

Slides MO2BCO02 [1.915 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO02

About •

Received ※ 05 October 2023 — Revised ※ 07 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 30 October 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO03

Strategy and Tools to Test Software in the SKA Project: The CSP. LMC Case

34

G. Marotta, C. Baffa, E. Giani
INAF - OA Arcetri, Firenze, Italy
G. Brajnik
IDS, Udine, Italy
M. Colciago, I. Novak
Cosylab Switzerland, Brugg, Switzerland

The Square Kilometre Array (SKA) Telescope will be one of the largest and most complex scientific instruments ever built. The development of a reliable software for monitoring and controlling its operations is critical to the success of the entire SKA project. The Local Monitoring and Control of the Central Signal Processor (CSP. LMC) is a software responsible for controlling a key subsystem of the telescope, i.e. the Central Signal Processor (CSP). The software is implemented as a "device" within the TANGO framework, written in Python. In this paper we describe a testing strategy that addresses some typical problems of such a large and complex instrument. It is a multi-level strategy, based on a combination of automated tests (unit/component/integration), in the context of CI/CD practices. Software is also tested against errors and anomalous conditions that can occur while the CSP. LMC is interacting with external subsystems, which can be simulated. The paper also discusses needs and solutions based on data mining test results. This allows us to obtain statistics of unexpected failures and to investigate their causes. Furthermore, a database containing test results supports discovery of interesting and unexpected patterns of behaviors of the tests based on correlations about different test-related events and data. This helps us to develop a deeper understanding of the code’s functioning and to find suitable solutions to minimize unexpected behaviors. In addition it can be used also to support reliability testing.

Slides MO2BCO03 [2.336 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO03

About •

Received ※ 06 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 13 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO04

Applying Standardised Software Architectural Concepts to Design Robust and Adaptable PLC Solutions

40

S.T. Huynh, B. Baranasic, M. Bueno, L. Feltrin Zanellatto, T. Freyermuth, P. Gessler, N. Jardón Bueno, N. Mashayekh, J. Tolkiehn
EuXFEL, Schenefeld, Germany

Between evolving requirements, additional feature requests and urgent maintenance tasks, the Programmable Logic Controllers (PLC) at the European X-Ray Free Electron Laser Facility (EuXFEL) have become subjected to an array of demands. As the maintainability effort towards the existing systems peak, the requirement for a sustainable solution become an ever pressing concern. Ultimately, in order to provide a PLC code base which can easily be supported and adapted to, a reworking was required from the ground up in the form of a new suite of libraries and tools. Through this, it was possible to bring standardised software principals into PLC design and development, conjunctively offering an interface into the existing code base for ongoing support of legacy code. The set of libraries are developed by incorporating software engineering principles and design patterns in test driven development within a layered architecture. In defining clear interfaces across all the architectural layers - from hardware, to the software representation of hardware, and clusters of software devices, the complexity of PLC development decreases down into modular blocks of unit tested code. Regular tasks such as the addition of features, modifications or process control can easily be performed due to the adaptability, flexibility and modularity of the core PLC code base.

Slides MO2BCO04 [0.910 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO04

About •

Received ※ 05 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 09 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO05

Enabling Transformational Science Through Global Collaboration and Innovation Using the Scaled Agile Framework

47

L.R. Brederode, S. Ujjainkar, S. Valame
SKAO, Macclesfield, United Kingdom
J. Coles
University of Cambridge, Cambridge, United Kingdom
F. Graser
VIVO, Somerset West, South Africa
J.A. Kolatkar
PSL, Pune, India

Funding: Square Kilometre Array Observatory
The SKAO is one observatory, with two telescopes on three continents. It will be the world’s largest radio telescope once constructed, and will be able to observe the sky with unprecedented sensitivity and resolution. The SKAO software and computing systems will largely be responsible for orchestrating the observatory and associated telescopes, and processing the science data, before data products are distributed to regional science centres. The Scaled Agile Framework (SAFe) is being leveraged to coordinate over thirty lean agile development teams that are distributed throughout the world. In this paper, we report on our experience in using the Scaled Agile Framework, the successes we have enjoyed, as well as the impediments and challenges that have stood in our way.

Slides MO2BCO05 [6.064 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO05

About •

Received ※ 06 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 12 October 2023 — Issued ※ 15 October 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO06

Embedded Controller Software Development Best Practices at the National Ignition Facility

54

V.K. Gopalan, A.I. Barnes, C.M. Estes, J.M. Fisher, V.J. Hernandez, P. Kale, A. Pao, P.K. Singh
LLNL, Livermore, USA

Funding: This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
Software development practices such as continuous integration and continuous delivery (CI/CD) are widely adopted by the National Ignition Facility (NIF) which helps to automate the software development, build, test, and deployment processes. However, using CI/CD in an embedded controller project poses several challenges due to the limited computing resources such as processing power, memory capacity and storage availability in such systems. This paper will present how CI/CD best practices were tailored and used to develop and deploy software for one of the NIF Master Oscillator Room (MOR) embedded controllers, which is based on custom designed hardware consisting of a microcontroller and a variety of laser sensors and drivers. The approach included the use of automated testing frameworks, customized build scripts, simulation environments, and an optimized build and deployment pipeline, leading to quicker release cycles, improved quality assurance and quicker defect correction. The paper will also detail the challenges faced during the development and deployment phases and the strategies used to overcome them. The experience gained with this methodology on a pilot project demonstrated that using CI/CD in embedded controller projects can be challenging, yet feasible with the right tools and strategies, and has the potential to be scaled and applied to the vast number of embedded controllers in the NIF control system.
LLNL Release Number: LLNL-ABS-848418

Slides MO2BCO06 [1.346 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO06

About •

Received ※ 29 September 2023 — Revised ※ 12 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 30 November 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

MO2BCO07

Continuous Integration and Debian Packaging for Rapidly Evolving Software

61

A.W.C. Barker, J. Georgpresenter, M. Hierholzer, M. Killenberg, T. Kozak, D. Rothe, N. Shehzad, C. Willner
DESY, Hamburg, Germany

We describe our Jenkins-based continuous integration system and Debian packaging methods, and their application to the rapid development of the ChimeraTK framework. ChimeraTK is a C++ framework for control system applications and hardware access with a high level of abstraction and consists of more than 30 constantly changing interdependent libraries. Each component has its own release cycle for rapid development, yet API and ABI changes must be propagated to prevent problems in dependent libraries and over 60 applications. We present how we configured a Jenkins-based continuous integration system to detect problems quickly and systematically for the rapid development of ChimeraTK. The Debian packaging system is designed to ensure the compatibility of binary interfaces (ABI) and of development files (API). We present our approach using build scripts that allow the deployment of rapidly changing libraries and their dependent applications as Debian packages. These even permit applications to load runtime plugins that draw from the same core library, yet are compiled independently.

Slides MO2BCO07 [0.805 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO07

About •

Received ※ 06 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 26 October 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)