Paper | Title | Other Keywords | Page |
---|---|---|---|
MO2BCO03 | Strategy and Tools to Test Software in the SKA Project: The CSP. LMC Case | software, controls, framework, software-component | 34 |
|
|||
The Square Kilometre Array (SKA) Telescope will be one of the largest and most complex scientific instruments ever built. The development of a reliable software for monitoring and controlling its operations is critical to the success of the entire SKA project. The Local Monitoring and Control of the Central Signal Processor (CSP. LMC) is a software responsible for controlling a key subsystem of the telescope, i.e. the Central Signal Processor (CSP). The software is implemented as a "device" within the TANGO framework, written in Python. In this paper we describe a testing strategy that addresses some typical problems of such a large and complex instrument. It is a multi-level strategy, based on a combination of automated tests (unit/component/integration), in the context of CI/CD practices. Software is also tested against errors and anomalous conditions that can occur while the CSP. LMC is interacting with external subsystems, which can be simulated. The paper also discusses needs and solutions based on data mining test results. This allows us to obtain statistics of unexpected failures and to investigate their causes. Furthermore, a database containing test results supports discovery of interesting and unexpected patterns of behaviors of the tests based on correlations about different test-related events and data. This helps us to develop a deeper understanding of the code’s functioning and to find suitable solutions to minimize unexpected behaviors. In addition it can be used also to support reliability testing. | |||
Slides MO2BCO03 [2.336 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO2BCO03 | ||
About • | Received ※ 06 October 2023 — Revised ※ 08 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 13 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
MO3AO02 | Implementation of Model Predictive Control for Slow Orbit Feedback Control in MAX IV Accelerators Using PyTango Framework | controls, feedback, operation, storage-ring | 116 |
|
|||
Achieving low emittance and high brightness in modern light sources requires stable beams, which are commonly achieved through feedback solutions. The MAX IV light source has two feedback systems, Fast Orbit Feedback (FOFB) and Slow Orbit Feedback (SOFB), operating in overlapping frequency regions. Currently in MAX IV, a general feedback device implemented in PyTango is used for slow orbit and trajectory correction, but an MPC controller for the beam orbit has been proposed to improve system robustness. The controller uses iterative optimisation of the system model, current measurements, dynamic states and system constraints to calculate changes in the controlled variables. The new device implements the MPC model according to the beam orbit response matrix, subscribes to change events on all beam position attributes and updates the control signal given to the slow magnets with a 10 Hz rate. This project aims to improve system robustness and reduce actuator saturation. The use of PyTango simplifies the implementation of the MPC controller by allowing access to high-level optimisation and control packages. This project will contribute to the development of a high-quality feedback control system for MAX IV accelerators. | |||
Slides MO3AO02 [4.234 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO3AO02 | ||
About • | Received ※ 05 October 2023 — Revised ※ 09 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 19 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
MO4BCO01 | Using BDD Testing in SKAO: Challenges and Opportunities | software, controls, distributed, interface | 183 |
|
|||
Defining what a system should do is one of the hardest parts of system design. Using Behaviour Driven Design (BDD) techniques can help, and also help define the tests needed to check that the desired behaviour is implemented. We describe the challenges and opportunities that arise when adopting these techniques, including both technical and social issues, and especially why in our case BDD techniques provide significant value. We present our pathway towards using BDD and the lessons learned. By trying to use BDD testing to run integration tests, it enabled the identification of gaps in the testing infrastructure, particularly the TANGO testing infrastructure, and gaps in developers’ understanding of the system design. This allowed SKAO to take steps to improve the tests, the infrastructure, and the design, by integrating BDD techniques into the full product development lifecycle and using them also for monitoring the development process and the quality of software products. | |||
Slides MO4BCO01 [1.496 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4BCO01 | ||
About • | Received ※ 06 October 2023 — Revised ※ 10 October 2023 — Accepted ※ 14 November 2023 — Issued ※ 09 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
MO4BCO04 | Improving Control System Software Deployment at MAX IV | software, controls, device-server, Linux | 201 |
|
|||
The control systems of large research facilities like synchrotrons are composed of many different hardware and software parts. Deploying and maintaining such systems require proper workflows and tools. MAX IV has been using Ansible to manage and deploy its full control system, both software and infrastructure, for many years with great success. We detail further improvements: defining Tango devices as configuration, and automated deployment of specific packages when tagging Gitlab repos. We have now adopted Conda as our primary packaging tool instead of the Red Hat Package Manager (RPM). This allows us to keep up with the rapidly changing Python ecosystem, while at the same time decoupling Operating System upgrades from the control system software. For better management, we have developed a Prometheus-based tool that reports on the installed versions of each package on each machine. This paper will describe our workflow and discuss the benefits and drawbacks of our approach. | |||
Slides MO4BCO04 [1.969 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-MO4BCO04 | ||
About • | Received ※ 06 October 2023 — Accepted ※ 13 October 2023 — Issued ※ 26 October 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUMBCMO09 | Front-End Monitor and Control Web Application for Large Telescope Infrastructures: A Comparative Analysis | controls, framework, interface, operation | 359 |
|
|||
A robust monitor and control front-end application is a crucial feature for large and scalable radio telescope infrastructures such LOFAR and SKA, whereas the control system is required to manage numerous attribute values at a high update rate, and thus the operators must rely on an affordable user-interface platform which covers the whole range of operations. In this paper two state-of-the-art web applications such Grafana and Taranta are taken into account, developing a comparative analysis between the two software suites. Such a choice is motivated mostly because of their widespread use together with the TANGO Controls Framework, and the necessity to offer a ground of comparison for large projects dealing with the development of a monitor and control GUI which interfaces to TANGO. We explain at first the general architecture of both systems, and then we create a typical use-case where an interactive dashboard is built to monitor and control a hardware device. Then, we set up some comparable metrics to evaluate the pros and cons of both platforms, regarding the technical and operational requirements, fault tolerances, developers and operators efforts, and so on. In conclusion, the comparative analysis and its results are summarized with the aim to offer the stakeholders a basis for future choices. | |||
Slides TUMBCMO09 [0.621 MB] | |||
Poster TUMBCMO09 [1.552 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUMBCMO09 | ||
About • | Received ※ 05 October 2023 — Revised ※ 12 October 2023 — Accepted ※ 22 November 2023 — Issued ※ 27 November 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUMBCMO19 | MAX IV Laboratory’s Control System Evolution and Future Strategies | controls, experiment, operation, detector | 395 |
|
|||
The MAX IV Laboratory, a 4th generation synchrotron radiation facility located in southern Sweden, has been operational since 2016. With multiple beamlines and experimental stations completed and in steady use, the facility is now approaching the third phase of development, which includes the final two of the 16 planned beamlines in user operation. The focus is on achieving operational excellence by optimizing reliability and performance. Meanwhile, the strategy for the coming years is driven by the need to accommodate a growing user base, exploring the possibility of operating a Soft X-ray Laser (SXL), and achieving the diffraction limit for 10 keV of the 3 GeV. The Technical Division is responsible for the control and computing systems of the entire laboratory. This new organization provides a coherent strategy and a clear vision, with the ultimate goal of enabling science. The increasing demand for more precise and efficient control systems has led to significant developments and maintenance efforts. Pushing the limits in remote access, data generation, time-resolved and fly-scan experiments, and beam stability requires the proper alignment of technology in IT infrastructure, electronics, software, data analysis, and management. This article discusses the motivation behind the updates, emphasizing the expansion of the control system’s capabilities and reliability. Lastly, the technological strategy will be presented to keep pace with the rapidly evolving technology landscape, ensuring that MAX IV is prepared for its next major upgrade. | |||
Slides TUMBCMO19 [8.636 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUMBCMO19 | ||
About • | Received ※ 06 October 2023 — Revised ※ 12 October 2023 — Accepted ※ 24 November 2023 — Issued ※ 29 November 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUMBCMO32 | DevPylon, DevVimba: Game Changers at LULI | laser, controls, device-server, software | 441 |
|
|||
Funding: CNRS, École polytechnique, CEA, Sorbonne Université Apollon, LULI2000 and HERA are three Research Infrastructures of the Centre national de la recherche scientifique (CNRS), École polytechnique (X), Commissariat à l’Énergie Atomique et aux Energies Alternatives (CEA) and Sorbonne University (SU). Past-commissioning phase, Apollon is a four beam laser, multi-petawatt laser facility fitted with instrumentation technologies on the cutting edge with two experimental areas (short–up to 1m–and long focal–up to 20m, 32m in the future). To monitor the laser beam characteristics through the interaction chambers, more than 500 devices are distributed in the facility and controlled through a Tango bus. This poster focuses on two linked software components: DevPylon and DevVimba. Each affected to a type of cameras: Basler via PyPylon wrapper interface of Pylon Software suite and Prosilica via Vimba SDK library, respectively. These two Tango devices are Python scripts constructed and generated via POGO. They offer a specific way to monitor more than 100 CCD cameras in the facility at an image acquisition and display rate up to 10Hz for a maximum of 300-shot at 1-minute rate per day and on an always-ON mode throughout the day. |
|||
Slides TUMBCMO32 [1.030 MB] | |||
Poster TUMBCMO32 [1.421 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUMBCMO32 | ||
About • | Received ※ 09 October 2023 — Revised ※ 20 November 2023 — Accepted ※ 20 December 2023 — Issued ※ 20 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUMBCMO38 | Towards the Zero Code Waste to Increase the Impact of Science | software, controls, FEL, survey | 456 |
|
|||
Accelerators and other big science facilities rely heavily on internally developed technologies, including control system software. Much of it can and is shared between labs, like the Tango Controls and EPICS. Then, some of it finds broad application outside science, like the famous World Wide Web. However, there are still a lot of duplicating efforts in the labs, and a lot of software has the potential to be applied in other areas. Increasing collaboration and involving private companies can help avoid redundant work. It can decrease the overall costs of laboratory development and operation. Having private industry involved in technology development also increases the chances of new applications. This can positively impact society, which means effective spending of public funds. The talk will be based on the results of a survey looking at how much scientific institutes and companies focus on collaboration and dissemination in the field of software technologies. It will also include remarks based on the authors’ experiences in building an innovative ecosystem. | |||
Slides TUMBCMO38 [0.294 MB] | |||
Poster TUMBCMO38 [1.016 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUMBCMO38 | ||
About • | Received ※ 06 October 2023 — Revised ※ 12 October 2023 — Accepted ※ 28 November 2023 — Issued ※ 06 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUPDP012 | Tango at LULI | laser, controls, GUI, network | 509 |
|
|||
Funding: CNRS, École polytechnique, CEA, Sorbonne Université Apollon, LULI2000 and HERA are three Research Infrastructures of the Centre national de la recherche scientifique (CNRS), École polytechnique (X), Commissariat à l’Énergie Atomique et aux Energies Alternatives (CEA) and Sorbonne University (SU). Now in past-commissioning phase, Apollon is a four beam laser, multi-petawatt laser facility fitted with instrumentation technologies on the cutting edge with two experimental areas (short–up to 1m–and long focal–up to 20m, 32m in the future). To monitor the laser beam characteristics through the interaction chambers, more than 300 devices are distributed in the facility and controlled through a Tango bus. This poster presents primarily a synthetic view of the Apollon facility, from network to hardware and from virtual machines to software under Tango architecture. We can here have an overview of the different types of devices which are running on the facility and some GUIs developed with the exploitation team to insure the best possible way of running the lasers. While developments are still currently under work for this facility, upgrading the systems of LULI2000 from one side and HERA from the other side are underway by the Control-Command & Supervision team and would follow the same specifications to offer shared protocols and knowledge. |
|||
Poster TUPDP012 [2.267 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUPDP012 | ||
About • | Received ※ 12 October 2023 — Revised ※ 09 November 2023 — Accepted ※ 17 December 2023 — Issued ※ 19 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUPDP022 | DALI Control System Considerations | EPICS, controls, software, interface | 547 |
|
|||
The Dresden Advanced Light Infrastructure (DALI) is part of the German national Helmholtz Photon Science Roadmap. It will be a high-field source of intense terahertz radiation based on accelerated electrons and the successor of the Center for High-Power Radiation Sources (ELBE) operated at HZDR since 2002. In the current phase of DALI the conceptional design report is in preparation and there are ongoing considerations which control system to use best. We will present the status of those considerations, that include defining the requirements for the control system and a discussion of control system candidates. In the early conceptional phase we are still open to any control system that can fulfill our requirements. Besides pure technical performance, features and security the requirements encompass modernity, well established support by community and companies, long term availability as well as collaboration potential and benefit. To collect opinions from the community on what is the optimal control system we prepared a survey. Like that we would like to benefit as much as possible from the community experience with different types of control systems. | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUPDP022 | ||
About • | Received ※ 05 October 2023 — Revised ※ 13 October 2023 — Accepted ※ 04 December 2023 — Issued ※ 18 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUPDP034 | GeCo: The Elettra 2.0 Beamline Control System | controls, PLC, interface, software | 583 |
|
|||
The Elettra Synchrotron, located in Italy near Trieste, has been operating for users since 1994 being the first third generation light source for soft X-rays in Europe. To stay competitive for world-class photon science, a massive upgrade of the storage ring has been planned in 2025. The goal is to build an ultra-low emittance light source with ultra-high brilliance in the same building as the present storage ring. The downtime for installation and commissioning of Elettra 2.0 will last 18 months. In this plan, 20 of the present beamlines should be upgraded and 12 new beamlines are scheduled to be built. In this scenario, also the original beamline interlock and personnel safety systems are going to be upgraded using state of the art technologies. Siemens PLCs will be used for low level control, while higher level applications will be developed using the Tango framework. This work presents and describes the architecture of the future Elettra 2.0 beamline control system named GeCo, Gestione e Controllo in italian. | |||
Poster TUPDP034 [1.917 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUPDP034 | ||
About • | Received ※ 06 October 2023 — Revised ※ 09 November 2023 — Accepted ※ 14 December 2023 — Issued ※ 15 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUPDP044 | Improving Performance of Taranta: Analysis of Memory Requests and Implementation of the Solution | software, interface, controls, MMI | 617 |
|
|||
Taranta is a software suite for generating graphical interfaces for Tango Controls software, currently adopted by MaxIV for scientific experiment usage, SKA during the current construction phase for the development of engineering interfaces for device debugging, and other institutions. A key feature of Taranta is the ability to create customizable dashboards without writing code, making it easy to create and share views among users by linking the dashboards to their own tango devices. However, due to the simplicity and capabilities of Taranta’s widgets, more and more users are creating complex dashboards, which can cause client-side resource problems. Through an analysis of dashboards, we have found that excessive memory requests are generated by a large amount of data. In this article, we report on the process we believe will help us solve this performance issue. Starting with an analysis of the existing architecture, the issues encountered, and performance tests, we identify the causes of these problems. We then study a new architecture exploiting all the potential of the Javascript framework React on which Taranta is built, before moving on to implementation of the solution. | |||
Poster TUPDP044 [1.549 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUPDP044 | ||
About • | Received ※ 04 October 2023 — Revised ※ 18 October 2023 — Accepted ※ 14 December 2023 — Issued ※ 16 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUPDP045 | Monitoring the SKA Infrastructure for CICD | monitoring, target, database, distributed | 622 |
|
|||
Funding: INAF The Square Kilometre Array (SKA) is an international effort to build two radio interferometers in South Africa and Australia, forming one Observatory monitored and controlled from global headquarters (GHQ) based in the United Kingdom at Jodrell Bank. The selected solution for monitoring the SKA CICD (continuous integration and continuous deployment) Infrastructure is Prometheus with the help of Thanos. Thanos is used for high availability, resilience, and long term storage retention for monitoring data. For data visualisation, the Grafana project emerged as an important tool for displaying data in order to make specific reasoning and debugging of particular aspect of the infrastructure in place. In this paper, the monitoring platform is presented while considering quality aspect such as performance, scalability, and data preservation. |
|||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUPDP045 | ||
About • | Received ※ 27 September 2023 — Revised ※ 18 October 2023 — Accepted ※ 14 December 2023 — Issued ※ 19 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUPDP076 | Preliminary Design for the ALBA II Control System Stack | controls, hardware, GUI, software | 685 |
|
|||
One of the main pillars of the ALBA Synchrotron Light Source (Barcelona, Spain) Strategy Plan is the preparation of ALBA to be upgraded to a fourth-generation light source. To accomplish this, a preliminary design of the accelerator has already been initiated in 2021. On the Computing side, the upgrade of the accelerator will require a comprehensive overhaul of most parts of the Control System, DAQ, Timing, and many other systems as well as DevOps strategies. This need for a major redesign will bring new architectural challenges, and opportunities to benefit from new technologies that were not present at the time ALBA was designed and build. This paper presents the preliminary design studies, pilot projects, new approaches to development coordination and management, and the preparation plan to acquire the knowledge and experience needed to excel in the ALBA II Control System Stack design. | |||
Poster TUPDP076 [1.095 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUPDP076 | ||
About • | Received ※ 06 October 2023 — Revised ※ 11 October 2023 — Accepted ※ 14 December 2023 — Issued ※ 17 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TUPDP083 | DAQ System Based on Tango, Sardana and PandABox for Millisecond Time Resolved Experiment at the CoSAXS Beamline of MAX IV Laboratory | experiment, laser, controls, detector | 713 |
|
|||
CoSAXS is the Coherent and Small Angle X-ray Scattering (SAXS) beamline placed at the diffraction-limited 3 GeV storage ring at MAX IV Laboratory. The beamline can deliver a very high photon flux ~1013 ph/s and it is equipped with state-of-the-art pixel detectors, suitable for experiments with a high time-resolution to be performed. In this work we present the upgraded beamline data acquisition strategy for a millisecond time-resolved SAXS/WAXS experiment, using laser light to induce temperature jumps or UV-excitation with the consequent structural changes on the system. In general terms, the beamline control system is based on TANGO and built on top of it, Sardana provides an advanced scan framework. In order to synchronize the laser light pulse on the sample, the X-ray fast shutter opening time and the X-ray detectors readout, hardware triggers are used. The implementation is done using PandABox, which generates the pulse train for the laser and for all active experimental channels, such as counters and detectors, in synchronization with the fast shutter opening time. PandABox integration is done with a Sardana Trigger Gate Controller, used to configure the pulses parameters as well to orchestrate the hardware triggers during a scan. This paper describes the experiment orchestration, laser light synchronization with multiple X-ray detector. | |||
Poster TUPDP083 [1.645 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TUPDP083 | ||
About • | Received ※ 06 October 2023 — Accepted ※ 11 December 2023 — Issued ※ 13 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
WE3BCO08 | Efficient and Automated Metadata Recording and Viewing for Scientific Experiments at MAX IV | experiment, interface, database, controls | 1041 |
|
|||
With the advancements in beamline instrumentation, synchrotron research facilities have seen a significant improvement. The detectors used today can generate thousands of frames within seconds. Consequently, an organized and adaptable framework is essential to facilitate the efficient access and assessment of the enormous volumes of data produced. Our communication presents a metadata management solution recently implemented at MAX IV, which automatically retrieves and records metadata from Tango devices relevant to the current experiment. The solution includes user-selected scientific metadata and predefined defaults related to the beamline setup, which are integrated into the Sardana control system and automatically recorded during each scan via the SciFish[1] library. The metadata recorded is stored in the SciCat[2] database, which can be accessed through a web-based interface called Scanlog[3]. The interface, built on ReactJS, allows users to easily sort, filter, and extract important information from the recorded metadata. The tool also provides real-time access to metadata, enabling users to monitor experiments and export data for post-processing. These new software tools ensure that recorded data is findable, accessible, interoperable and reusable (FAIR[4]) for many years to come. Collaborations are on-going to develop these tools at other particle accelerator research facilities.
[1] https://gitlab.com/MaxIV/lib-maxiv-scifish [2] https://scicatproject.github.io/ [3] https://gitlab.com/MaxIV/svc-maxiv-scanlog [4] https://www.nature.com/articles/sdata201618 |
|||
Slides WE3BCO08 [1.914 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-WE3BCO08 | ||
About • | Received ※ 06 October 2023 — Revised ※ 23 October 2023 — Accepted ※ 14 December 2023 — Issued ※ 16 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TH1BCO03 | The Tango Controls Collaboration Status in 2023 | controls, Windows, device-server, software | 1100 |
|
|||
Since 2021 the Tango Controls collaboration has improved and optimised its efforts in many areas. Not only have Special Interest Group meetings (SIGs) been introduced to speed up the adoption of new technologies or improvements, the kernel has switched to a fixed six-month release cycle for quicker adoption of stable kernel versions by the community. CI/CD provides now early feedback on test failures and compatibility issues. Major code refactoring allowed for a much more efficient use of developer resources. Relevant bug fixes, improvements and new features are now adopted at a much higher rate than ever before. The community participation has also noticeably improved. The kernel switched to C++14 and the logging system is undergoing a major refactoring. Among many new features and tools is jupyTango, Jupyter Notebooks on Tango Controls steroids. PyTango is now easy to install via binary wheels, old Python versions are no longer supported, the build-system is switching to CMake, and releases are now made much closer to stable cppTango releases. | |||
Slides TH1BCO03 [1.357 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TH1BCO03 | ||
About • | Received ※ 05 October 2023 — Revised ※ 24 October 2023 — Accepted ※ 21 November 2023 — Issued ※ 13 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TH1BCO04 | Asynchronous Execution of Tango Commands in the SKA Telescope Control System: An Alternative to the Tango Async Device | controls, status, GUI, network | 1108 |
|
|||
Equipment controlled by the Square Kilometre Array (SKA) Control System will have a TANGO interface for control and monitoring. Commands on TANGO device servers have a 3000 milliseconds window to complete their execution and return to the client. This timeout places a limitation on some commands used on SKA TANGO devices which take longer than the 3000 milliseconds window to complete; the threshold is more stricter in the SKA Control System (CS) Guidelines. Such a command, identified as a Long Running Command (LRC), needs to be executed asynchronously to circumvent the timeout. TANGO has support for an asynchronous device which allows commands to be executed slower than 3000 milliseconds by using a coroutine to put the task on an event loop. During the exploration of this, a decision was made to implement a custom approach in our base repository which all devices depend on. In this approach, every command annotated as ¿long running¿ is handed over to a thread to complete the task and its progress is tracked through attributes. These attributes report the queued commands along with their progress, status and results. The client is provided with a unique identifier which can be used to track the execution of the LRC and take further action based on the outcome of that command. LRCs can be aborted safely using a custom TANGO command. We present the reference design and implementation of the Long Running Commands for the SKA Controls System. | |||
Slides TH1BCO04 [0.674 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TH1BCO04 | ||
About • | Received ※ 06 October 2023 — Revised ※ 24 October 2023 — Accepted ※ 20 December 2023 — Issued ※ 22 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
TH2AO06 | SKA Tango Operator | controls, device-server, network, software | 1155 |
|
|||
Funding: INAF The Square Kilometre Array (SKA) is an international effort to build two radio interferometers in South Africa and Australia, forming one Observatory monitored and controlled from global headquarters (GHQ) based in the United Kingdom at Jodrell Bank. The software for the monitoring and control system is developed based on the TANGO-controls framework, which provide a distributed architecture for driving software and hardware using CORBA distributed objects that represent devices that communicate with ZeroMQ events internally. This system runs in a containerised environment managed by Kubernetes (k8s). k8s provides primitive resource types for the abstract management of compute, network and storage, as well as a comprehensive set of APIs for customising all aspects of cluster behaviour. These capabilities are encapsulated in a framework (Operator SDK) which enables the creation of higher order resources types assembled out of the k8s primitives (\verb|Pods|, \verb|Services|, \verb|PersistentVolumes|), so that abstract resources can be managed as first class citizens within k8s. These methods of resource assembly and management have proven useful for reconciling some of the differences between the TANGO world and that of Cloud Native computing, where the use of Custom Resource Definitions (CRD) (i.e., Device Server and DatabaseDS) and a supporting Operator developed in the k8s framework has given rise to better usage of TANGO-controls in k8s. |
|||
Slides TH2AO06 [2.622 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-TH2AO06 | ||
About • | Received ※ 27 September 2023 — Revised ※ 24 October 2023 — Accepted ※ 14 December 2023 — Issued ※ 21 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THMBCMO01 | New Developements on HDB++, the High-performance Data Archiving for Tango Controls | database, controls, interface, extraction | 1190 |
|
|||
The Tango HDB++ project is a high performance event-driven archiving system which stores data with micro-second resolution timestamps. HDB++ supports many different backends, including MySQL/MariaDB, TimeScaleDB (a time-series PostgreSQL extension), and soon SQLite. Building on its flexible design, latest developments made supporting new backends even easier. HDB++ keeps improving with new features such as batch insertion and by becoming easier to install or setup in a testing environment, using ready to use docker images and striving to simplify all the steps of deployment. The HDB++ project is not only a data storage installation, but a full ecosystem to manage data, query it, and get the information needed. In this effort a lot of tools were developed to put a powerful backend to its proper use and be able to get the best out of the stored data. In this paper we will present as well the latest developments in data extraction, from low level libraries to web viewer integration such as grafana. Pointing out strategies in use in terms of data decimation, compression and others to help deliver data as fast as possible. | |||
Slides THMBCMO01 [0.926 MB] | |||
Poster THMBCMO01 [0.726 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THMBCMO01 | ||
About • | Received ※ 05 October 2023 — Revised ※ 24 October 2023 — Accepted ※ 08 December 2023 — Issued ※ 16 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THMBCMO14 | Development of the SKA Control System, Progress, and Challenges | controls, software, interface, operation | 1221 |
|
|||
The SKA Project is a science mega-project whose mission is to build an astronomical observatory that comprises two large radio-telescopes: the SKA-Low Telescope, located in the Inyarrimanha Ilgari Bundara, the CSIRO Murchison Radio-astronomy Observatory in Western Australia, with the observing range 50 to 350 MHz, and the SKA Mid Telescope, located in the Karoo Region, South Africa, with the observing range 350 MHz to 15 GHz. The SKA Global Headquarters is in the Jodrell Bank Observatory, near Manchester, UK. When completed, the SKA Telescopes will surpass existing radio-astronomical facilities not only in the scientific criteria such as sensitivity, angular resolution, and survey speed, but also in the number of receptors and the range of the observing and processing modes. The Observatory, and each of the Telescopes, will be delivered in stages, thus supporting incremental development of the collecting area, signal and data processing capacity, and the observing and processing modes. Unlike scientific capability, which, in some cases, may be delivered in the late releases, the control system is required from the very beginning to support integration and verification. Development of the control system to support the first delivery of the Telescopes (Array Assembly 0.5) is well under way. This paper describes the SKA approach to the development of the Telescope Control System, and discusses opportunities and challenges resulting from the distributed development and staged approach to the Telescope construction. | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THMBCMO14 | ||
About • | Received ※ 06 October 2023 — Revised ※ 12 October 2023 — Accepted ※ 12 December 2023 — Issued ※ 22 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THMBCMO15 | Conan for Building C++ Tango Devices at SOLEIL | software, factory, Windows, Linux | 1227 |
|
|||
At SOLEIL, our Tango devices are mainly developed in C++, with around 450 projects for building libraries and device servers for our accelerators and beamlines. We have a software factory that has enabled us to achieve continuous integration of our developments using Maven, which manages project dependencies. However, Maven is uncommon for C++. In addition, it has limitations that hinder us from supporting future platforms and new programming standards, leading us to replace it with Conan. Conan is a dependency and package manager for C and C++ that works on all platforms and integrates with various build systems. Its features are designed to enable modern continuous integration workflows with C++ and are an ideal alternative to Maven for our C++ build system. This transition is essential for the upgrade of SOLEIL (SOLEIL II*), as we continue to develop new devices and update existing systems. We are confident that Conan will improve our development process and benefit our users. This paper will provide an overview of the integration process and describe the progress of deploying the new build system. We will share our insights and lessons learned throughout the transition process.
*SOLEIL II: Towards A Major Transformation of the Facility. Conan - C and C++ Open-Source Package Manager |
|||
Slides THMBCMO15 [0.824 MB] | |||
Poster THMBCMO15 [0.867 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THMBCMO15 | ||
About • | Received ※ 04 October 2023 — Revised ※ 10 October 2023 — Accepted ※ 13 December 2023 — Issued ※ 16 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THPDP001 | New Generation Qt Control Components for Hi Level Software | controls, storage-ring, EPICS, framework | 1291 |
|
|||
A new generation of Qt graphical components, namely cumbia-qtcontrols-ng is under development at ELETTRA. A common engine allows each component to be rendered on traditional QWidgets and scalable QGraphicsItems alike. The latter technology makes it possible to integrate live controls with static SVG in order to realize any kind of synoptic with touch and scaling capabilities. A pluggable zoomer can be installed on any widget or graphics item. Apply numeric controls, Cartesian and Circular (Radar) plots are the first components realized. | |||
Poster THPDP001 [0.935 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THPDP001 | ||
About • | Received ※ 29 September 2023 — Revised ※ 14 November 2023 — Accepted ※ 20 December 2023 — Issued ※ 20 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THPDP007 | Rolling Out a New Platform for Information System Architecture at SOLEIL | MMI, operation, database, software | 1301 |
|
|||
SOLEIL Information System is a 20-year legacy with multiple software and IT solutions following constantly evolving business requirements. Lots of non-uniform and siloed information systems have been experienced increasing the IT complexity. The future of SOLEIL (SOLEIL II*) will be based on a new architecture embracing native support for continuous digital transformation and will enhance user experience. Redesigning an information system given synchrotron-based science challenges requires a homogeneous and flexible approach. A new organizational setup is starting with the implementation of a transversal architectural committee. Its missions will be to set the foundation of architecture design principles and to foster all projects’ teams to apply them. The committee will support the building of architectural specifications and will drive all architecture gate reviews. Interoperability is a key pillar for SOLEIL II. Therefore, a synchronous and asynchronous inter-processes communications is being built as a platform to connect existing systems and future ones; it is based both on an event broker and an API manager. An implementation has been developed to interconnect our existing operational tools (CMMS** and our ITSM*** portal). Our current use case is a brand new application dedicated to samples’ lifecycle interconnected with various existing business applications. This paper will detail our holistic approach for addressing the future evolution of our information system, made mandatory given the new requirements from SOLEIL II.
* SOLEIL II: Towards A Major Transformation of the Facility ** CMMS: Computerized Maintenance Management System *** ITSM: Information Technology Service Management |
|||
Poster THPDP007 [1.397 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THPDP007 | ||
About • | Received ※ 05 October 2023 — Revised ※ 25 October 2023 — Accepted ※ 13 December 2023 — Issued ※ 16 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THPDP010 | Update on the EBS Storage Ring Beam Dynamics Digital Twin | controls, optics, storage-ring, SRF | 1306 |
|
|||
The EBS storage ring control system is presently paired with an electron beam dynamics digital twin (the EBS control system simulator, EBSS*). The EBSS reproduces many of the beam dynamics related quantities relevant for machine operation. This digital twin is used for the preparation and debug of software to deploy for operation. The EBSS is presently working only for the main storage ring and it is not directly connected to the machine operation but works in parallel and on demand. We present here the steps taken towards an on-line continuous use of the EBSS to monitor the evolution of not directly observable parameters such as beam optics.
* Simone Liuzzo, et al. The ESRF-EBS Simulator: A Commissioning Booster. 18th ICALEPCS, Oct 2021, Shanghai, China. MOPV012 |
|||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THPDP010 | ||
About • | Received ※ 27 September 2023 — Revised ※ 25 October 2023 — Accepted ※ 10 December 2023 — Issued ※ 16 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THPDP040 | Control System of the ForMAX Beamline at the MAX IV Synchrotron | controls, detector, experiment, synchrotron | 1402 |
|
|||
This paper describes the design and implementation of the control system for the ForMAX beamline at the MAX IV synchrotron. MAX IV is a Swedish national laboratory that houses one of the brightest synchrotron light sources in the world. ForMAX is one of the beamlines at MAX IV and is funded by the Knut and Alice Wallenberg Foundation and Swedish industry via Treesearch. To meet the specific demands of ForMAX, a new control system was developed using the TANGO Controls and Sardana frameworks. Using these frameworks enables seamless integration of hardware and software, ensuring efficient and reliable beamline operation. The control system was designed to support a variety of experiments, including multiscale structural characterization from nanometer to millimeter length scales by combining full-field tomographic imaging, small- and wide-angle X-ray scattering (SWAXS), and scanning SWAXS imaging in a single instrument. The system allows for precise control of the beam position, energy, intensity, and sample position. Furthermore, the system provides real-time feedback on the status of the experiments, allowing for adjustments to be made quickly and efficiently. In conclusion, the design and implementation of the control system for the ForMAX beamline at the MAX IV synchrotron has resulted in a highly flexible and efficient experimental station. TANGO Controls and Sardana have allowed for seamless integration of hardware and software, enabling precise and reliable control of the beamline for a wide range of experiments. | |||
Poster THPDP040 [0.668 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THPDP040 | ||
About • | Received ※ 04 October 2023 — Accepted ※ 08 December 2023 — Issued ※ 12 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THPDP050 | Improving User Experience and Performance in Sardana and Taurus: A Status Report and Roadmap | controls, interface, software, SCADA | 1420 |
|
|||
Sardana Suite is an open-source scientific SCADA solution used in synchrotron light beamlines at ALBA, DESY, MAXIV and SOLARIS and in laser labs at MBI-Berlin. It is formed by Sardana and Taurus - both mature projects, driven by a community of users and developers for more than 10 years. Sardana provides a low level interface to the hardware, middle level abstractions and a sequence engine. Taurus is a library for developing graphical user interfaces. Sardana Suite uses client - server architecture and is built on top of TANGO. As a community, during the last few years, on one hand we were focusing on improving user experience, especially in terms of reliability and performance and on the other hand renewing the dependency stack. The system is now more stable, easier to debug and recover from a failure. An important effort was put in profiling and improving performance of Taurus applications startup. The codebase has been migrated to Python 3 and the plotting widgets were rewritten with pyqtgraph. This didn’t prevent us from delivering new features, like for example the long-awaited configuration tools and format based on YAML which is easy and intuitive to edit, browse, and track historical changes. Now we conclude this phase in the project’s lifetimes and are preparing for new challenging requirements in the area of continuous scans like higher data throughput and more complex synchronization configurations. Here we present the status report and the future roadmap. | |||
Poster THPDP050 [0.605 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THPDP050 | ||
About • | Received ※ 06 October 2023 — Revised ※ 26 October 2023 — Accepted ※ 13 December 2023 — Issued ※ 21 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THPDP077 | Tango Integration of the SKA-Low Power and Signal Distribution System | controls, hardware, software, monitoring | 1526 |
|
|||
Funding: Square Kilometre Array Observatory The Power and Signal Distribution System (PaSD) is a key component of the SKA-Low telescope, responsible for control and monitoring of local power to the electronic components of the RF signal chain for the antennas, and collecting the RF signals for transmission to the Central Processing Facility. The system comprises "SMART boxes" (SMART: Small Modular Aggregation and RFoF Trunk) which each connect directly to around 10 antennas to provide local monitoring and control, and one Field Node Distribution Hub (FNDH) per station which distributes power to all the SMART boxes and provides a communications gateway as well as additional local monitoring. All communication to the SMART boxes is funnelled through the FNDH on a multi-drop serial bus using the Modbus ASCII protocol. This paper will describe how the PaSD will be integrated into the Tango-based SKA-Low Monitoring Control and Calibration Subsystem (MCCS) software, including the facility for a drop-in Python simulator which can be used to test the software. |
|||
Poster THPDP077 [20.237 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THPDP077 | ||
About • | Received ※ 04 October 2023 — Accepted ※ 08 December 2023 — Issued ※ 14 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THPDP079 | Integration of Bespoke Daq Software with Tango Controls in the SKAO Software Framework: From Problems to Progress | controls, GPU, data-acquisition, software | 1533 |
|
|||
The Square Kilometre Array Observatory (SKAO) project is an international effort to build two radio interferometers in South Africa and Australia to form one Observatory monitored and controlled from the global headquarters in the United Kingdom at Jodrell Bank. The Monitoring, Control and Calibration System (MCCS) is the "front-end" management software for the Low telescope which provides monitoring and control capabilities as well as implementing calibration processes and providing complex diagnostics support. Once completed the Low telescope will boast over 130, 000 individual log-periodic antennas and so the scale of the data generated will be huge. It is estimated that an average of 8 terabits per second of data will be transferred from the SKAO telescopes in both countries to Central Processing Facilities (CPFs) located at the telescope sites. In order to keep pace with this magnitude of data production an equally impressive data acquisition (DAQ) system is required. This paper outlines the challenges encountered and solutions adopted whilst incorporating a bespoke DAQ library within the SKAO’s Kubernetes-Tango ecosystem in the MCCS subsystem in order to allow high speed data capture whilst maintaining a consistent deployment experience. | |||
Poster THPDP079 [0.981 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THPDP079 | ||
About • | Received ※ 02 October 2023 — Accepted ※ 08 December 2023 — Issued ※ 19 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
THSDSC05 | The SKAO Engineering Data Archive: From Basic Design to Prototype Deployments in Kubernetes | software, controls, database, extraction | 1590 |
|
|||
During its construction and production life cycles, the Square Kilometre Array Observatory (SKAO) will generate non-scientific, i.e. engineering, data. The sources of the engineering data are either hardware devices or software programs that generate this data. Thanks to the Tango Controls software framework, the engineering data can be automatically stored in a relational database, which SKAP refers to as the Engineering Data Archive (EDA). Making the data in the EDA accessible and available to engineers and users in the observatory is as important as storing the data itself. Possible use cases for the data are verification of systems under test, performance evaluation of systems under test, predictive maintenance and general performance monitoring over time. Therefore we tried to build on the knowledge that other research facilities in the Tango Controls collaboration already gained, when they designed, implemented, deployed and ran their engineering data archives. SKAO implemented a prototype for its EDA, that leverages several open-source software packages, with Tango Controls’ HDB++, the Timescaledb time series database and Kubernetes at its core. In this overview we will answer the immediate question "But why do we not just do, what others are doing?" and explain the reasoning behind our choices in the design and in the implementation. | |||
Poster THSDSC05 [3.062 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THSDSC05 | ||
About • | Received ※ 05 October 2023 — Revised ※ 27 October 2023 — Accepted ※ 05 December 2023 — Issued ※ 11 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
FR2BCO02 | A Lean UX Approach for Developing Effective Monitoring and Control User Interfaces: A Case Study for the SKA CSP. LMC Subsystem | controls, software, monitoring, interface | 1650 |
|
|||
The Central Signal Processor Local Monitor and Control (CSP. LMC) is a software component that allows the flow of information and commands between the Telescope Manager (TM) and the subsystems dedicated to signal processing, namely the correlator and beamformer, the pulsar search and the pulsar timing engines. It acts as an adapter by specialising the commands and associated data from the TM to the subsystems and by exposing the subsystems as a unified entity while monitoring their status. In this paper, we approach the problem of creating a User Interface (UI) for such a component. Through a series of short learning cycles, we want to explore different ways of looking at the system and build an initial set of UIs that can be refined to be used as engineering UIs in the first Array Assembly of the Square Kilometre Array. The process heavily involves some of the developers of the CSP. LMC in creating the dashboards, and other ones as participants in informal evaluations. In fact, the opportunities offered by Taranta, a tool to develop web UIs without needing web-development skills, make it possible to quickly realise a working dashboard that can be promptly tested. This also supports the short feedback cycle advocated by a Lean UX approach and maps well in a bi-weekly sprint cadence. In this paper, we will describe the method and present the results highlighting strengths and pain points where faced. | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-FR2BCO02 | ||
About • | Received ※ 06 October 2023 — Revised ※ 20 November 2023 — Accepted ※ 05 December 2023 — Issued ※ 13 December 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||
FR2BCO03 | Taranta Project - Update and Current Status | controls, database, factory, experiment | 1657 |
|
|||
Taranta, developed jointly by MAX IV Laboratory and SKA Observatory, is a web based no-code interface for remote control of instruments at accelerators and other scientific facilities. It has seen a great success in system development and scientific experiment usage. In the past two years, the panel of users has greatly expanded. The first generation of Taranta was not able to handle the challenges introduced by the user cases, notably the decreased performance when a high number of data points are requested, as well as new functionality requests. Therefore, a series of refactoring and performance improvements of Taranta are ongoing, to prepare it for handling large data transmission between Taranta and multiple sources of information, and to provide more possibilities for users to develop their own dashboards. This article presents the status of the Taranta project from the aspects of widgets updates, packages management, optimization of the communication with the backend TangoGQL, as well as the investigation on a new python library compatible with the newest python version for TangoGQL. In addition to the technical improvements, more facilities other than MAX IV and SKAO are considering to join Taranta project. One workshop has been successfully held and there will be more in the future. This article also presents the lesson learned from this project, the road map, and the GUI strategy for the near future. | |||
Slides FR2BCO03 [4.759 MB] | |||
DOI • | reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-FR2BCO03 | ||
About • | Received ※ 06 October 2023 — Accepted ※ 21 November 2023 — Issued ※ 23 November 2023 | ||
Cite • | reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml) | ||