JACoW is a publisher in Geneva, Switzerland that publishes the proceedings of accelerator conferences held around the world by an international collaboration of editors.
TY - CONF AU - Nguyen, L.K. AU - Brown, K.A. AU - Costanzo, M.R. AU - Gao, Y. AU - Harvey, M. AU - Jamilkowski, J.P. AU - Morris, J. AU - Schoefer, V. ED - Schaa, Volker RW ED - Götz, Andy ED - Venter, Johan ED - White, Karen ED - Robichon, Marie ED - Rowland, Vivienne TI - A Physics-Based Simulator to Facilitate Reinforcement Learning in the RHIC Accelerator Complex J2 - Proc. of ICALEPCS2023, Cape Town, South Africa, 09-13 October 2023 CY - Cape Town, South Africa T2 - International Conference on Accelerator and Large Experimental Physics Control Systems T3 - 19 LA - english AB - The successful use of machine learning (ML) in particle accelerators has greatly expanded in recent years; however, the realities of operations often mean very limited machine availability for ML development, impeding its progress in many cases. This paper presents a framework for exploiting physics-based simulations, coupled with real machine data structure, to facilitate the investigation and implementation of reinforcement learning (RL) algorithms, using the longitudinal bunch-merge process in the Booster and Alternating Gradient Synchrotron (AGS) at Brookhaven National Laboratory (BNL) as examples. Here, an initial fake wall current monitor (WCM) signal is fed through a noisy physics-based model simulating the behavior of bunches in the accelerator under given RF parameters and external perturbations between WCM samples; the resulting output becomes the input for the RL algorithm and subsequent pass through the simulated ring, whose RF parameters have been modified by the RL algorithm. This process continues until an optimal policy for the RF bunch merge gymnastics has been learned for injecting bunches with the required intensity and emittance into the Relativistic Heavy Ion Collider (RHIC), according to the physics model. Robustness of the RL algorithm can be evaluated by introducing other drifts and noisy scenarios before the algorithm is deployed and final optimization occurs in the field. PB - JACoW Publishing CP - Geneva, Switzerland SP - 1630 EP - 1636 KW - cavity KW - controls KW - booster KW - simulation KW - diagnostics DA - 2024/02 PY - 2024 SN - 2226-0358 SN - 978-3-95450-238-7 DO - doi:10.18429/JACoW-ICALEPCS2023-FR2AO04 UR - https://jacow.org/icalepcs2023/papers/fr2ao04.pdf ER -