BNL, Upton, New York, USA
The successful use of machine learning (ML) in particle accelerators has greatly expanded in recent years; however, the realities of operations often mean very limited machine availability for ML development, impeding its progress in many cases. This paper presents a framework for exploiting physics-based simulations, coupled with real machine data structure, to facilitate the investigation and implementation of reinforcement learning (RL) algorithms, using the longitudinal bunch-merge process in the Booster and Alternating Gradient Synchrotron (AGS) at Brookhaven National Laboratory (BNL) as examples. Here, an initial fake wall current monitor (WCM) signal is fed through a noisy physics-based model simulating the behavior of bunches in the accelerator under given RF parameters and external perturbations between WCM samples; the resulting output becomes the input for the RL algorithm and subsequent pass through the simulated ring, whose RF parameters have been modified by the RL algorithm. This process continues until an optimal policy for the RF bunch merge gymnastics has been learned for injecting bunches with the required intensity and emittance into the Relativistic Heavy Ion Collider (RHIC), according to the physics model. Robustness of the RL algorithm can be evaluated by introducing other drifts and noisy scenarios before the algorithm is deployed and final optimization occurs in the field.
