CHEMISTRY
OIST Japan finds machine learning useful for quantum control
In the ordinary, everyday world, we can perform measurements with nearly unlimited precision. But, in the quantum world—the realm of atoms, electrons, photons, and other tiny particles—this becomes much harder. Every measurement made disturbs the object and results in measurement errors. Everything from the instruments used to the system’s properties might impact the outcome, which scientists call noise. Using noisy measurements to control quantum systems, particularly in real-time, is problematic. So, finding the means for accurate measurement-based control is essential for use in quantum technologies like powerful quantum supercomputers and devices for healthcare imaging. 
Now, an international group of researchers from the Quantum Machines Unit at the Okinawa Institute of Science and Technology Graduate University (OIST), Japan, and the University of Queensland, Australia, has shown, through simulations, that reinforcement learning, a type of machine learning, can be used to produce accurate quantum control even with noisy measurements. Their research was recently published in Physical Review Letters.
Dr. Sangkha Borah, Postdoctoral Scholar within the Unit and lead author of the paper, explained the idea using a simple example. “Imagine a ball on top of a hill. The ball can easily roll to the left or the right, but the aim is to keep it in the same place. To achieve this, one needs to see which way it is going to roll. If it is inclined to go to the left, the force needs to be applied on the right and vice versa. Now, imagine that a machine is applying that force, and, using reinforcement learning, the machine can be taught how much force to apply and when.”
Reinforcement learning is often used in robotics where a robot might learn to walk through a trial-and-error approach. But such applications within the realm of quantum physics are rare. Although the ball-atop-a-hill is a tangible example, the system that the researchers were simulating was on a much smaller scale. Instead of a ball, the object was a small particle moving in a double-well which Dr. Borah and his colleagues were trying to control using real-time measurements.
“The bottom of the two wells is called the quantum ground state,” said Dr. Bijita Sarma, Postdoctoral Scholar within the Unit and co-author of the paper. “That’s where we wanted the particle to eventually be located. For that, we need to perform measurements continuously to extract information about the particle’s state and depending on that, apply some force to push it to the ground state. However, the measurements typically used in quantum mechanics do not allow us to do that. Hence, we need to have a smarter way to control the system.”
Interestingly, when in the ground state, the particle will be in both wells simultaneously. This is called quantum superposition, and it’s a necessary state for the system to be in, given its importance in various quantum technologies. To detect the location (or locations) of the particle in the well, the machine agent is given the measurement records from continuous weak measurements in real-time that it uses as data points for learning. And because this used a reinforcement loop, any information that the machine learned from the system would be used to make its future measurements more accurate.
Adding to the complexity of this system was the fact that it is nonlinear, meaning that the change in its output was not related to the changes in its input. These systems are confusing and chaotic when compared to so-called linear systems. For such nonlinear systems, there is no standard method of quantum control, but this research has shown that with reinforcement learning, the machine can learn to control the quantum system completely autonomously.
"As we gradually move towards a future largely dominated by artificial intelligence, the time is ripe to explore the utility of artificial intelligence, such as machine learning, in solving some problems that cannot be solved by conventional means,” concluded Dr. Borah. “This is especially applicable to controlling particle dynamics at the quantum level, where everything is dramatically counterintuitive.”
Prof. Jason Twamley, who leads the OIST Unit, added: "For nonlinear systems, there is no known method of efficient feedback control. In this work, we have shown that reinforcement learning can indeed be effective for such control, which is amazing and futuristic.”
