Compositional Design of Multicomponent Alloys Using Reinforcement Learning

Abstract

The design of alloys has typically involved adaptive experimental synthesis and characterization guided by machine learning models fitted to available data. Often a bottleneck for sequential design, e.g., by Bayesian Global Optimization (BGO), be it for self-driven or manual synthesis, is that the search space becomes intractable as the number of alloy elements and its compositions exceeds a threshold. Here we overcome this limitation by performing compositional design of alloys using reinforcement learning (RL) within a closed loop with manual synthesis and characterization. Moreover, the training efficiency is increased by incorporating uncertainty within the reward. Existing alloy data is often limited, however, with pre-training the agent can access regions of higher reward values more frequently. In addition, the experimental feedback enables the agent to gradually explore new regions with higher rewards, compositionally different from the initial dataset. We demonstrate this strategy by designing a phase change multicomponent alloy (Ti27.2Ni47Hf13.8Zr12) with the highest transformation enthalpy (ΔH) -37.1 J/g within the TiNi-based family of alloys from a space of over 2×108 candidates, although the initial training is only on a compact dataset of 112 alloys. The approach directly applies to processing conditions where the actions would be performed in a given order.

Keywords: Compositional design, Reinforcement learning, Multicomponent alloys, Transformational enthalpy, phase change materials