CrystalSTAR: Structured Action Orchestration with Trio-Reflection for Constrained Novel Crystal Discovery

This paper studies the problem of constrained novel crystal discovery, which aims to identify a set of physical feasible and property-aligned material candidates within a complicated search space. Existing approaches typically integrate large language models (LLMs) with domain knowledge of material properties and constraints for material discovery. However, these approaches usually adopt shallow memory mechanisms or unstructured toolsets, leading to suboptimal performance. Towards this end, we propose a novel approach named Structured Action Orchestration with Trio-Reflection (CrystalSTAR) for constrained novel crystal discovery. The core of our CrystalSTAR is to incorporate hierarchical memories into a structured action orchestration from modification scopes and dominant components, facilitating a smooth and efficient crystal discovery procedure. In particular, our CrystalSTAR first formulates the toolset as a structured action space along two orthogonal dimensions, resulting in four complementary tools for material exploration. More importantly, we adopt a trio-reflection mechanism to guide the trial process in a hierarchical manner. Here, our bottom-up memory records local anomalies, stuck trajectories, and cross-run knowledge, enabling effective strategy planning with grounding evidence. Extensive experiments across multiple benchmark datasets validate the effectiveness of our proposed CrystalSTAR in comparison to extensive state-of-the-art approaches. Our codebase is open-sourced at https://anonymous.4open.science/r/CrystalSTAR-5524