Modeling Air Combat Behavior for Simulation-Based Pilot Training: A Survey of Machine Learning Approaches

Fighter pilots train maneuvers and missions in simulators with and against simulated entities that must exhibit realistic behavior for effective training. However, current simulated entities often display simplified behavior, and traditional methods for improving realism require extensive manual effort to integrate domain knowledge. Recent advancements in machine learning, specifically reinforcement learning, imitation learning, and evolutionary algorithms, offer scalable alternatives by enabling agents to learn complex behavior from data. Despite improved methods, challenges remain in ensuring that agents reliably learn tactical maneuvers and proper employment of weapons and sensors in diverse scenarios. Another significant challenge is transferring agents from learning environments to military simulation systems because of differences in observation formats, action formats, and flight dynamic models. This survey categorizes the relevant literature in terms of applications, mission tasks, behavior model types, and machine learning methods, followed by a comparative analysis with an emphasis on the learning process. Based on this analysis, we present four primary recommendations: 1) standardization and research collaboration initiatives, 2) enhanced focus on multi-agent machine learning and cooperation, 3) utilization of hierarchical behavior models, and 4) emphasis on beyond-visual-range scenarios. These recommendations aim to address the current limitations and guide the development of more comprehensive, adaptable, and realistic machine learning-based behavior models for simulation-based pilot training.