Individualized training solutions are increasingly important to military training programs. To effectively adapt training to each student requires valid and reliable measures of human performance. Historically, the military has relied on behavioral and subjective reporting methods to assess and evaluate human performance (e.g., speed, accuracy, reported workload); such measures are effective for simple, well-controlled tasks with a strong behavioral element, but they lack the diagnostic sensitivity required to measure meaningful differences in individuals' performance on complex tasks that stress cognitive performance. Brain-based measures of functional brain activity in naturalistic settings may lead to improved understanding of a trainee's progress. Specifically, functional Near-Infrared Spectroscopy (fNIRS) is an optical brain imaging technology that can be used in the context of simulation-based training to measure changes in brain activity related to executive cognitive functions. Previous work describes how these changes could be used to measure the transition from novice to expert performance by accurately and reliably assessing the transition of cognitive skills out of working memory into automaticity. In this paper, the authors describe how brain-based metrics derived from fNIRS and task performance data were developed for a realistic pilot flying task in a simulation environment. Results from this study demonstrate that fNIRS can be used to quantify meaningful differences as novice pilots learn to navigate a prescribed flight path. Behavioral performance data confirmed that navigational proficiency improved across trials, while the average oxygen concentrations declined in several areas of the prefrontal cortex, as hypothesized. Furthermore, there were statistically significant correlations between the neural and behavioral data., These results show that neurological data may provide a powerful complement to existing behavioral measures by allowing instructor pilots not only to observe trainee outward behavior, but also to gain a perspective of neurological changes occurring in the brain itself.