Producing effective leaders is a concern for training departments across the military, industry, and academia. The specific skill requirements for leaders across these domains varies, but effectively interacting with people is a requirement in any leadership role. Despite the broad utility of interpersonal leadership skills, methods available to systematically assess those skills are limited. Some organizations rely on self-report measures or situational judgment tests of leadership skills. Others may use performance measures gathered during observations of live assessments. The former set of methods is disadvantaged by social desirability bias and ability to identify criteria distorting participant responses. The latter set of methods is costly in time and human resources, and may suffer from observer subjectivity. The current research investigated another option for assessing interpersonal leadership skills: reactive, computer-based scenarios using unprompted, natural language responses as inputs. This method helps to mitigate the problems of self-report measures and may be widely used at a fraction of the costs associated with live assessments, but it faces two challenges. First, the assessment tool must be able to interpret natural language responses accurately. Second, virtual agent behaviors must be flexible enough to believably react to unguided inputs. In an experiment, US Army Officer Candidates interacted with virtual agents representing leaders, peers, and subordinates in three scenarios composed of 4 to 7 related vignettes. Free-text responses provided during real-time conversations with the agents influenced the outcomes of each scenario. Interactions with the agents were analyzed to determine if the assessment method could accurately detect differences in interpersonal leadership skills among Officer Candidates. Results of this research provided initial evidence that such differences can be detected using the experimental method. Further, results provided insights into the amount of training data needed for language libraries to accurately interpret unprompted inputs and for developing sufficiently flexible agents.