Realizing the considerable training potential of Distributed Interactive Simulation (DIS) technology will require training and performance evaluation methodologies. The objective of this paper is to identify key performance measurement issues in DIS-based training environments. The major premise is that, in DIS-based training systems, problems precluding reliable measurement are likely to be aggravated over those encountered in conventional simulation, presenting unusually complex measurement challenges. As a way to describe the inherent problems, a framework is described which a) identifies those factors which adversely affect measurement and feedback in conventional simulators, and b) describes how those factors tend to have greater impact in DIS environments. The identification of these issues is based on consideration of factors known to affect measurement in operational or simulated operational settings. They encompass task sampling, measurement procedures and the nature of trainee differences. Each of the factors is discussed and recommendations are provided for the reduction of their impact.