Abstract
Training and education programs often employ criterion-referenced evaluation, where individuals must perform tasks to a defined standard under specified conditions. However, in domains requiring high-performance differentiation, selection and career progression also depend on norm-referenced comparisons, where individuals are evaluated relative to their current and past peers' achievements.
This paper examines a method for continuously normalizing scores, enabling comparison in a dynamic environment where assessment standards, grading criteria, and course structures evolve. A statistical and artificial intelligence (AI)-driven model was developed to enable normalization of scores while adapting to course changes over a multi-year period. The approach estimates the similarity of assessment components drawn from different time frames to prevent comparisons of dissimilar data, ensuring that only compatible measures are analyzed. Additionally, it extrapolates missing data caused by course changes to mitigate the impact of the changes on rolling norm groups, preserving the integrity of comparative evaluation.
Through analysis of both collected assessment data and statistical modeling of hypothetical cases, results demonstrate that this method effectively enables comparisons of individuals to a norm group despite curriculum adjustments. The empirical results also show the model is robust to some sources of potential discrepancies such as differences in the number and timing of assessments.
By improving the accuracy of norm-referenced comparisons, this approach can provide data-driven insights to support course syllabus adjustments, enhance individual development, and optimize allocation of resources for teaching and training. More accurate information for selection and evaluation processes can contribute to improved retention, performance optimization, and overall readiness in high-stakes operational environments.