One of the objectives of the Air Force Research Laboratory’s Gaming Research Integration for Learning Lab® (GRILL®) is to leverage commercial modeling and simulation technology to support secondary student education and engagement in STEM disciplines. As with any community-focused intervention, traditional evaluation approaches are not always well-suited for assessing outcomes. As such, researchers identified methodological opportunities to explore student and community impacts in novel ways. This paper outlines the developed evaluation methods utilizing a constellation of student survey and self-report data, alumni academic and career experiences, and teacher/mentor feedback regarding such topics as models of student achievement, test-based accountability systems, and curricular interventions.
The findings presented herein foster the understanding of the degree to which STEM outreach activities, such as the GRILL®, are influencing the formal education community and generating desirable outcomes for participants. The objective is therefore to refine the measures of progress for STEM initiatives by evaluating multi-dimensional data sources for students, mentors, and educators participating in the GRILL® program. Constructs including the following are explored: the extent to which outreach activities increase educator awareness and confidence in implementation of STEM curricula, types of positive student outcomes (e.g., access to STEM materials and the pursuit of STEM related careers), and student-requested STEM learning experiences. These identify investment priorities and focus areas concerned with the underlying structure of learning opportunities. This paper highlights current identified impacts and describes our continued efforts to increase the precision for gathering more direct and quantifiable data. These lessons-learned and future directions provide guidance for evaluating STEM exposure and its impact on the community, regional workforce, and educational opportunities in other similar initiatives and programs.