Abstract
For the Department of Defense (DoD) to actualize its $3.3 billion in ongoing investments in artificial intelligence (AI) and machine learning (ML), it is crucial to establish a quality assurance (QA) cycle that meets the high demands of this rapidly evolving field. Current QA processes rely heavily on manual input, making them costly and prone to human error. Continuous integration and continuous delivery (CI/CD) tools, which are commonly used in software development, offer a potential solution to automate QA tasks and accelerate deployment cycles. However, adapting CI/CD processes for AI/ML introduces unique challenges, such as bridging the gap between simulation environments and real-world deployment. To fully leverage CI/CD for AI/ML warfighting solutions, additional critical components, like modeling and simulation, must be considered when architecting the solution. Effectively overcoming this QA problem will be vital for the DoD to develop AI systems that are scalable, reliable, and capable of performing at the highest levels in mission-critical scenarios.
This paper details how to architect and implement a CI/CD solution following the AI software development life cycle (AI SDLC), with a focus on modeling and simulation to enhance testing coverage for AI software. We address current reliance on manpower over automation and show through a cognitive modeling technique, GOMS (Goals, Operators, Methods, and Selection rules), that this method enhances testing coverage while decreasing human computer interactions by up to 60%. Ultimately, results show an exponential decrease in human in the loop testing time, all while enhancing testing coverage using automated M&S testing processes. This compelling evidence demonstrates to the DoD the need for well architected and extensive CI/CD processes to support warfighters in rapidly changing environments, who rely on AI/ML systems.