The use of Artificial Intelligence (AI) in high-consequence decision-making tasks presents legal, moral, and ethical challenges. Complicating the issue, AI systems have become more complex and are less explainable than deterministic or rules-based systems. AI agent behaviors are viewed as “black boxes”, as their decisions can seem arbitrary or opaque to users, and in some cases, even for the developers of the system. Additionally, algorithmic performance is not guaranteed as complexity and number of inputs grows so does the difficulty in covering all corner cases. This lack of trust has significant negative consequences for the adoption of AI in decision-making.
To remedy this, the trust gap must be addressed by adapting the systems of verification and validation (V&V) used for military technologies, particularly the Test and Evaluation (T&E) capabilities. This includes developing individual testable metrics from contract requirements. This requires breaking apart functionality into small testable elements, with objective figures of merit, to make incremental improvements.
To advance the state-of-the art in establishing trust in AI for complex aerospace systems, we propose an approach we call “Digital Neurology” that can provide a real-time in-situ, trained, observer “agent” that can monitor and evaluate AI-based Neural Networks (NN). Our paper will describe our concept for experimentation with autonomous aircraft. The observation agent would observe and infer behaviors that are indicative of:
- Normal and abnormal computational processing
- Anticipated and unexpected situations
- Regions of high and low computational use
- Regions of importance and insignificance
Similar to a human instructor following a human pilot through the training experience, our AI observer agent will be deployed with the autonomous aircraft systems, following and learning behavior to build trust. Our paper will include a scientific literature survey and will detail our scientific contributions associated with advancing trust in AI for autonomous aircraft.
Keywords
AI, BEHAVIOR MODELING, ENHANCING PERFORMANCE, VERIFICATION, VALIDATION AND ACCREDITATION (VV&A)
Additional Keywords
Neural Networks