The data management and data exploitation issues for large-scale, distributed DoD simulations have striking parallels within a number of existing large-scale High Energy Physics (HEP) projects, in particular, the experiments associated with the Large Hadron Collider (LHC) in Geneva, Switzerland. The significant commonalities include: data rates of 10-100 GBytes/day, data distribution and database operations over very large scale, high-speed networks, and sophisticated data exploitation objectives. In this regard, the lessons learned over the past decade of preparations for LHC operations have obvious significance and relevance for operational (fielded) DoD information exploitation systems. The requirements for persistent, scheduled, secure data access and data mining within the HEP environment are similar to many aspects of future large-scale DoD simulation environments, such as the Sentient World Simulation (SWS).
This paper explores three particular areas of DoD data exploitation needs with significant parallels within existing HEP/LHC work. The first involves robust, scalable database design and management, such as the distributed simulation and data system within the Joint SemiAutomated Forces project now under development within the US Joint Forces Command. Important aspects here include operational transparency and efficiency from the perspective of a single user/analyst at a workstation. The second general area involves support for "user toolkits" - significant additional computational subsystems such as data-mining/knowledge-discovery procedures and "what if" Monte Carlo excursions that go well beyond straightforward queries of a distributed database. The final area has to do with "real-time" considerations, where this term is to be understood in the more general sense of legitimate, possibly urgent user needs that exceed available computational resources. Strategies are discussed for leveraging the demonstrated HEP expertise toward DoD data management and exploitation problems, using SWS, on occasion, as a template for some specific DoD requirements.