Main Subcommittee Category: Policy, Standards, Management, and Acquisition

How would you label your submission?: Industry

Abstract:

The rising use of generative Artificial Intelligence (AI) via Large Language Models (LLMs) can pose significant security risks unintentionally by being too intuitive. Both industry and Government are currently considering or already using this technology; however, integrating LLMs into these environments can cause security violations by aggregating data into its model or reference archive.

Besides the traditional problems of data aggregation, LLMs present novel problems that amplify the risk of keeping a model at its intended classification level. As LLMs ingest data, they gradually contextualize information not explicitly linked within a dataset. With inhuman attention and speed, the AI can find relationships between intentionally obfuscated data while it learns. Thus, when prompted, it can reveal content that it should not know based upon its deductions. Over time, a model may be rendered useless for its intended purpose as its knowledge level leads to its classification designation being elevated.

Additional risk exists from using a model for reverse engineering efforts; for example, by feeding it prompts, an analyst could glean information by using the LLM's responses to find word associations similar to the auto-complete feature of a search engine. Another area of concern stems from CUI, which carries an inherent requirement of “lawful government purpose” to access the data. A user may become an unintended insider threat by querying an LLM that has processed CUI data with gaps in its safeguards.

In preparation for mass adoption of LLMs, the authors highlight the spillage and insider threats resulting from the abductive capabilities of LLMs. This paper includes a survey on the current state of the art by detailing concerns and possible mitigations related to the aggregation of information by LLMs. This effort is intended to provide clarity on LLM usage, and aid readers with developing initial security policies and strategies.

Keywords: POLICY

Additional Keywords not in the list above: Large Language Models, Data Aggregation, Generative AI, Spillage