The Government's “Cloud Firstâ€� policy of 2011 set an accelerated course of government technology migration to cloud resources. The benefits of cloud services and infrastructure are appealing for use in simulation and training for many reasons, including the ability to provide point-of-need (PoN) simulation, freedom from hardware maintenance and upgrades, reduction of capital expenditure and hardware footprint, and practically limitless resources that allow ease of scalability. Evaluation of the fitness of visual system services for migration to the cloud as per the cloud-first guidance of readiness and value is highly dependent on the intended use case and architecture of a cloud-based simulator. While attractive in concept, serious limitations in training quality and effectiveness can exist depending on the implementation strategy of a cloud-based visual system. This paper explores the technical challenges and functional ramifications of distributing visual system components across the cloud compared to on-premises resources. Topics include latency, performance, distributed visual system architectures, latency tolerance of basic visual system components, and edge device computing. A wide spectrum of use cases exists within the simulation and training realm, and cloud-based visual systems must provide a flexible and adaptable hybrid cloud architecture to achieve required goals across very diverse training needs and physical infrastructure.