A number of embedded training systems are currently in use or under development by the U.S. Navy. Consequently, there is a need to consolidate the experiences and findings of these embedded training development efforts, to evaluate the effectiveness of various high-level features of in-place embedded training systems and to assess the applicability of those features to future embedded training systems and sub-systems. This paper documents the lessons learned from the development and use of 15 operational training systems in the Navy. The choice of systems to be studied was based upon accessibility of the systems, their capabilities, general applicability, and technological and instructional complexity. The instructional features of the systems selected were identified and described and a taxonomy was created. Effectiveness evaluation criteria were developed and on site collection of data was accomplished by interviewing users of in place embedded training systems and administering a standardized assessment instrument. The analysis and evaluation of these training systems found few systems which could be considered true embedded training systems. Many systems made use of test target generators or data input devices which provide only rudimentary tools for training. Recommendations for the design of future embedded training systems are presented. These guidelines address the following areas of ET design: configuration, training characteristics, support, and policy.