There is a need to develop a physics based 3D modeling and simulation (M&S) software to generate multi-modal datasets for machine learning of human activity detection and recognition, due to the high cost and difficulty in collecting of synchronized, multi-view human sensing data. This paper presents the effort in developing a novel, integrated, high-fidelity M&S tool: HumanView (HumV) of human signatures. Its key elements include: (a) HumV editor module, which allows users to view and manage available models and associated configurations using intuitive graphical user interface; (b)  HumV models module, which is  a data store containing human models, scene models of environment, and relevant electro-optical/infrared (EO/IR) sensor models; and (c) HumV simulator module, which allows users to simulate multiple scenarios for generation of synthetic sensor data and truth labels for analytics. HumV builds a pipeline that seamlessly integrates off-the-shelf free and open-source multi-physics M&S tools and material properties databases with the newly developed models and algorithms to address the multi-disciplinary M&S requirements. Specifically, we developed the Human Activity Replication Tool (HART) – a Blender 3D add-on to provide bio-fidelic M&S of clothed avatars that realistically represent the diversity of human shape, motion, and clothing characteristics. This is followed up by an innovative human thermal model that takes the scene and HART activity models to produce an output of temperature estimates for all the mesh facets of skin and clothing of the human avatars. The thermal dynamics considers the activity/heart rate, environmental radiance, and body/clothing interaction. Finally the various models of human activity, thermal dynamics, scene, materials, environment, sensor, and atmosphere are assembled into the Digital Imaging and Remote Sensing Image Generation (DIRSIG) tool to generate synthetic images or videos. A model validation has been conducted against the experimental data collected using commercial cameras at an outdoor setting.