This virtual simulation provides an immersive learning experience designed to help users evaluate fire scenarios and take the appropriate corrective actions to stay safe while operating a fire extinguisher using the PASS technique (pull, aim, squeeze, sweep). The simulation structure consists of four key sections: "Introduction,""Navigation,""Interactable Objects," and "Practical Scenarios." This format facilitates a gradual acquisition of essential knowledge and skills, leading users to achieve key learning outcomes.
The digital agent introduces himself and provides an overview of the learning experience.
Integration of third-party libraries enhances the agent's animation, real-time lip synchronisation, eye movement, and voice-over capabilities.
Controllers highlight their buttons to guide users in performing specific actions such as teleportation, rotation and object manipulation.
Teleporting from one point to another is a standard method to avoid motion sickness, as it eliminates the need for continuous movement.
Follow-up user interface panels help support and complement the digital agent's instructions.
Manipulation and interaction with simple 3D objects help users prepare for handling more complex objects.
Virtual hands mimic real-life hand movements more closely, providing a more natural and engaging user experience.
A brief hands-on experience with an ABC fire extinguisher, one of the most common types, before engaging in various fire scenarios.
Showcasing a small fire and extinguishing agent in action. A UI gauge assists users in getting familiar with pressure levels.
Featuring a 3D movement detection wristwatch that triggers quick UI menu actions to streamline navigation in virtual environments.
Project Details
A virtual agent serves as a knowledgeable guide, leading users through the simulation, providing instructions, and offering support as they navigate the environment. This interactive feature enhances the learning experience, making it more engaging and effective. Users can teleport and interact with the interface and 3D objects, facilitating a hands-on learning experience. The virtual instructor ensures a consistent and uninterrupted learning journey. Additionally, virtual controllers highlight their buttons to guide users in performing specific actions. As users gain proficiency, the virtual controllers are replaced by virtual hands, increasing the feeling of being present in the virtual environment.
Development
This training simulation was developed using the Unity3D engine and HTC VIVE Pro technology. The virtual agent was created using Adobe Fuse CC, rigged using Mixamo and animated within the Unity environment using HTC VIVE equipment. The agent's voice was generated using Microsoft Text-to-Speech technology. Additionally, the 3D model of the extinguisher was meticulously crafted in Autodesk Maya, with detailed texturing applied using Adobe Subtance 3D Painter. Finally, Simplygon was used for texturing and mesh optimisation.
Team and Collaboration
The project involved a team of four members, including a subject matter expert, an instructional designer, and two developers. As the lead developer, I coordinated with team members to ensure seamless integration of all components and tasks, delivering optimal performance and milestone achievement.
Key Responsibilities
Team Coordination: Facilitated communication and collaboration within an agile team.
Coding Infrastructure: Developed scalable and reusable code architecture.
Documentation: Ensured comprehensive documentation of code, systems, and processes.
Mechanics: Developed interactive elements and user interfaces in C# within Unity3D.
Lighting Setup: Optimised performance through lighting setup and lightmap baking.
Locomotion System: Integrated teleportation mechanics for user movement.
Visual Effects: Managed visual effects using Unity's particle system (Shuriken).
Animation: Created, rigged and animated the virtual agent using Adobe Fuse CC, Mixamo and Unity.
3D Modelling: Designed and textured models using Autodesk Maya and Adobe Substance 3D Painter.
Optimisation: Addressed performance issues with Unity's profiling tools.
Challenges
Throughout the project, we encountered several challenges requiring thoughtful solutions. Initially, integrating third-party libraries for tasks like inverse kinematic animation, lip synchronisation, and eye movement posed some complications. We meticulously reviewed documentation, conducted iterative testing, and ensured seamless feature integration to overcome these. Designing an intuitive user interface for virtual controllers was another concern, requiring continuous refinement based on user feedback to enhance functionality and user experience. Additionally, balancing smooth performance with high visual fidelity was crucial. We achieved this by employing optimisation techniques such as texture and mesh simplification using Simplygon alongside Unity's profiling tools to pinpoint and resolve performance bottlenecks.
The VR simulation received positive feedback for its realistic and immersive training environment. Safety officers found the simulation invaluable as it provided a complementary tool to traditional training methods, allowing them to practice their skills in a controlled setting. The simulation was successfully integrated into the training curriculum, enabling officers to engage in various emergency scenarios without the risks associated with real-life drills.
