Virtual Reality Technology in Mining Machinery: Virtual Assembly, Virtual Planning and Virtual Monitoring

Description

This book focuses on the application of virtual reality (VR) technology in mining machinery. It gives a detailed introduction to the application of VR technology in virtual assembly, virtual planning, and virtual monitoring. Based on the theory of digital twin, VR technology and collaborative control technology are applied to coal mining machinery equipment, which lays a foundation for the digitalization and intellectualization of coal machinery equipment and broadens the application scope of virtual reality technology in the mechanical engineering field. Through the application of VR technology in coal machinery equipment, this book provides new methods and ideas for teaching activities, scientific research activities, and actual production with rich illustrations, related table introduction, unique research ideas, and other unique contents. This book could be a useful reference for researchers in mining machinery, simulation and modeling, computer-aided engineering (CAD and CAE) and design, visualization, mechanical engineering, and other disciplines. Preface Contents 1 Application Overview 1.1 Introduction 1.2 Research Developments in Virtual Reality Assembly Technology and Systems 1.2.1 Virtual Assembly 1.2.2 Coal Mine Scenario Simulation 1.2.3 Virtual Reality Interactive Tools 1.2.4 3D Modelling of Coal Mining Equipment 1.2.5 Web-Based Information Service Platform and Virtual Assembly Technology 1.2.6 Web-Based Digital Modelling and 3D Dynamic Design 1.2.7 Design of Public Service Platform for Mechanical Equipment 1.3 Research and Development of Monitoring and Dynamic Planning in Fully Mechanized Coal-Mining Face Under Virtual Reality Environment 1.3.1 The Monitoring Method in Fully Mechanized Coal Mining Face 1.3.2 VR Scene Simulation of “Three Machines” in Fully Mechanized Coal-Mining Face 1.3.3 VR Monitoring of “Three Machines” in Fully Mechanized Coal-Mining Face 1.3.4 VR Planning of “Three Machines” in Fully Mechanized Coal-Mining Face 1.4 The Structure and Content Arrangement of the Book References 2 System Overall Design 2.1 Introduction 2.2 Overall Model and Operation Process of Design and Operation Mode of Comprehensive Working Face Production System Based on Digital Twin 2.2.1 Overall Model 2.2.2 Operation Process 2.3 The First Stage: The Theory and Method of Production System Design of Fully Mechanized Working Face 2.3.1 Carry on the Design of Full Life Cycle of Fully Mechanized Mining Equipment Products 2.3.2 Coal Mine Enterprise Selection and Configuration Design 2.3.3 Case 2.4 The Second Stage: Virtual Fully Mechanized Mining Simulation Analysis 2.4.1 Development of Integrated Software for Element Layout and Virtual Simulation Operation of Fully Mechanized Working Face 2.4.2 New Method of Simulation Credibility Evaluation 2.4.3 Collaborative Intelligent Evolution Method for Modeling and Simulation of Unmanned Fully Mechanized Mining Equipment System 2.4.4 Simulation Technology of Real Coal Seam Operation Based on Digital Twin 2.4.5 Laboratory Cases 2.4.6 Technical Conception and Practice of “Coal Seam + Equipment” Joint Virtual Simulation Operation in Fully Mechanized Mining Face 2.4.7 Agent Based Three Machine Virtual Collaboration and Planning Method 2.5 The Third Stage: Online Virtual Monitoring and Control of Fully Mechanized Production System 2.5.1 Virtual Monitoring 2.5.2 Virtual Control 2.5.3 Monitoring System of Three Machines in Fully Mechanized Mining Face Working Condition in Fully Mechanized Working Face Under VR Environment 2.5.4 Cases 2.6 Digital Twin Development Trend of Production System in Fully Mechanized Working Face References 3 Virtual Simulation Element Model Transformation Technology and Model Base 3.1 Introduction 3.2 Framework Design 3.3 Technology of Structural Hierarchy 3.4 CAD Modeling Technology 3.5 CAD Model Transformation and Optimization Technology 3.6 CAD Model Restoration Technology 3.7 Search Technology of Google 3dwarehouse Resources 3.8 3DMAX Scene and Animation Technology 3.9 System Implementation References 4 Virtual Assembly Method Based on OSG 4.1 Introduction 4.2 Coal Machine Equipment Assembly Sequence and Path Planning Method 4.3 The Framework of the Combination of OSG and CEGUI 4.3.1 Combining OSG and CEGUI to Develop System Interface 4.3.2 System Scene Management 4.3.3 Scene Menu Content Design 4.4 Model Manipulation 4.4.1 Model Selection 4.4.2 Model Reset 4.5 Virtual Assembly and Disassembly Demonstration 4.6 Path Record and Playback 4.6.1 Path record 4.6.2 Path Playback 4.7 Automatic Positioning Constraints 4.8 Network Collaborative Assembly 4.8.1 Work Process 4.8.2 Network Cooperative Assembly Based on Windows Sockets 4.8.3 Specific Implementation Process 4.9 Stereoscopic Display 4.9.1 Details and Types 4.9.2 Method for Realizing Binocular Parallax Stereo Display 4.9.3 Stereoscopic Imaging Technology in OSG 4.10 Summary References 5 Virtual Assembly Method Based on UG 5.1 Introduction 5.2 Overall Design of Virtual Assembly System of Coal Shearer Based on UG 5.2.1 System Design Goals 5.2.2 System Overall Structure Design 5.2.3 System Development Environment Selection 5.2.4 System Function Design 5.3 Data Model Construction Method of Virtual Assembly System 5.3.1 Virtual Assembly Model Construction Method 5.3.2 Virtual Assembly Information Framework Based on XML 5.3.3 Assembly Information Data Model 5.3.4 Assembly Information Data File Based on XML Schema Standard 5.3.5 Construction of Assembly Information Model 5.3.6 Algorithm for Establishing Assembly Information Model 5.4 The Function Realization of the Virtual Assembly System of the Shearer Based on UG 5.4.1 Key Data Planning of Virtual Assembly Environment 5.4.2 Automatic Assembly 5.4.3 Assembly Sequence and Assembly Path Planning 5.4.4 Dynamic Simulation of Assembly Process 5.5 Development of Coal Shearer Virtual Assembly System Based on UG 5.5.1 System Application Framework 5.5.2 System Main Menu Design 5.5.3 Assembly Import Module Development 5.5.4 Automatic Assembly Module Development 5.5.5 Development of Assembly Planning Module References 6 Human–Machine Interaction Method of Virtual Assembly 6.1 Introduction 6.2 Mouse and Keyboard Human–Computer Interaction 6.2.1 Scene Interaction Based on Trackball 6.2.2 Mouse-Based Assembly Interaction 6.3 Virtual Hand Human–Computer Interaction Subsystem 6.3.1 Technical Route 6.3.2 Establishment of Virtual Hand Model 6.3.3 Establishing the Relationship Between Location Tracker and Data Glove 6.3.4 Virtual Hand Assembly Operation 6.3.5 Implementation of Assembly Interaction Based on Virtual Hands 6.4 Force Feedback Human–Computer Interaction Subsystem 6.4.1 Introduction to Phantom Desktop Equipment 6.4.2 Subsystem Frame Design 6.4.3 Model Import Technology 6.4.4 Force Rendering of Objects 6.4.5 Principle of Force Feedback Control Model 6.4.6 Automatic Positioning Constraints 6.4.7 Tactile and Visual Rendering Modes References 7 Virtual Assembly Network Method 7.1 Introduction 7.2 Subsystem Frame Design 7.2.1 Subsystem Hardware Design 7.2.2 Subsystem Software Design 7.2.3 Subsystem Structure Design 7.3 Virtual Reality Resource Library 7.4 ActiveX Control Technology 7.4.1 Writing OSG-ActiveX Controls 7.4.2 Server-Side Control Release 7.4.3 Client Environment Configuration 7.5 Basic Interface Design 7.6 Back-End Database Design 7.7 Public Service Edition 7.7.1 Choose Video Production Software 7.7.2 Select the Format to Be Played 7.7.3 Network Playback Code and Effect Test 7.7.4 Multi-view Playback 7.8 Summary 8 Method and Technology of Virtual Single Machine Simulation 8.1 Introduction 8.2 Establishment of Physical Information Sensing System 8.2.1 Sensor Layout of Shearer 8.2.2 Sensor Arrangement of Hydraulic Support 8.2.3 Sensor Arrangement of Scraper Conveyor 8.3 Single Machine Attitude Monitoring Method of Fully Mechanized Mining Equipment 8.3.1 Single Machine Attitude Monitoring Method of Fully Mechanized Mining Equipment 8.3.2 Attitude Monitoring Method of Scraper Conveyor 8.3.3 Attitude Monitoring Method of Hydraulic Support 8.4 Seamless Linkage Method of Hydraulic Support Components Based on Unity3D 8.4.1 Overall Idea of Virtual Simulation Method for Hydraulic Support 8.4.2 Model Construction and Repair 8.4.3 Seamless Linkage Method Between Virtual and Reality 8.4.4 Human-Computer Interaction Modes and Methods 8.5 Virtual Bending Technology of Scraper Conveyor Based on Unity3D 8.5.1 Model Construction and Repair 8.5.2 Virtual Bending Technology of Scraper Conveyor 8.6 Virtual Memory Cutting Method of Coal Shearer Based on Unity3D 8.6.1 Theories and Methods of Virtual Memory Cutting 8.6.2 Mathematical Model of Memory Cutting 8.6.3 Real-Time Virtual Shearer Drum Height Compensation Strategy 8.6.4 Virtual Controller 8.6.5 Virtual Interaction 8.6.6 Shearer Virtual Memory Cutting Interface 8.7 Summary 9 The Method and Technology of Virtual Collaborative Simulation Running 9.1 Introduction 9.2 Overall Research Framework 9.3 Establishment of Virtual Equipment and Coal Seam Model 9.3.1 Two Basic Methods of Coal Seam Modeling 9.3.2 Establishment of Virtual Inherent Coal Seam 9.3.3 Establishment of Virtual Real-Time Updating Coal Seam 9.3.4 Coal Seam Modeling Method with Bidirectional-Driving 9.4 Operation Simulation Method Among Equipment 9.4.1 Cooperation Between Shearer and Scraper Conveyor 9.4.2 Scraper Conveyor and Hydraulic Support Coordination 9.4.3 Collaboration of Shearer and Hydraulic Support Group 9.5 Establishment of Interaction Model Between Coal Seam and Equipment 9.5.1 Action Model of Coal Seam and Equipment 9.5.2 Hydraulic Support and Coal Seam Floor 9.5.3 Scraper Conveyor and Coal Seam Floor 9.5.4 Shearer Cutting Coal Seam 9.6 System Integration Technology Path 9.7 Conclusions References 10 A Virtual Reality Collaborative Planning Simulator Based on a Multi-agent System 10.1 Introduction 10.2 Framework of FMUnitySim 10.2.1 Overall Framework 10.2.2 Collaborative Mathematical Model of Three Machines 10.2.3 Collaborative Planning Model Based on an MAS 10.2.4 VR Planning Method 10.3 Collaborative Mathematical Model of the Three Machines 10.3.1 Coupling of the Shearer Haulage Speed and Scraper Conveyor Load 10.3.2 Coupling of the Shearer Haulage Speed and Adjustment of the Front Drum Height with the Underground Environment 10.3.3 Coupling of the Following Control of Hydraulic Supports and the Shearer Haulage Speed 10.3.4 Coupling of the Following Control of Hydraulic Supports with the Condition of the Roof and Floor 10.3.5 Coupling of the Shape of the Scraper Conveyor and Advancing Units of the Hydraulic Supports 10.3.6 Delay or Data Loss 10.4 Establishment of Agent 10.4.1 Model of the Shearer Agent 10.4.2 Model of the Scraper Conveyor Agent 10.4.3 Model of the Hydraulic Support Agent 10.4.4 Model of the Hydraulic System Agent 10.4.5 Model of the Underground Environment Agent 10.5 VR Planning Method (FMUnitySim) 10.5.1 3D Model of the Three Machines 10.5.2 Model of the Underground Environment 10.5.3 GUI Interface 10.6 Summary Reference 11 Sensor Information Architecture and Virtual Reality Interaction Technology 11.1 Introduction 11.2 Real-Time and Reliable Information Acquisition and “Virtual-Real Fusion” Channel Technology 11.2.1 Sensor Placement and Perceptual Information Acquisition 11.2.2 Key Technologies of Real-Time Interactive Channel Interface 11.2.3 Secondary Fusion and Correction of Sensing Information Data 11.2.4 Distributed Collaborative Driving Patterns 11.3 Presentation of Fusion of Virtual-Reality and Perceptual Consistency 11.3.1 Real-Time Driving Framework for Complex Fully-Mechanized Mining Virtual Scene 11.3.2 Key Technologies of Driving Virtual Stand-Alone Support Based on Underlying Model 11.4 Transparent Fully Mechanized Mining Face Industrial Internet Infrastructure Construction 11.4.1 Wired/Wireless Full Coverage Network Communication Platform 11.4.2 Digital Sensing Element 11.4.3 Remote Monitoring System 11.5 Development of Application Layer of Transparent Fully Mechanized Mining Face System 11.5.1 Two-Dimensional Data Monitoring 11.5.2 Video Monitoring 11.5.3 VR Integrated Attitude Monitoring and Early Warning System 11.5.4 Three-Dimensional Transparent Monitoring System Integration 11.6 System Test and Application 11.6.1 Digital Sensing Element Testing 11.6.2 Network Performance and Functional Testing 11.6.3 Visualized Coal Mining Experiment with Remote Intervention 11.6.4 Coordinated Operation Test of Three Engines 11.7 Conclusion References 12 Working Condition Monitoring and Virtual Simulation Method 12.1 Introduction 12.2 “Three Machine” Virtual Co-simulation Under the Condition of Horizontal Ideal Floor 12.2.1 Key Technologies of Shearer Virtual Walking 12.2.2 Mutual Perception Technology Between Shearer and Hydraulic Support 12.2.3 Mutual Perception Technology Between Hydraulic Supports 12.2.4 Mutual Perception Technology of Hydraulic Support and Scraper Conveyor 12.2.5 Mutual Perception Technology of Virtual Three Machines in Fully Mechanized Mining Face and Coal Mining Technology 12.2.6 Principle of Consistency in Time and Unit 12.3 Coupling Method of Cutter Attitude Between Shearer and Scraper Conveyor 12.3.1 Solving Calculation Process Model of Bending Section 12.3.2 Solution of Bending Section Chute Attitude 12.3.3 Update and Calculation of Shearer’s Walking Path 12.3.4 Calculation of Elongation Length of Hydraulic Support Push Cylinder 12.4 Joint Positioning and Attitude Determination Method of Shearer and Scraper Conveyor 12.4.1 Coupling Analysis of Positioning and Attitude Between Shearer and Scraper Conveyor in Lateral Single Tool Operation 12.4.2 Planning Software Development Based on Unity3D 12.4.3 Positioning and Attitude Determination Fusion Strategy Based on Information Fusion Technology 12.4.4 Reverse Mapping Labeling Strategy Based on a Priori Perspective 12.5 Method of Memorizing Posture Between Groups of Hydraulic Supports 12.5.1 Thought Source of Memory Posture of Hydraulic Support 12.5.2 Analysis of the Relationship Between the Support Height of Hydraulic Support and the Cutting Roof Track of Shearer 12.5.3 VR Monitoring Method of Memory Posture 12.6 Summary 13 Virtual Monitoring System 13.1 Introduction 13.2 Digital Twin Theory of Fully Mechanized Working Face Equipment 13.2.1 Introduction of Digital Twin Theory 13.2.2 Digital Twin Fully Mechanized Face Equipment 13.3 Overall Framework Design of VR + LAN “Three Machine” Condition Monitoring System 13.3.1 System Design Objectives 13.3.2 Hardware Design 13.3.3 Software Design 13.3.4 Real Time Sensing System 13.4 VR Monitoring Method Based on Unity3D 13.4.1 Reservation of State Variables in VR Environment 13.4.2 Real Time Data Reading and Access Method 13.4.3 Real Time Calculation Method of Underlying Mathematical Model 13.4.4 Real Time Rendering of Mining Environment Information 13.4.5 Fault Occurrence Screen Representation 13.4.6 Implementation of Real-Time Switching Video Monitoring Screen in VR Environment 13.5 Virtual Monitoring and Real Time Synchronization Method Based on LAN 13.5.1 Collaboration and Data Flow Based on RPC Technology 13.6 Real Time Coupling Strategy of Multi Software 13.6.1 Kingview + SQL Server 13.6.2 SQL SERVER + Unity3D 13.6.3 Matlab Calculation Result Processing 13.7 Prototype System Development 13.8 Summary 14 Summary and Conclusions 14.1 Work Summary 14.2 Main Conclusions