Design and Implementation of Virtual Assembly System for Machine Tool Fixtures Based on Kinect and Unity3D
Main Article Content
Abstract
Traditional machine-tool fixture assembly training relies on heavy physical equipment, entails substantial purchase and maintenance costs, and limits repeated practice because of safety and space constraints. This study develops a low-cost virtual assembly system by integrating Kinect V2 somatosensory interaction with Unity3D. Kinect captures 25 human skeletal joints in real time, while Unity3D provides model rendering, physical simulation, and interactive scene management. A constrained dynamic time warping method is introduced to reduce the gesture-matching search region and reject invalid actions through a distortion threshold. Ray casting, collision detection, and multi-coordinate transformation are combined to support part selection, movement, rotation, alignment, assembly, and viewpoint control. Fixture models are preprocessed in SolidWorks and 3ds Max, assigned rigid-body and collider properties, and rendered with differentiated metallic materials. Experimental evaluation shows that the improved recognition method achieves 92.5% accuracy with a response delay below 78.2 ms, while increasing computational efficiency by more than 40% relative to conventional DTW. Compared with traditional physical training, the system increases assembly efficiency by 32.5% and reduces the operation-error rate by 46.8%. The results demonstrate the feasibility of using natural gestures for repeatable fixture-assembly teaching and provide a practical digital platform for engineering education, design verification, and pre-job training.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
D. Gu, J. Zhang, Z. Ma, et al., “Integration of virtual and real to reshape the classroom—Changchun University of Technology’s innovative teaching model,” CCTV, 2026-01-07.
Ministry of Education, “Notice on the collection of typical application scenarios of “artificial intelligence + higher education”,” (2025-12-10). [Online]. Available: http://www.moe.gov.cn.
Southeast University. CiviX3-LAB: "AI + Virtual +.
F. Peters, “Christiani´s digital twin teaching system,” Worlddidac, vol. 15, no. 2, pp. 34–38, 2024.
X. N. Lai, T. Y. Zhang, L. Wang, et al., “Fingertip interactive tracking registration method for AR assembly system,” Advances in Computational Intelligence, vol. 2, no. 2, pp. 1–12, 2022.
Scientific Reports, “Application of virtual assembly for complex mechanical structures based on digital twin technology,” Scientific Reports, vol. 15, p. 30306, 2025.
B. Schmidt, L. Wang, and D. Zhang, “Toward safe human robot collaboration by using multiple Kinects based real-time human tracking,” Assembly Automation, vol. 42, no. 3, pp. 289–298, 2022.
A. Zillner, “Variations of 3D real-time avatar creation for virtual office collaboration using Azure Kinect and HoloLens 2,” Munich: Technical University of Munich, 2026.
M. Zhang, H. Li, and W. Wang, “Unity realizes linkage between physical and virtual workshops,” Computer Simulation, vol. 41, no. 4, pp. 223–228, 2024.
J. Zhang and B. Zhang, “Dynamic gesture recognition method for human-computer interaction considering optical flow matching,” Modern Electronic Technology, vol. 48, no. 23, pp. 151–154, 2025.
H. Zhang, T. Wang, and G. Li, “Research on virtual assembly gesture recognition based on Azure Kinect and SVM,” Mechanical Design and Manufacturing, vol. 38, no. 2, pp. 112–118, 2024.
L. Wang, H. Chen, and Y. Liu, “Gesture scoring based on Gaussian distance-improved DTW,” Journal of Virtual Reality and Intelligent Hardware, vol. 6, no. 2, pp. 145–156, 2024.
Guangdong Mechanical and Electrical Vocational and Technical College, “Construction practice of collaborative innovation center for intelligent production line fixtures,” (2025-11-14). [Online]. Available: https://gdmec.edu.cn.
Z. Huang, S. Luo, B. Zhou, et al., “Few-shot neural differentiable simulat or: real-to-sim rigid-contact modeling,” arXiv, 2603.06218.
NetEase Yidun, “Description of coordinate system changes in spatial calculation positioning,” (2024-11-15). [Online]. Available: https://dongjian-web.netease.com.
G. Li and J. Wang, “Review of research on dynamic gesture recognition based on deep learning,” Journal of Computer Science, vol. 47, no. 5, pp. 1023–1048, 2024.
Y. Liu and W. Zhang, “Improvement and application of dynamic time warping algorithm,” Pattern Recognition and Artificial Intelligence, vol. 38, no. 2, pp. 156–169, 2025.
EasyAR, “EasyAR dense spatial map developer guide,” (2025-11-14). [Online]. Available: https://help.easyar.com.
M. Zach, “A volume-based penetration measure for 6DOF haptic rendering of streaming point clouds,” Bremen: University of Bremen, 2017.
J. Wang and L. Chen, “Research on the application of Unity3D physics engine in virtual assembly,” Journal of System Simulation, vol. 37, no. 4, pp. 892–905, 2025.
J. Tan and Z. Liu, “Intelligent manufacturing and digital twins: key technologies and development trends,” Chinese Journal of Mechanical Engineering, vol. 61, no. 1, pp. 1–18, 2025.