IDEAS home Printed from https://ideas.repec.org/a/spr/joinma/v34y2023i8d10.1007_s10845-022-02021-z.html
   My bibliography  Save this article

A cyber-physical prototype system in augmented reality using RGB-D camera for CNC machining simulation

Author

Listed:
  • PengYu Wang

    (Nanjing University of Aeronautics and Astronautics)

  • Wen-An Yang

    (Nanjing University of Aeronautics and Astronautics)

  • YouPeng You

    (Nanjing University of Aeronautics and Astronautics)

Abstract

Numerical control (NC) codes verification is an important issue in computer numerical control (CNC) machining simulation because wrong NC codes will lead to the workpiece scrap and collision. The NC code verification methods both in physical space and cyber space (such as 3D computer graphics environment) have been widely investigated in recent years. However, physical verification methods have the problems that the simulation takes time and improper operations may cause danger. On the other hand, cyber verification methods only support some types of machines and cannot reflect the actual conditions of machine tools. This study proposes a cyber-physical prototype system for NC codes verification and CNC machining simulation. Based on the RGB-D camera, the depth-to-stereo model is constructed to obtain the 3D information in images. Without connecting with the CNC controller, the cutting tool and workpiece coordinate system (WCS) movement information in physical space can be got from images captured by the RGB-D camera through a convolutional neural network (CNN). Workpiece size and NC codes are imported into cyber space to render virtual workpiece with augmented reality (AR) technology. So that the operator can directly see the virtual workpiece in the physical machining scene. The virtual workpiece is machined by the cyber-physical system according to cutting tool movement in physical space. This research further confirms the feasibility of using computer vision (CV) methods to build the cyber-physical CNC simulation system based on an RGB-D camera. The potential application of the system is to obtain simulation results from CNC machine tools (especially those that are forbidden to connect the controller) and transfer the machining results to the Internet of Things (IoT).

Suggested Citation

  • PengYu Wang & Wen-An Yang & YouPeng You, 2023. "A cyber-physical prototype system in augmented reality using RGB-D camera for CNC machining simulation," Journal of Intelligent Manufacturing, Springer, vol. 34(8), pages 3637-3658, December.
    • Handle: RePEc:spr:joinma:v:34:y:2023:i:8:d:10.1007_s10845-022-02021-z
    DOI: 10.1007/s10845-022-02021-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10845-022-02021-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10845-022-02021-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xin Tong & Qiang Liu & Shiwei Pi & Yao Xiao, 2020. "Real-time machining data application and service based on IMT digital twin," Journal of Intelligent Manufacturing, Springer, vol. 31(5), pages 1113-1132, June.
    2. A. J. H. Redelinghuys & A. H. Basson & K. Kruger, 2020. "A six-layer architecture for the digital twin: a manufacturing case study implementation," Journal of Intelligent Manufacturing, Springer, vol. 31(6), pages 1383-1402, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ahmed Ktari & Mohamed El Mansori, 2022. "Digital twin of functional gating system in 3D printed molds for sand casting using a neural network," Journal of Intelligent Manufacturing, Springer, vol. 33(3), pages 897-909, March.
    2. Ayman AboElHassan & Soumaya Yacout, 2023. "A digital shadow framework using distributed system concepts," Journal of Intelligent Manufacturing, Springer, vol. 34(8), pages 3579-3598, December.
    3. Chi Ma & Hongquan Gui & Jialan Liu, 2023. "Self learning-empowered thermal error control method of precision machine tools based on digital twin," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 695-717, February.
    4. Aniket Nagargoje & Pavan Kumar Kankar & Prashant Kumar Jain & Puneet Tandon, 2023. "Application of artificial intelligence techniques in incremental forming: a state-of-the-art review," Journal of Intelligent Manufacturing, Springer, vol. 34(3), pages 985-1002, March.
    5. Hassan Alimam & Giovanni Mazzuto & Marco Ortenzi & Filippo Emanuele Ciarapica & Maurizio Bevilacqua, 2023. "Intelligent Retrofitting Paradigm for Conventional Machines towards the Digital Triplet Hierarchy," Sustainability, MDPI, vol. 15(2), pages 1-30, January.
    6. Zander, Bennet & Lange, Kerstin & Haasis, Hans-Dietrich, 2021. "Designing the data supply chain of a smart construction factory," Chapters from the Proceedings of the Hamburg International Conference of Logistics (HICL), in: Kersten, Wolfgang & Ringle, Christian M. & Blecker, Thorsten (ed.), Adapting to the Future: How Digitalization Shapes Sustainable Logistics and Resilient Supply Chain Management. Proceedings of the Hamburg Internationa, volume 31, pages 41-62, Hamburg University of Technology (TUHH), Institute of Business Logistics and General Management.
    7. Zengya Zhao & Sibao Wang & Zehua Wang & Shilong Wang & Chi Ma & Bo Yang, 2022. "Surface roughness stabilization method based on digital twin-driven machining parameters self-adaption adjustment: a case study in five-axis machining," Journal of Intelligent Manufacturing, Springer, vol. 33(4), pages 943-952, April.
    8. Jyrki Savolainen & Michele Urbani, 2021. "Maintenance optimization for a multi-unit system with digital twin simulation," Journal of Intelligent Manufacturing, Springer, vol. 32(7), pages 1953-1973, October.
    9. Vivek Warke & Satish Kumar & Arunkumar Bongale & Ketan Kotecha, 2021. "Sustainable Development of Smart Manufacturing Driven by the Digital Twin Framework: A Statistical Analysis," Sustainability, MDPI, vol. 13(18), pages 1-49, September.
    10. Pan, Yanghua & Zhong, Ray Y. & Qu, Ting & Ding, Liqiang & Zhang, Jun, 2024. "Multi-level digital twin-driven kitting-synchronized optimization for production logistics system," International Journal of Production Economics, Elsevier, vol. 271(C).
    11. Saporiti, Nicolò & Cannas, Violetta Giada & Pozzi, Rossella & Rossi, Tommaso, 2023. "Challenges and countermeasures for digital twin implementation in manufacturing plants: A Delphi study," International Journal of Production Economics, Elsevier, vol. 261(C).
    12. Bettiol, Marco & Capestro, Mauro & Di Maria, Eleonora & Ganau, Roberto, 2024. "Is this time different?: how Industry 4.0 affects firms' labor productivity," LSE Research Online Documents on Economics 124545, London School of Economics and Political Science, LSE Library.
    13. Benjamin Lutz & Dominik Kisskalt & Andreas Mayr & Daniel Regulin & Matteo Pantano & Jörg Franke, 2021. "In-situ identification of material batches using machine learning for machining operations," Journal of Intelligent Manufacturing, Springer, vol. 32(5), pages 1485-1495, June.
    14. Marco Bettiol & Mauro Capestro & Eleonora Di Maria & Roberto Ganau, 2024. "Is this time different? How Industry 4.0 affects firms’ labor productivity," Small Business Economics, Springer, vol. 62(4), pages 1449-1467, April.
    15. Weifei Hu & Jinyi Shao & Qing Jiao & Chuxuan Wang & Jin Cheng & Zhenyu Liu & Jianrong Tan, 2023. "A new differentiable architecture search method for optimizing convolutional neural networks in the digital twin of intelligent robotic grasping," Journal of Intelligent Manufacturing, Springer, vol. 34(7), pages 2943-2961, October.
    16. Kaishu Xia & Thorsten Wuest & Ramy Harik, 2023. "Automated manufacturability analysis in smart manufacturing systems: a signature mapping method for product-centered digital twins," Journal of Intelligent Manufacturing, Springer, vol. 34(7), pages 3069-3090, October.
    17. Georgios Falekas & Athanasios Karlis, 2021. "Digital Twin in Electrical Machine Control and Predictive Maintenance: State-of-the-Art and Future Prospects," Energies, MDPI, vol. 14(18), pages 1-26, September.
    18. Nguyen, Tiep & Duong, Quang Huy & Nguyen, Truong Van & Zhu, You & Zhou, Li, 2022. "Knowledge mapping of digital twin and physical internet in Supply Chain Management: A systematic literature review," International Journal of Production Economics, Elsevier, vol. 244(C).
    19. Hazrathosseini, Arman & Moradi Afrapoli, Ali, 2023. "The advent of digital twins in surface mining: Its time has finally arrived," Resources Policy, Elsevier, vol. 80(C).
    20. Khairul Jauhari & Achmad Zaki Rahman & Mahfudz Huda & Achmad Widodo & Toni Prahasto, 2024. "Building digital-twin virtual machining for milling chatter detection based on VMD, synchro-squeeze wavelet, and pre-trained network CNNs with vibration signals," Journal of Intelligent Manufacturing, Springer, vol. 35(7), pages 3083-3114, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joinma:v:34:y:2023:i:8:d:10.1007_s10845-022-02021-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.