DOI: https://doi.org/10.32515/2664-262X.2024.10(41).1.60-67
Algorithm for Optimizing the Reliability of Operation and Efficiency of Use of Production Equipment Using Artificial Intelligence Methods
About the Authors
Viktor Aulin, Professor, Doctor in Technics (Doctor of Technic Sciences), Central Ukraіnian National Technical University, Kropyvnytskyi, Ukraine, e-mail: AulinVV@gmail.com, ORCID ID: 0000-0003-2737-120X
Serhiy Kovalov, Doctoral student, PhD in Pedagogicals (Candidate of Pedagogical Sciences), Central Ukraіnian National Technical University, Kropyvnytskyi, Ukraine, e-mail: kovalyovserggr@ukr.net, ORCID ID: 0009-0002-3922-8697
Andriy Hrynkiv, Senior Researcher, PhD in Technics (Candidate of Technics Sciences), Central Ukraіnian National Technical University, Kropyvnytskyi, Ukraine, e-mail: avgrinkiv@gmail.com, ORCID ID: 0000-0002-1888-6685
Valeriy Varvarov, Leading researcher співробітник, PhD in Technics (Candidate of Technics Sciences), Kharkiv National Ivan Kozhedub Air Force University, Kharkiv, Ukraine, e-mail: varvarov_@ukr.net, ORCID ID: 0000-0003-1273-5605
Abstract
This paper presents an innovative approach that involves the optimization of maintenance for production equipment by leveraging advanced artificial intelligence (AI) algorithms. The study explores the application of Markov process theory within the context of reinforcement learning and its integration into the modeling of production environments. The focus of this research is to address critical issues related to enhancing the reliability and efficiency of production processes. This is achieved by reducing maintenance costs and minimizing equipment downtime.
The proposed model for optimizing the use of the production equipment system is described as an environment with discrete states, where an agent operates with the capability to perform specific actions. This model emphasizes the implementation of the Q-learning algorithm, a form of reinforcement learning that aims to optimize production processes by enabling the agent to learn and make decisions that enhance system performance. Through the iterative process of Q-learning, the agent evaluates the potential benefits of various actions in different states, gradually refining its strategy to maximize long-term rewards.
Q-learning, with its ability to handle environments with unknown dynamics, is particularly effective in this context. It helps the agent to develop an optimal maintenance policy by balancing immediate maintenance costs against the long-term benefits of reduced downtime and extended equipment lifespan. The iterative nature of Q-learning ensures continuous improvement and adaptation, making the system robust against varying operational conditions and unforeseen disruptions.
Through rigorous analysis and experimentation, the findings demonstrate a significant improvement in the reliability and productivity of the production equipment system. The introduction of AI technologies, specifically through the Q-learning framework, not only streamlines maintenance practices but also ensures a more efficient utilization of resources, ultimately leading to a more robust and cost-effective production environment. The results of this study highlight the potential of AI in transforming traditional production maintenance strategies and setting new standards for operational excellence.
Keywords
productionsystem, productionequipment, artificialintelligence, Markovprocesses, reliabilityoffunctioning, efficiencyofuse
Full Text:
PDF
References
1. Neves, M., Vieira, M., & Neto, P. (2021). A study on a Q-Learning algorithm application to a manufacturing assembly problem. Journal of Manufacturing Systems, P. 426–440.
2. Aulin, V. V., Hrynkyv, A. V., Lysenko, S. V., & Holub, D. V. (2019). Synergetyka pidvyshchennia nadiinosti mashyn vykorystanniam modelei markivskykh protsesiv [Synergetics of increasing machine reliability using Markov process models]. Perspektyvy i tendentsii rozvytku konstruktsii ta tekhnichnoho servisu skh mashyn i znariad: zb. materialiv dop. uchasnykiv V Vseukrainskoi naukovo-praktychnoi konf [Perspectives and trends in the development of constructions and technical service of machines and tools: collection. additional materials participation V All-Ukrainian Scientific and Practical Conf] Zhytomyr: Zhytomyrskyi ahrotekhnichnyi koledzh, P. 242-245 (in Ukrainian).
3. Aulin, V. V., Hrynkyv, A. V., Holovaty, A. O., Lysenko, S. V., Holub, D. V., Kuzyk, O. V., & Tykhyi, A. A. (2020). Metodolohichni osnovy proektuvannia ta funktsionuvannia intelektualnykh transportnykh i vyrobnychykh system [Methodological principles of designing and functioning of intelligent transport and production system] (Prof. V.V.Aulin Ed) Kropyvnytskyi: Vydavets Lysenko V.F., 2020. 428 p.
4. Zhao, M., Lu, H., Yang, S., & Guo, F. (2020). The Experience-Memory Q-Learning Algorithm for Robot Path Planning in Unknown Environment. IEEE Access, 8. P. 47824–47844.
5. Palacio, J.C., Jiménez, Y.M., Schietgat, L., Van Doninck, B., & Nowé, A. (2022). A Q-Learning algorithm for flexible job shop scheduling in a real-world manufacturing scenario. Procedia CIRP, 106, P. 227–232.
6. Ha, D. (2019). Reinforcement learning for improving agent design. Artificial Life, 25(4), P. 352–365.
7. Han, R., Chen, K., & Tan, C. (2020). Curiosity-driven recommendation strategy for adaptive learning via deep reinforcement learning. British Journal of Mathematical and Statistical Psychology, 73(3), P. 522–540.
8. Sun, S. (2020). Inverse reinforcement learning-based time-dependent A* planner for human-aware robot navigation with local vision. Advanced Robotics, 34(13), P. 888–901
9. L. A. P., & Fu, M. C. (2022). Risk-Sensitive reinforcement learning via policy gradient search. Foundations and Trends® in Machine Learning, 15(5), P. 537–693.
10. He, S., et al. (2019). Reinforcement learning and adaptive optimization of a class of Markov jump systems with completely unknown dynamic information. Neural Computing and Applications, 32(18), P. 14311–14320.
11. Moore, B. L., et al. (2011). Reinforcement learning. Anesthesia & Analgesia, 112(2), P. 360–367.
12. Yan, Y., et al. (2022). Reinforcement learning for logistics and supply chain management: methodologies, state of the art, and future opportunities. Transportation Research Part E: Logistics and Transportation Review. 102712.
13. Wu, Y., et al. (2021). Dynamic handoff policy for RAN slicing by exploiting deep reinforcement learning. EURASIP Journal on Wireless Communications and Networking, 2021(1).
14. Jesus, J. C., et al. (2019). Deep deterministic policy gradient for navigation of mobile robots in simulated environments. 19th International Conference on Advanced Robotics (ICAR), Belo Horizonte, Brazil.
15. Kovalov, S. H., & Kovalov, Yu. H. (2024). Osoblyvosti realizatsiia modeli shtuchnoi neironnoi merezhi aparatnymy zasobamy [Features of implementing the model of artificial neural network with hardware means]. Nauka i tekhnika sohodni.., No6(34), 1131. (in Ukrainian).
Citations
1. Miguel Neves, Miguel Vieira, and Pedro Neto. “A study on a Q-Learning algorithm application to a manufacturing assembly problem”. In: Journal of Manufacturing Systems 59 (2021), P. 426–440.
2. Аулін В.В., Гриньків А.В., Лисенко С.В., Голуб Д.В. Синергетика підвищення надійності машин використанням моделей марківських процесів. Перспективи і тенденції розвитку конструкцій та технічного сервісу сх машин і знарядь: зб. матеріалів доп. учасн. V Всеукраїнської науково-практичної конф. Житомир: Житомирський агротехнічний коледж, 2019. С. 242-245.
3. Аулін В. В., Гриньків А. В., Головатий А. О., Лисенко С. В., Голуб Д. В., Кузик О.В., Тихий А. А. Методологічні основи проектування та функціонування інтелектуальних транспортних і виробничих систем: монографія під заг. ред. д.т.н., проф. Ауліна В.В. Кропивницький: Видавець Лисенко В.Ф., 2020. 428 с.
4. Zhao, M.; Lu, H.; Yang, S.; Guo, F. The Experience-Memory Q-Learning Algorithm for Robot Path Planning in Unknown Environment. IEEE Access 2020, 8. P. 47824–47844.
5. Palacio, J.C.; Jiménez, Y.M.; Schietgat, L.; Van Doninck, B.; Nowé, A. A Q-Learning algorithm for flexible job shop scheduling in a real-world manufacturing scenario. Procedia CIRP 2022, 106. Р. 227–232.
6. Ha D. Reinforcement learning for improving agent design. Artificial life. 2019. Т. 25, № 4. Р. 352–365. URL: https://doi.org/10.1162/artl_a_00301.
7. Han R., Chen K., Tan C. Curiosity‐driven recommendation strategy for adaptive learning via deep reinforcement learning. British journal of mathematical and statistical psychology. 2020. Т. 73, № 3. Р. 522–540. URL: https://doi.org/10.1111/bmsp.12199.
8. Inverse reinforcement learning-based time-dependent A* planner for human-aware robot navigation with local vision / S. Sun et al. Advanced robotics. 2020. Т. 34, № 13. Р. 888–901. URL: https://doi.org/10.1080/01691864.2020.1753569.
9. L. A. P., Fu M. C. Risk-Sensitive reinforcement learning via policy gradient search. Foundations and trends® in machine learning. 2022. Т. 15, № 5. Р. 537–693. URL: https://doi.org/10.1561/2200000091.
10. Reinforcement learning and adaptive optimization of a class of Markov jump systems with completely unknown dynamic information / S. He et al. Neural computing and applications. 2019. Т. 32, № 18. Р. 14311–14320.et al URL: https://doi.org/10.1007/s00521-019-04180-2.
11. Reinforcement learning / B. L. Moore et al Anesthesia & analgesia. 2011. Т. 112, № 2. Р. 360–367. URL: https://doi.org/10.1213/ane.0b013e31820334a7.
12. Reinforcement learning for logistics and supply chain management: methodologies, state of the art, and future opportunities / Y. Yan et al. Transportation research part E: logistics and transportation review. 2022. Vol. 162. 102712. URL: https://doi.org/10.1016/j.tre.2022.102712.
13. Dynamic handoff policy for RAN slicing by exploiting deep reinforcement learning / Y. Wu et al. EURASIP journal on wireless communications and networking. 2021. Т. 2021, № 1. URL: https://doi.org/10.1186/s13638-021- 01939-x.
14. Deep deterministic policy gradient for navigation of mobile robots in simulated environments / J. C. Jesus et al. 2019 19th international conference on advanced robotics (ICAR), м. Belo Horizonte, Brazil, 2–6 груд. 2019 р. 2019. URL: https://doi.org/10.1109/icar46387.2019.8981638 .
15. Ковальов С.Г. Ковальов Ю.Г. Особливості реалізація моделі штучної нейронної мережі апаратними засобами. «Наука і техніка сьогодні» (Серія «Педагогіка», Серія «Право», Серія «Економіка», Серія «Фізико-математичні науки», Серія «Техніка»)»: журнал. 2024. No6(34) 2024. 1131c. URL: DOI: https://doi.org/10.52058/2786-6025-2024-6(34)
Copyright (c) 2024 Victor Aulin, SerhiyKovalov, Andriy Hrynkiv, Valeriy Varvarov
Algorithm for Optimizing the Reliability of Operation and Efficiency of Use of Production Equipment Using Artificial Intelligence Methods
About the Authors
Viktor Aulin, Professor, Doctor in Technics (Doctor of Technic Sciences), Central Ukraіnian National Technical University, Kropyvnytskyi, Ukraine, e-mail: AulinVV@gmail.com, ORCID ID: 0000-0003-2737-120X
Serhiy Kovalov, Doctoral student, PhD in Pedagogicals (Candidate of Pedagogical Sciences), Central Ukraіnian National Technical University, Kropyvnytskyi, Ukraine, e-mail: kovalyovserggr@ukr.net, ORCID ID: 0009-0002-3922-8697
Andriy Hrynkiv, Senior Researcher, PhD in Technics (Candidate of Technics Sciences), Central Ukraіnian National Technical University, Kropyvnytskyi, Ukraine, e-mail: avgrinkiv@gmail.com, ORCID ID: 0000-0002-1888-6685
Valeriy Varvarov, Leading researcher співробітник, PhD in Technics (Candidate of Technics Sciences), Kharkiv National Ivan Kozhedub Air Force University, Kharkiv, Ukraine, e-mail: varvarov_@ukr.net, ORCID ID: 0000-0003-1273-5605
Abstract
Keywords
Full Text:
PDFReferences
1. Neves, M., Vieira, M., & Neto, P. (2021). A study on a Q-Learning algorithm application to a manufacturing assembly problem. Journal of Manufacturing Systems, P. 426–440.
2. Aulin, V. V., Hrynkyv, A. V., Lysenko, S. V., & Holub, D. V. (2019). Synergetyka pidvyshchennia nadiinosti mashyn vykorystanniam modelei markivskykh protsesiv [Synergetics of increasing machine reliability using Markov process models]. Perspektyvy i tendentsii rozvytku konstruktsii ta tekhnichnoho servisu skh mashyn i znariad: zb. materialiv dop. uchasnykiv V Vseukrainskoi naukovo-praktychnoi konf [Perspectives and trends in the development of constructions and technical service of machines and tools: collection. additional materials participation V All-Ukrainian Scientific and Practical Conf] Zhytomyr: Zhytomyrskyi ahrotekhnichnyi koledzh, P. 242-245 (in Ukrainian).
3. Aulin, V. V., Hrynkyv, A. V., Holovaty, A. O., Lysenko, S. V., Holub, D. V., Kuzyk, O. V., & Tykhyi, A. A. (2020). Metodolohichni osnovy proektuvannia ta funktsionuvannia intelektualnykh transportnykh i vyrobnychykh system [Methodological principles of designing and functioning of intelligent transport and production system] (Prof. V.V.Aulin Ed) Kropyvnytskyi: Vydavets Lysenko V.F., 2020. 428 p.
4. Zhao, M., Lu, H., Yang, S., & Guo, F. (2020). The Experience-Memory Q-Learning Algorithm for Robot Path Planning in Unknown Environment. IEEE Access, 8. P. 47824–47844.
5. Palacio, J.C., Jiménez, Y.M., Schietgat, L., Van Doninck, B., & Nowé, A. (2022). A Q-Learning algorithm for flexible job shop scheduling in a real-world manufacturing scenario. Procedia CIRP, 106, P. 227–232.
6. Ha, D. (2019). Reinforcement learning for improving agent design. Artificial Life, 25(4), P. 352–365.
7. Han, R., Chen, K., & Tan, C. (2020). Curiosity-driven recommendation strategy for adaptive learning via deep reinforcement learning. British Journal of Mathematical and Statistical Psychology, 73(3), P. 522–540.
8. Sun, S. (2020). Inverse reinforcement learning-based time-dependent A* planner for human-aware robot navigation with local vision. Advanced Robotics, 34(13), P. 888–901
9. L. A. P., & Fu, M. C. (2022). Risk-Sensitive reinforcement learning via policy gradient search. Foundations and Trends® in Machine Learning, 15(5), P. 537–693.
10. He, S., et al. (2019). Reinforcement learning and adaptive optimization of a class of Markov jump systems with completely unknown dynamic information. Neural Computing and Applications, 32(18), P. 14311–14320.
11. Moore, B. L., et al. (2011). Reinforcement learning. Anesthesia & Analgesia, 112(2), P. 360–367.
12. Yan, Y., et al. (2022). Reinforcement learning for logistics and supply chain management: methodologies, state of the art, and future opportunities. Transportation Research Part E: Logistics and Transportation Review. 102712.
13. Wu, Y., et al. (2021). Dynamic handoff policy for RAN slicing by exploiting deep reinforcement learning. EURASIP Journal on Wireless Communications and Networking, 2021(1).
14. Jesus, J. C., et al. (2019). Deep deterministic policy gradient for navigation of mobile robots in simulated environments. 19th International Conference on Advanced Robotics (ICAR), Belo Horizonte, Brazil.
15. Kovalov, S. H., & Kovalov, Yu. H. (2024). Osoblyvosti realizatsiia modeli shtuchnoi neironnoi merezhi aparatnymy zasobamy [Features of implementing the model of artificial neural network with hardware means]. Nauka i tekhnika sohodni.., No6(34), 1131. (in Ukrainian).
Citations
1. Miguel Neves, Miguel Vieira, and Pedro Neto. “A study on a Q-Learning algorithm application to a manufacturing assembly problem”. In: Journal of Manufacturing Systems 59 (2021), P. 426–440.
2. Аулін В.В., Гриньків А.В., Лисенко С.В., Голуб Д.В. Синергетика підвищення надійності машин використанням моделей марківських процесів. Перспективи і тенденції розвитку конструкцій та технічного сервісу сх машин і знарядь: зб. матеріалів доп. учасн. V Всеукраїнської науково-практичної конф. Житомир: Житомирський агротехнічний коледж, 2019. С. 242-245.
3. Аулін В. В., Гриньків А. В., Головатий А. О., Лисенко С. В., Голуб Д. В., Кузик О.В., Тихий А. А. Методологічні основи проектування та функціонування інтелектуальних транспортних і виробничих систем: монографія під заг. ред. д.т.н., проф. Ауліна В.В. Кропивницький: Видавець Лисенко В.Ф., 2020. 428 с.
4. Zhao, M.; Lu, H.; Yang, S.; Guo, F. The Experience-Memory Q-Learning Algorithm for Robot Path Planning in Unknown Environment. IEEE Access 2020, 8. P. 47824–47844.
5. Palacio, J.C.; Jiménez, Y.M.; Schietgat, L.; Van Doninck, B.; Nowé, A. A Q-Learning algorithm for flexible job shop scheduling in a real-world manufacturing scenario. Procedia CIRP 2022, 106. Р. 227–232.
6. Ha D. Reinforcement learning for improving agent design. Artificial life. 2019. Т. 25, № 4. Р. 352–365. URL: https://doi.org/10.1162/artl_a_00301.
7. Han R., Chen K., Tan C. Curiosity‐driven recommendation strategy for adaptive learning via deep reinforcement learning. British journal of mathematical and statistical psychology. 2020. Т. 73, № 3. Р. 522–540. URL: https://doi.org/10.1111/bmsp.12199.
8. Inverse reinforcement learning-based time-dependent A* planner for human-aware robot navigation with local vision / S. Sun et al. Advanced robotics. 2020. Т. 34, № 13. Р. 888–901. URL: https://doi.org/10.1080/01691864.2020.1753569.
9. L. A. P., Fu M. C. Risk-Sensitive reinforcement learning via policy gradient search. Foundations and trends® in machine learning. 2022. Т. 15, № 5. Р. 537–693. URL: https://doi.org/10.1561/2200000091.
10. Reinforcement learning and adaptive optimization of a class of Markov jump systems with completely unknown dynamic information / S. He et al. Neural computing and applications. 2019. Т. 32, № 18. Р. 14311–14320.et al URL: https://doi.org/10.1007/s00521-019-04180-2.
11. Reinforcement learning / B. L. Moore et al Anesthesia & analgesia. 2011. Т. 112, № 2. Р. 360–367. URL: https://doi.org/10.1213/ane.0b013e31820334a7.
12. Reinforcement learning for logistics and supply chain management: methodologies, state of the art, and future opportunities / Y. Yan et al. Transportation research part E: logistics and transportation review. 2022. Vol. 162. 102712. URL: https://doi.org/10.1016/j.tre.2022.102712.
13. Dynamic handoff policy for RAN slicing by exploiting deep reinforcement learning / Y. Wu et al. EURASIP journal on wireless communications and networking. 2021. Т. 2021, № 1. URL: https://doi.org/10.1186/s13638-021- 01939-x.
14. Deep deterministic policy gradient for navigation of mobile robots in simulated environments / J. C. Jesus et al. 2019 19th international conference on advanced robotics (ICAR), м. Belo Horizonte, Brazil, 2–6 груд. 2019 р. 2019. URL: https://doi.org/10.1109/icar46387.2019.8981638 .
15. Ковальов С.Г. Ковальов Ю.Г. Особливості реалізація моделі штучної нейронної мережі апаратними засобами. «Наука і техніка сьогодні» (Серія «Педагогіка», Серія «Право», Серія «Економіка», Серія «Фізико-математичні науки», Серія «Техніка»)»: журнал. 2024. No6(34) 2024. 1131c. URL: DOI: https://doi.org/10.52058/2786-6025-2024-6(34)