Advanced AI-Based Real-Time Industrial Safety Sentinel for Smart Hazard Detection and Workplace Safety
Article Sidebar
Main Article Content
Abstract
Industrial environments — including factories, construction sites, warehouses, and chemical plants — continue to experience hazardous incidents due to PPE noncompliance, unauthorised zone entry, and unsafe proximity to workers. Simultaneously, web, IoT, and edge applications deployed in these environments remain vulnerable to well-documented cyber threats, including SQL Injection, XSS, and broken access control. This paper presents the Intelligent Integrated Cyber-Physical Safety and Security Framework (IICPSSF) [11], a novel hybrid edge-cloud AI system that uniquely and simultaneously enforces (i) real-time vision-based physical industrial safety monitoring, and (ii) adaptive cybersecurity design pattern enforcement — governed by a shared Unified LLMAssisted Natural Language Rule Engine (ULNLRE). The edge layer employs YOLO-E, an open-vocabulary object detection model, for promptable real-time perception at approximately 60 FPS. At the same time, a deterministic symbolic rule engine enforces auditable safety and security policies. A Pattern Knowledge Base and contextaware adaptive selection engine handle cybersecurity pattern recommendations for web, IoT, and edge applications. The system achieves 92.5% precision, 90.7% F1-score, and 99.5% specificity on physical safety violation detection across four violation types, and demonstrates effective coverage of six OWASP Top 10 vulnerability classes.
Downloads
Article Details
Section
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, NV, USA, 2016, pp. 779–788. DOI: http://doi.org/10.1109/CVPR.2016.91
J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Honolulu, HI, USA, 2017, pp. 7263–7271. DOI: http://doi.org/10.1109/CVPR.2017.690
A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection," arXiv preprint 2020. [Online]. Available: https://arxiv.org/abs/2004.10934
S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, Jun. 2017. DOI: http://doi.org/10.1109/TPAMI.2016.2577031
W. Liu et al., "SSD: Single Shot MultiBox Detector," in Proc. Eur. Conf. Comput. Vis. (ECCV), Amsterdam, Netherlands, 2016, pp. 21–37.
DOI: http://doi.org/10.1007/978-3-319-46448-0_2
K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, NV, USA, 2016, pp. 770–778. DOI: http://doi.org/10.1109/CVPR.2016.90
Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, "Object Detection in 20 Years: A Survey," Proc. IEEE, vol. 111, no. 3, pp. 257–276, Mar. 2023.
DOI: http://doi.org/10.1109/JPROC.2023.3238524
H. Wang, Z. Li, X. Ji, and Y. Wang, "Helmet Detection Based on Improved YOLO Algorithm," IEEE Access, vol. 8, pp. 133694–133703, 2020.
DOI: http://doi.org/10.1109/ACCESS.2020.3011223
S. Nath, P. Banerjee, and A. Chakrabarti, "Vision-Based PPE Detection Using Deep Learning," Int. J. Comput. Vis. Appl., vol. 12, no. 3, pp. 45–54, 2021. [Online]. Available: DOI: https://doi.org/10.17645/si.v9i4.4925
OWASP Foundation, "OWASP Top Ten Project," 2021. [Online]. Available: https://owasp.org/Top10
V. S. Chundawat, T. Sharma, R. Deshpande, U. Idhole, and P. A. Jadhav, "Intelligent Integrated Cyber-Physical Safety and Security Framework (IICPSSF)," Indian Patent Application, Bharati Vidyapeeth (Deemed to be University) College of Engineering, Pune, 2025.
S. Singh, A. Yadav, J. Jain, H. Shi, J. Johnson, and K. Desai, "Benchmarking Object Detectors with COCO: A New Path Forward," in Proc. Eur. Conf. Comput. Vis. (ECCV), Milan, Italy, 2024, vol. 15102, pp. 271–288. [Online]. Available: https://arxiv.org/abs/2403.18819
DOI: https://doi.org/10.1007/978-3-031-72784-9_16
International Labour Organization, "Safety and Health at Work: Global Trends and Challenges," ILO, Geneva, 2022. [Online]. Available:
https://www.ilo.org/global/topics/safety-and-health-at-work
W. Fang, P. E. D. Love, H. Luo, and L. Ding, "Computer Vision for Safety Management in the Construction Industry," Saf. Sci., vol. 135, p. 105130, 2021. DOI: http://doi.org/10.1016%20/j.ssci.2021.105130
G. L. Tortorella, R. Giglio, and R. van Dun, "Industry 4.0 Adoption as a Moderator of the Impact of Lean Production Practices on Operational Performance Improvement," Int. J. Oper. Prod. Manag., vol. 39, no. 6/7/8, pp. 860–886, 2019.
DOI: http://doi.org/10.1108/IJOPM-01-2019-0005
Y. Cherdantseva et al., "A Review of Cyber Security Risk Assessment Methods for SCADA Systems," Comput. Secur., vol. 56, pp. 1–27, 2016. DOI: http://doi.org/10.1016/j.cose.2015.09.009
X. Fang, H. Luo, Q. Zhou, and B. Li, "Automated Detection and Logging of Construction Site Safety Violations Using Deep Neural Networks," Saf. Sci., vol. 159, p. 106001, 2023. DOI: http://doi.org/10.1016%20/j.ssci.2022.106001
T. Cheng, "Safety Supervision and Management in the Construction Industry: Review," Int. J. Environ. Res. Public Health, vol. 19, no. 12, p. 7397, 2022. DOI: http://doi.org/10.3390/ijerph19127397
A. G. Howard et al., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications," arXiv preprint arXiv:1704.04861, 2017. [Online]. Available: https://arxiv.org/abs/1704.04861
C. Zhang, P. Patras, and H. Haddadi, "Deep Learning in Mobile and Wireless Networking: A Survey," IEEE Commun. Surv. Tutor, vol. 21, no. 3, pp. 2224–2287, 2019. DOI: http://doi.org/10.1109/COMST.2019.2904897
E. Altulaihan, A. Alismail, and M. Frikha, "A Survey on Web Application Penetration Testing," Electronics, vol. 12, no. 5, p. 1229, 2023. DOI: http://doi.org/10.3390/electronics12051229
T. Rashid, S. Arabzadeh, C. A. Peña-Solorzano, and P. Austin, "Automated Safety Compliance Monitoring in Construction Using Computer Vision: A Systematic Review," Autom. Constr., vol. 153, p. 104963, 2023. DOI: http://doi.org/10.1016/j.autcon.2023.104963
D. Ding, G. Tian, Z. Li, and X. Li, "IoT-Enabled Real-Time Safety Monitoring and Warning System for Construction Sites," Sensors, vol. 23, no. 9, p. 4392, 2023. DOI: http://doi.org/10.3390/s23094392
C. Feichtenhofer, H. Fan, J. Malik, and K. He, "SlowFast Networks for Video Recognition," in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Seoul, South Korea, 2019, pp. 6202–6211. DOI: http://doi.org/10.1109/ICCV.2019.00630
Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, "YOLOX: Exceeding YOLO Series in 2021," arXiv preprint arXiv:2107.08430, 2021. [Online]. Available: https://arxiv.org/abs/2107.08430
J. J. Losada del Olmo, Á. L. Perales Gómez, A. Ruiz, and P. E. López de Teruel, "A Few-Shot Learning Methodology for Improving Safety in Industrial Scenarios Through Universal Self-Supervised Visual Features and Dense Optical Flow," Appl. Soft Comput., vol. 165, p. 112094, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1568494624011499 DOI: https://doi.org/10.1016/j.asoc.2024.112094
M. A. R. Alif, "Enhancing Construction Site Safety: A Lightweight Convolutional Network for Effective Helmet Detection," arXiv preprint arXiv:2409.12669, 2024. [Online]. Available: https://arxiv.org/abs/2409.12669
H. Washizaki et al., "Systematic Literature Review of Security Pattern Research," Information, vol. 12, no. 1, p. 36, Jan. 2021. [Online]. Available: https://www.mdpi.com/2078-2489/12/1/36 DOI: https://doi.org/10.3390/info12010036
W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, "Edge Computing: Vision and Challenges," IEEE Internet Things J., vol. 3, no. 5, pp. 637–646, Oct. 2016. DOI: http://doi.org/10.1109/JIOT.2016.2579198
T. Cheng, L. Song, Y. Ge, W. Liu, X. Wang, and X. Zhu, "YOLO-World: Real-Time Open-Vocabulary Object Detection," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Seattle, WA, USA, 2024, pp. 16901–16911. DOI: http://doi.org/10.1109/CVPR52733.2024.01599
A. Fu, X. Zhang, N. Xiong, Y. Gao, and H. Wang, "VFL: A Verifiable Federated Learning with Privacy-Preserving for Big Data in Industrial IoT," IEEE Trans. Ind. Inform., vol. 17, no. 4, pp. 2849–2859, Apr. 2021. DOI: http://doi.org/10.1109/TII.2020.3005923
G. Jocher, A. Chaurasia, and J. Qiu, "Ultralytics YOLOv8," GitHub Repository, 2023. [Online]. Available:
https://github.com/ultralytics/ultralytics
T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, "Focal Loss for Dense Object Detection," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Venice, Italy, 2017, pp. 2980–2988. DOI: http://doi.org/10.1109/ICCV.2017.324
J. Dai, Y. Li, K. He, and J. Sun, "R-FCN: Object Detection via Region-Based Fully Convolutional Networks," in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), Barcelona, Spain, 2016, pp. 379–387. [Online]. Available: https://arxiv.org/abs/1605.06409
A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, "Simple Online and Realtime Tracking," in Proc. IEEE Int. Conf. Image Process. (ICIP), Phoenix, AZ, USA, 2016, pp. 3464–3468. DOI: http://doi.org/10.1109/ICIP.2016.7533003
N. Wojke, A. Bewley, and D. Paulus, "Simple Online and Realtime Tracking with a Deep Association Metric," in Proc. IEEE Int. Conf. Image Process. (ICIP), Beijing, China, 2017, pp. 3645–3649. DOI: http://doi.org/10.1109/ICIP.2017.8296962
F. Pereira, P. Sousa, A. Bessani, and N. F. Neves, "Resilient Security Patterns for Microservice Architectures," IEEE Trans. Netw. Serv. Manag., vol. 20, no. 3, pp. 3182–3196, Sep. 2023. DOI: http://doi.org/10.1109/TNSM.2023.3263407
E. B. Fernandez, N. Yoshioka, and H. Washizaki, "Abstract Security Patterns and the Design of Secure Systems," Cybersecurity, vol. 5, no. 1, p. 7, Apr. 2022. [Online]. DOI: https://doi.org/10.1186/s42400-022-00109-w Available: https://link.springer.com/article/10.1186/s42400-022-00109-w
M. Aydos, Ç. Aldan, E. Coşkun, and A. Soydan, "Security Testing of Web Applications: A Systematic Mapping of the Literature," J. King Saud Univ. Comput. Inf. Sci., vol. 34, no. 9, pp. 6775–6792, 2022. DOI: http://doi.org/10.1016/j.jksuci.2021.02.016
S. P. Maniraj, C. S. Ranganathan, and S. Sekar, "Securing Web Applications with OWASP ZAP for Comprehensive Security Testing," Int. J. Adv. Signal Image Sci., vol. 10, no. 2, pp. 12–23, 2024. [Online]. Available: DOI: https://doi.org/10.29284/ijasis.10.2.2024.12-2
D. Priyawati, S. Rokhmah, and I. C. Utomo, "Website Vulnerability Testing and Analysis of Internet Management Information System Using OWASP," Int. J. Comput. Inf. Syst. (IJCIS), vol. 03, no. 03, 2022. DOI: http://doi.org/10.29040/ijcis.v3i3.77
M. Noman, M. Iqbal, and A. Manzoor, "A Survey on Detection and Prevention of Web Vulnerabilities," Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 6, 2020. DOI: http://doi.org/10.14569/IJACSA.2020.0110659
H. Wang, M. Xu, Y. Guo, W. Han, H. W. Lim, and J. S. Dong, "RulePilot: An LLM-Powered Agent for Security Rule Generation," in Proc. IEEE/ACM Int. Conf. Softw. Eng. (ICSE), 2025. [Online]. Available: https://arxiv.org/abs/2503.07808
P. Fernández Saura, K. R. Jayaram, V. Isahagian, J. Bernal Bernabé, and A. Skarmeta, "On Automating Security Policies with Contemporary LLMs," arXiv preprint arXiv:2506.04838, 2025. [Online]. Available: https://arxiv.org/abs/2506.04838
P. Aghaei et al., "Executable Governance for AI: Translating Policies into Rules Using LLMs," in Proc. AAAI Conf. Artif. Intell., 2025. [Online]. Available: https://arxiv.org/abs/2501.13138
J. Judvaitis et al., "A Set of Tools and Data Management Framework for the IoT–Edge–Cloud Continuum," Sensors, vol. 24, no. 23, p. 7794, 2024. DOI: https://doi.org/10.3390/s24237794
S. Ramírez, "FastAPI", GitHub Repository, 2018. [Online]. Available: https://github.com/tiangolo/fastapi
Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, "Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression," in Proc. AAAI Conf. Artif. Intell., vol. 34, no. 7, pp. 12993–13000, 2020. DOI: https://doi.org/10.1609/aaai.v34i07.6999
V. Poggi, F. Tosi, K. Kim, F. Aleotti, D. De Gregorio, L. Di Stefano, J. Park, and S. Im, "On the Synergies Between Machine Learning and Stereo: A Survey," IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 4584–4601, 2023. DOI: http://doi.org/10.1109/TPAMI.2022.3212542
L. Luo et al., "Privacy-Preserving and Traceable Federated Learning for Data Sharing in Industrial IoT Applications," Expert Syst. Appl., vol. 213, p. 119036, 2023. DOI: http://doi.org/10.1016/j.eswa.2022.119036
P. Zanuttigh, G. Marin, C. Dal Mutto, F. Dominio, L. Minto, and G. M. Cortelazzo, Time-of-Flight and Structured Light Depth Cameras: Technology and Applications. Cham: Springer, 2016. DOI: http://doi.org/10.1007/978-3-319-30973-6
Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, "OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields," IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 1, pp. 172–186, Jan. 2021. DOI: http://doi.org/10.1109/TPAMI.2019.2929041
Q. Li, Z. Wen, and B. He, "Practical Federated Gradient Boosting Decision Trees," in Proc. AAAI Conf. Artif. Intell., New York, NY, USA, 2020, pp. 4642–4649. DOI: http://doi.org/10.1609/aaai.v34i04.5895
X. He, K. Zhao, and X. Chu, "AutoML: A Survey of the State-of-the-Art," Knowl.-Based Syst., vol. 212, p. 106622, 2021.
DOI: http://doi.org/10.1016/j.knosys.2020.106622
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, "Communication-Efficient Learning of Deep Networks from Decentralized Data," in Proc. Int. Conf. Artif. Intell. Stat. (AISTATS), Fort Lauderdale, FL, USA, 2017, pp. 1273–1282. [Online]. Available: https://arxiv.org/abs/1602.05629.