Image Retrieval Through Free-Form Query using Intelligent Text Processing
Article Sidebar
Main Article Content
Abstract
Image Retrieval is the process of retrieving images from the image/multimedia databases. Retrieval of images are carried out with various types of queries, free-form query is a text-query that consists of single or multiple keywords and/or concepts or descriptions of images with or without the inclusion of wild-card characters and/or punctuations. This work aims to handle image retrieval based on free-form text queries. Simple & complex queries of conceptual descriptions of images are explored and an intelligent processing system with free-form queries based on the Bag-of-Words model is modified and built for natural scene images and on Diverse Social Images using the Damerau-Levenshtein edit distance measure. The efficacy of the proposed system is evaluated by testing 1500 free-form text queries and has resulted in a recall accuracy of 91.3% on natural scene images (of Wang/Corel database) and 100% on Diverse Social Images (of DIV400 dataset). These results show that the system proposed has produced satisfactory performance compared to published results such as the harmonic mean of precision and recall (i.e. F1-Score) of 76.70% & 63.32% at retrieval of 20 images etc in reported works
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
Mafla, A., Tito, R., Dey, S., Gómez, L., Rusiñol, M., Valveny, E., & Karatzas, D. (2020),“Real-Time Lexicon-Free Scene Text Retrieval”, Pattern Recognition, 107656. Vol. 110, ISSN 0031-3203. doi:10.1016/j.patcog.2020.10765.
Bo Yuan and Xinbo Gao. (2019),“Diversified Textual based imge retrieval”, Neurocomputing, vol.357, pp 116-124.Sep 2019,doi:10.1016/j.neucom.2019.03.048.
Torjmen-Khemakhem, M., & Gasmi, K. (2019),“Document/query expansion based on selecting significant concepts for context based retrieval of medical images”, Journal of Biomedical Informatics, vol. 95, ISSN 1532-0464, 103210. doi:10.1016/j.jbi.2019.103210
Qimin Cheng, Qian Zhang, Peng Fu, Conghuan Tu &, Sen L. (2018), “A survey and analysis on automatic image annotation”, Pattern Recognition, Vol. 79, pp. 242-259. doi:10.1016/j.patcog.2018.02.017
Yilu Chen, Xiaojun Zeng, Xing Chen and Wenzhong Guo. (2020), “A survey on automatic image annotation”, Applied Intelligence, vol. 50, pp. 3412–3428. doi:10.1007/s10489-020-01696-2
Tamura, H. and Yokoya, N. (1984), “Image database systems: A survey”, Pattern Recognition, Vol.17, No.1, pp. 29–43, 1984. doi:10.1016/0031-3203(84)90033-5.
V. Vijayarajan, M. Dinakaran, Priyam Tejaswin and Mayank Lohani. (2016),“A generic framework for ontology‑based information retrieval and image retrieval in web data”, Human-centric Computing and Information Sciences, vol.6, no.1, pp. 1-30, 2016. DOI 10.1186/s13673-016-0074-1
P. M. Ashok Kumar, T. Subha Mastan Rao, L. Arun Raj, and E. Pugazhendi. (2021),“An Efficient Text-Based Image Retrieval Using Natural Language Processing (NLP) Techniques”, Intelligent System Design, Advances in Intelligent Systems and Computing, vol. 1171, https://doi.org/10.1007/978-981-15-5400-1_52
Pastra, K., Saggion, H., & Wilks, Y. (2003), “NLP for indexing and retrieval of captioned photographs”, Proceedings of the Tenth Conference on European Chapter of the Association for Computational Linguistics - EACL ’03. Vol.2, pp. 143-146. doi:10.3115/1067737.1067769
Sohail Sarwar, Zia Ul Qayyum, Saqib Majeed. (2013),“Ontology Based Image Retrieval Framework using Qualitative Semantic Image Descriptions”, Procedia Computer Science, Vol. 22,pp.285 – 294, 2013. doi: 10.1016/j.procs.2013.09.105
Hui Hui Wang, Dzulkifli Mohamad, N.A. Ismail. (2010),“Approaches, Challenges and Future Direction of Image Retrieval”, Journal of Computing, vol. 2, issue 6, June 2010, ISSN 2151-9617
Levenshtein, V. (1966),“Binary codes capable of correcting deletions, insertions, and reversals”, Soviet Physics-Doklady, Vol.10, No. 8, pp. 707-710, February 1966.
Damerau, F. (1964),“A technique for computer detection and correction of spelling errors”, Communications of the ACM, Volume 7, Issue 3, March 1964 pp. 171–176. https://doi.org/10.1145/363958.363994
Grzegorz Kondrak. (2005),“N-gram similarity and distance”, SPIRE'05: Proceedings of the 12th international conference on String Processing and Information Retrieval, November 2005,vol. 3772, Pages 115–126.https://doi.org/10.1007/11575832_13
Htet Htet Htun and Virach Sornlertlamvanich. (2017),“Concept Name Similarity Measure on SNOMED CT”, vol. 780, pp.76-90, DOI: 10.1007/978-981-10-6989-5_7
Mihai Lintean and Vasile Rus. (2012),“Measuring Semantic Similarity in Short Texts through Greedy Pairing and Word Semantics”, Proceedings of the Twenty-Fifth International Florida Artificial Intelligence Research Society Conference.
Farouk, M. (2019, “Measuring Sentences Similarity: A Survey”, Indian Journal of Science and Technology, vol.12, issue 25, pp. 1–11. doi:10.17485/ijst/2019/v12i25/143977
Thi Thuy Anh Nguyen and Stefan Conrad. (2015),“Ontology Matching using Multiple Similarity Measures”, In Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2015), Volume. 1 KDIR, pages 603-611.
Jin Zhang;Xiaohai He;Linbo Qing;Luping Liu;Xiaodong Luo. (2021). “Cross-modal multi-relationship aware reasoning for image-text matching”, Multimedia Tools and Applications, vol. 81, pp. 12005-12027. doi:10.1007/s11042-020-10466-8
Chunbin Gu, Jiajun Bu, Xixi Zhou, Chengwei Yao, Dongfang Ma, Zhi Yu, Xifeng Yan. (2022),“Cross-modal image retrieval with deep mutual information maximization”, Neurocomputing, Volume 496, 2022, Pages 166-177, ISSN 0925-2312. https://doi.org/10.1016/j.neucom.2022.01.078
Zafran Khan, Bushra Latif, Joonmo Kim, Hong Kook Kim, Moongu Jeon. (2022), “DenseBert4Ret: Deep bi-modal for image retrieval”, Information Sciences, Volume 612, 2022, Pages 1171-1186, ISSN 0020-0255, https://doi.org/10.1016/j.ins.2022.08.119.
Zheyuan Liu, Cristian Rodriguez-Opazo, Damien Teney, Stephen Gould. (2022), “Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models”, https://doi.org/10.48550/arXiv.2108.04024
Porter, M.F. (1980), “An algorithm for suffix stripping”, Program: electronic library and information systems, vol.14, issue 3, pp. 130–137. doi:10.1108/eb046814
Bard, Gregory V. (2007),“Spelling-error tolerant, order-independent pass-phrases via the Damerau–Levenshtein string-edit distance metric”, Proceedings of the Fifth Australasian Symposium on ACSW Frontiers : 2007, Ballarat, Australia, January 30 - February 2, vol. 68, pp. 117–124, 2007. ISBN 978-1-920682-49-1.
Wang James Z., Li Jia, Gio Wiederhold. (2001),“SIMPLIcity: Semantics-sensitive Integrated Matching for Picture Libraries”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol 23, no.9, pp. 947-963, 2001.
Li Jia, Wang James Z. (2003),“Automatic Linguistic Indexing Of Pictures By A Statistical Modeling Approach”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1075-1088, 2003.
B. Ionescu, A.-L. Radu, M. Menéndez, H. Müller, A. Popescu, B. Loni, “Div400: A Social Image Retrieval Result Diversification Dataset”, ACM Multimedia Systems - MMSys2014, 19-21 March, Singapore, 2014.
Siradjuddin, I. A., Bahruddin, R. P., & Sophan, M. K. (2019), “Combination of Term Weighting and Integrated Color Intensity Co-occurrence Matrix for Two-Level Image Retrieval on Social Media Data”, Procedia Computer Science, vol. 157, pp. 329–336. doi:10.1016/j.procs.2019.08.174