Application of Prompt Engineering Techniques to Optimize Information Retrieval in the Metaverse

Year 2024, Volume: 4 Issue: 2, 157 - 164

Muhammed Abdulhamid Karabıyık , Fatma Gülşah Tan , Asım Sinan Yüksel

https://doi.org/10.57019/jmv.1543077

Abstract

Prompt engineering techniques are instructions that enable large language models (LLMs) to solve real-world problems more effectively. These techniques enhance the capabilities of LLMs to generate accurate and efficient responses. Our study examines the challenge of acquiring comprehensive and efficient information in the metaverse through the application of various prompt engineering techniques. The main objective is to improve the accuracy and effectiveness of metaverse-related responses by leveraging LLM capabilities. In this study, 100 questions were generated using GPT, GEMINI, QWEN, and MISTRAL language models focusing on the metaverse. Our experiments indicated that responses often included unrelated information, highlighting the need for prompt engineering techniques. We applied knowledge-based, rule-based, few-shot, and template-based prompt engineering techniques to refine the responses. The performance of GPT, GEMINI, QWEN, and MISTRAL models were evaluated based on criteria including accuracy, timeliness, comprehensiveness, and consistency. Our findings reveal that prompt engineering techniques significantly enhance the efficacy of LLMs in providing improved information retrieval and response generation, aiding users in efficiently acquiring information in complex environments like the metaverse.

Keywords

Prompt engineering, large language models, metaverse, information retrieval, response generation

References

• Mystakidis, S. (2022). Metaverse. Encyclopedia, 2(1), 486-497.
• Phoenix, J., & Taylor, M. (2024). Prompt engineering for generative AI: future-proof inputs for reliable AI outputs at scale. O'Reilly Media, Inc..
• Islam, R., & Ahmed, I. (2024, May). Gemini-the most powerful LLM: Myth or Truth. In 2024 5th Information Communication Technologies Conference (ICTC) (pp. 303-308). IEEE.
• Bai, S., Zheng, Z., Wang, X., Lin, J., Zhang, Z., Zhou, C., … & Yang, Y. (2021). Connecting language and vision for natural language-based vehicle retrieval. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 6, 4029-4038. https://doi.org/10.1109/cvprw53098.2021.00455.
• Wilkins, J. and Rodriguez, M. (2024). Higher performance of mistral large on mmlu benchmark through two-stage knowledge distillation.. https://doi.org/10.21203/rs.3.rs-4410506/v1.
• Sahoo, P., Singh, A. K., Saha, S., Jain, V., Mondal, S., & Chadha, A. (2024). A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. arXiv preprint arXiv:2402.07927 . https://doi.org/10.48550/arXiv.2402.07927.
• Huang, J. (2023). Engineering ChatGPT prompts for EFL writing classes. International Journal of TESOL Studies, 5(4), 73-79.
• Tassoti, S. (2024). Assessment of students use of generative artificial intelligence: prompting strategies and prompt engineering in chemistry education. Journal of Chemical Education, 101(6), 2475-2482. https://doi.org/10.1021/acs.jchemed.4c00212.
• Cui, G., Hu, S., Ding, N., Huang, L., & Liu, Z. (2022). Prototypical verbalizer for prompt-based few-shot tuning. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653/v1/2022.acl-long.483
• Schick, T. and Schütze, H. (2021). It’s not just size that matters: small language models are also few-shot learners. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua. https://doi.org/10.18653/v1/2021.naacl-main.185.
• Gao, T., Fisch, A., & Chen, D. (2021). Making pre-trained language models better few-shot learners. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer. https://doi.org/10.18653/v1/2021.acl-long.295.
• Wang, L., Chen, R., & Li, L. (2023). Knowledge-guided prompt learning for few-shot text classification. Electronics, 12(6), 1486. https://doi.org/10.3390/electronics12061486.
• Shin, T., Razeghi, Y., Logan, R. L., Wallace, E., & Singh, S. (2020). Autoprompt: eliciting knowledge from language models with automatically generated prompts. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10.18653/v1/2020.emnlp-main.346.
• Shi, C., Zhai, R., Song, Y., Yu, J., Li, H., Wang, Y., … & Wang, L. (2023). Few-shot sentiment analysis based on adaptive prompt learning and contrastive learning. Information Technology and Control, 52(4), 1058-1072. https://doi.org/10.5755/j01.itc.52.4.34021
• Elsadig, M., Alohali, M. A., Ibrahim, A. O., & Abulfaraj, A. W. (2024). Roles of blockchain in the metaverse: concepts, taxonomy, recent advances, enabling technologies, and open research issues. IEEE Access, 12, 38410-38435. https://doi.org/10.1109/access.2024.3367014.
• Haque, M. A., Rahman, M., Md. Faizanuddin, & Anwar, D. (2023). Educational horizons of the metaverse: vision, opportunities, and challenges. Metaverse Basic and Applied Research, 3, 60. https://doi.org/10.56294/mr202460.
• Sun, P., Zhao, S., Yang, Y., Liu, C., & Pan, B. (2022). How do plastic surgeons use the metaverse: a systematic review. Journal of Craniofacial Surgery, 34(2), 548-550. https://doi.org/10.1097/scs.0000000000009100.
• Lee, J. and Kwon, K. H. (2022). Future value and direction of cosmetics in the era of metaverse. Journal of Cosmetic Dermatology, 21(10), 4176-4183. https://doi.org/10.1111/jocd.14794.
• Sahoo, P., Singh, A. K., Saha, S., Jain, V., Mondal, S., ve Chadha, A. (2024). A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. arXiv preprint arXiv:2402.07927.
• Liu, J., Liu, A., Lu, X., Welleck, S., West, P., Bras, R. Le, … Hajishirzi, H. (2021). Generated Knowledge Prompting for Commonsense Reasoning. arXiv .
• Yang, J., Guo, X., Li, Y., Marinello, F., Ercisli, S., ve Zhang, Z. (2022). A survey of few-shot learning in smart agriculture: developments, applications, and challenges. Plant Methods, 18(1), 28. https://doi.org/10.1186/s13007-022-00866-2
• Ma, R., Zhou, X., Gui, T., Tan, Y., Li, L., Zhang, Q., ve Huang, X. (2021). Template-free Prompt Tuning for Few-shot NER. arXiv preprint arXiv:2109.13532.
• Jiang, Z., Xu, F. F., Araki, J., & Neubig, G. (2020). How can we know what language models know?. Transactions of the Association for Computational Linguistics, 8, 423-438. https://doi.org/10.1162/tacl_a_00324.
• Yong, G., Jeon, K., Gil, D., & Lee, G. (2022). Prompt engineering for zero‐shot and few‐shot defect detection and classification using a visual‐language pretrained model. Computer-Aided Civil and Infrastructure Engineering, 38(11), 1536-1554. https://doi.org/10.1111/mice.12954.
• Liu, J., Liu, A., Lu, X., Welleck, S., West, P., Bras, R. Le, … Hajishirzi, H. (2021). Generated Knowledge Prompting for Commonsense Reasoning. arXiv preprint arXiv:2110.08387.
• Ma, R., Zhou, X., Gui, T., Tan, Y., Li, L., Zhang, Q., ve Huang, X. (2021). Template-free Prompt Tuning for Few-shot NER. arXiv preprint arXiv:2109.13532.
• Lester, B., Al‐Rfou, R., & Constant, N. (2021). The power of scale for parameter-efficient prompt tuning. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/2021.emnlp-main.243