KoLA: Carefully Benchmarking World Knowledge of Large Language Models

Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-Li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li

Type: Preprint

Publication Date: 2023-01-01

Citations: 10

DOI: https://doi.org/10.48550/arxiv.2306.09296

View Publication

Locations

arXiv (Cornell University) - View
DataCite API - View

Similar Works

Action	Title	Year	Authors
+ PDF Chat	Law of the Weakest Link: Cross Capabilities of Large Language Models	2024	Ming Zhong Aston Zhang Xuewei Wang Rui Li Wenhan Xiong Chenguang Zhu Zhengxing Chen Liang Tan Chloe Bi Michael Lewis
+	How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances	2023	Zihan Zhang Meng Fang Ling Chen Mohammad‐Reza Namazi‐Rad Jun Wang
+ PDF Chat	Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall	2024	Jiaqing Yuan Lin Pan Chung-Wei Hang Jiang Guo Jiarong Jiang Bonan Min Patrick Ng Zhiguo Wang
+	Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach	2023	Zheyuan Zhang Jifan Yu Juanzi Li Lei Hou
+ PDF Chat	Knowledge Mechanisms in Large Language Models: A Survey and Perspective	2024	Mengru Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng Peng Wang Xiang Chen Jia-Chen Gu Yong Jiang Pengjun Xie
+ PDF Chat	How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances	2023	Zihan Zhang Meng Fang Ling Chen Mohammad‐Reza Namazi‐Rad Jun Wang
+	Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach	2023	Zheyuan Zhang Jifan Yu Juanzi Li Lei Hou
+	The Life Cycle of Knowledge in Big Language Models: A Survey	2023	Boxi Cao Hongyu Lin Xianpei Han Le Sun
+	Augmenting LLMs with Knowledge: A survey on hallucination prevention	2023	Κωνσταντίνος Ανδριόπουλος Johan Pouwelse
+ PDF Chat	Elements of World Knowledge (EWOK): A cognition-inspired framework for evaluating basic world knowledge in language models	2024	Anna A. Ivanova Aalok Sathe Benjamin Lipkin Unnathi Kumar Setayesh Radkani Thomas H. Clark Carina Kauf Jennifer Hu R. T. Pramod Gabriel Grand
+ PDF Chat	Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models	2024	Sitao Cheng Liangming Pan Xunjian Yin Xinyi Wang William Yang Wang
+ PDF Chat	A Survey on Knowledge Distillation of Large Language Models	2024	Xiaohan Xu Ming Li Chongyang Tao Tao Shen Reynold Cheng Jinyang Li Can Xu Dacheng Tao Tianyi Zhou
+	Head-to-Tail: How Knowledgeable are Large Language Models (LLM)? A.K.A. Will LLMs Replace Knowledge Graphs?	2023	Kai Sun Yifan Xu Hanwen Zha Yue Liu Xin Luna Dong
+ PDF Chat	PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations	2024	Jiatong Li Renjun Hu Kunzhe Huang Zhuang Yan Qi Liu Mengxiao Zhu Xing Shi Lin Wei
+	ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models	2022	Junyi Li Tianyi Tang Zheng Gong Lixin Yang Zhuohao Yu Zhipeng Chen Jingyuan Wang Wayne Xin Zhao Ji-Rong Wen
+	Mitigating Hallucinations of Large Language Models via Knowledge Consistent Alignment	2024	Fanqi Wan Xinting Huang Leyang Cui Xiaojun Quan Wei Bi Shuming Shi
+	A Principled Framework for Knowledge-enhanced Large Language Model	2023	Saizhuo Wang Zhihan Liu Zhaoran Wang Jian Guo
+	KGQuiz: Evaluating the Generalization of Encoded Knowledge in Large Language Models	2023	Yuyang Bai Shangbin Feng Vidhisha Balachandran Zhaoxuan Tan Shiqi Lou Tianxing He Yulia Tsvetkov
+	Assessing the Reliability of Large Language Model Knowledge	2023	Weixuan Wang Barry Haddow Alexandra Birch Wei Peng
+ PDF Chat	Evaluating the External and Parametric Knowledge Fusion of Large Language Models	2024	Hao Zhang Yuyang Zhang Xiaoguang Li Wenxuan Shi Haonan Xu Huanshuo Liu Yasheng Wang Lifeng Shang Qun Liu Yong Liu

Works That Cite This (6)

Action	Title	Year	Authors
+	A survey of GPT-3 family large language models including ChatGPT and GPT-4	2023	Katikapalli Subramanyam Kalyan
+ PDF Chat	Enhancing CLIP with GPT-4: Harnessing Visual Descriptions as Prompts	2023	Mayug Maniparambil Chris Vorster Derek Molloy Noel Murphy Kevin McGuinness Noel E. O’Connor
+	A Survey on Evaluation of Large Language Models	2024	Yupeng Chang Xu Wang Jindong Wang Yuan Wu Linyi Yang Kaijie Zhu Hao Chen Xiaoyuan Yi Cunxiang Wang Yidong Wang
+ PDF Chat	Can Large Language Models Understand Real-World Complex Instructions?	2024	Qianyu He Jie Zeng Wenhao Huang Lina Chen Jin Xiao Qianxi He Xunzhe Zhou Jiaqing Liang Yanghua Xiao
+ PDF Chat	Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods	2024	Yuji Cao Huan Zhao Yuheng Cheng Ting Shu Yue Chen Guolong Liu Gaoqi Liang J. Leon Zhao Jinyue Yan Yun Li
+	ZhuJiu: A Multi-dimensional, Multi-faceted Chinese Benchmark for Large Language Models	2023	Baoli Zhang Haining Xie Pengfan Du Junhao Chen Pengfei Cao Yubo Chen Shengping Liu Kang Liu Jun Zhao

Works Cited by This (0)

Action	Title	Year	Authors