Measuring Faithfulness in Chain-of-Thought Reasoning

Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion

Type: Preprint

Publication Date: 2023-01-01

Citations: 7

DOI: https://doi.org/10.48550/arxiv.2307.13702

View Publication

Locations

arXiv (Cornell University) - View - PDF
DataCite API - View

Similar Works

Action	Title	Year	Authors
+ PDF Chat	Dissociation of Faithful and Unfaithful Reasoning in LLMs	2024	Evelyn Yee Alice Li Chenyu Tang Yeon Ho Jung Ramamohan Paturi Leon Bergen
+	Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters	2022	Boshi Wang Sewon Min Xiang Deng Jiaming Shen You Wu Luke Zettlemoyer Huan Sun
+ PDF Chat	Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey	2024	Philipp Mondorf Barbara Plank
+ PDF Chat	Calibrating Reasoning in Language Models with Internal Consistency	2024	Zhihui Xie Jizhou Guo Tong Yu Шуай Ли
+ PDF Chat	Chain-of-Probe: Examing the Necessity and Accuracy of CoT Step-by-Step	2024	Z. Wang Xingshan Zeng Weiwen Liu Yufei Wang Liangyou Li Yasheng Wang Lifeng Shang Xin Jiang Qun Liu Kam‐Fai Wong
+	Question Decomposition Improves the Faithfulness of Model-Generated Reasoning	2023	Ansh Radhakrishnan Karina Nguyen Anna Chen Carol Chen Carson Denison Danny Hernandez Esin Durmus Evan Hubinger Jackson Kernion Kamilė Lukošiūtė
+	The Impact of Reasoning Step Length on Large Language Models	2024	Mingyu Jin Qinkai Yu Shuwen Dong Hantao Zhao Wenyue Hua Yanda Meng Yongfeng Zhang Mengnan Du
+ PDF Chat	On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models	2024	Sree Harsha Tanneru Dan Ley Chirag Agarwal Himabindu Lakkaraju
+	Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting	2023	Miles Turpin Julian Michael Ethan Perez Samuel R. Bowman
+ PDF Chat	Towards Faithful Chain-of-Thought: Large Language Models are Bridging Reasoners	2024	Jiachun Li Pengfei Cao Yubo Chen Kang Liu Jun Zhao
+ PDF Chat	Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning	2024	Debjit Paul Robert L. West Antoine Bosselut Boi Faltings
+ PDF Chat	Chain-of-Thought Unfaithfulness as Disguised Accuracy	2024	Oliver Bentham Nathan Stringham Ana Marasović
+ PDF Chat	LLMs with Chain-of-Thought Are Non-Causal Reasoners	2024	Guangsheng Bao Hongbo Zhang Linyi Yang Cunxiang Wang Yue Zhang
+	On Measuring Faithfulness of Natural Language Explanations	2023	Letiția Pârcălăbescu Anette Frank
+	Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning	2023	Murong Yue Jie Zhao Min Zhang Liang Du Ziyu Yao
+ PDF Chat	Patience Is The Key to Large Language Model Reasoning	2024	Yongang Yu
+	Stress Testing Chain-of-Thought Prompting for Large Language Models	2023	Aayush Mishra Karan Thakkar
+ PDF Chat	Fine-Tuning with Divergent Chains of Thought Boosts Reasoning Through Self-Correction in Language Models	2024	Haritz Puerto Tilek Chubakov Xiaodan Zhu Harish Tayyar Madabushi Iryna Gurevych
+	Automatic Chain of Thought Prompting in Large Language Models	2022	Zhuosheng Zhang Aston Zhang Mu Li Alex Smola
+	SCORE: A framework for Self-Contradictory Reasoning Evaluation	2023	Ziyi Liu Isabelle Lee Yongkang Du Soumya Sanyal Jieyu Zhao

Works That Cite This (5)

Action	Title	Year	Authors
+ PDF Chat	AI deception: A survey of examples, risks, and potential solutions	2024	Peter S. Park Simon Goldstein Aidan O’Gara Michael Chen Dan Hendrycks
+ PDF Chat	Explainable artificial intelligence: A survey of needs, techniques, applications, and future direction	2024	Melkamu Mersha Khang Nhứt Lâm Joseph Wood Ali K. AlShami Jugal Kalita
+ PDF Chat	A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions	2024	Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng Haotian Wang Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin
+	Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges	2024	Emanuele La Malfa Aleksandar Petrov Simon Frieder Christoph Weinhuber Ryan Burnell Raza Nazar Anthony G. Cohn Nigel Shadbolt Michael Wooldridge
+	Evaluating Language Model Agency through Negotiations	2024	Tim R. Davidson Veniamin Veselovsky Martin Josifoski Maxime Peyrard Antoine Bosselut Michał Kosiński Robert L. West

Works Cited by This (0)

Action	Title	Year	Authors