Specializing Smaller Language Models towards Multi-Step Reasoning

Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, Tushar Khot

Type: Preprint

Publication Date: 2023-01-01

Citations: 33

DOI: https://doi.org/10.48550/arxiv.2301.12726

View Publication

Locations

arXiv (Cornell University) - View
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models	2023	Yifan Hou Jiaoda Li Yu Fei Alessandro Stolfo Wangchunshu Zhou Guangtao Zeng Antoine Bosselut Mrinmaya Sachan
+ PDF Chat	Fine-Tuning with Divergent Chains of Thought Boosts Reasoning Through Self-Correction in Language Models	2024	Haritz Puerto Tilek Chubakov Xiaodan Zhu Harish Tayyar Madabushi Iryna Gurevych
+ PDF Chat	BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning	2025	Beichen Zhang Yuhong Liu Xiaoyi Dong Yuhang Zang Pan Zhang Haodong Duan Yuhang Cao Dahua Lin Jiaqi Wang
+ PDF Chat	Towards Understanding How Transformer Perform Multi-step Reasoning with Matching Operation	2024	Zhiwei Wang Yunji Wang Zhongwang Zhang Zhangchen Zhou Hui Jin Tianyang Hu Jiacheng Sun Zhenguo Li Yaoyu Zhang Zhi‐Qin John Xu
+ PDF Chat	Unlocking Structured Thinking in Language Models with Cognitive Prompting	2024	Oliver Krämer Jill Baumann
+	The Impact of Reasoning Step Length on Large Language Models	2024	Mingyu Jin Qinkai Yu Shuwen Dong Hantao Zhao Wenyue Hua Yanda Meng Yongfeng Zhang Mengnan Du
+ PDF Chat	SAAS: Solving Ability Amplification Strategy for Enhanced Mathematical Reasoning in Large Language Models	2024	Hyeonwoo Kim Gyoungjin Gim Yungi Kim Jihoo Kim Byungju Kim Wonseok Lee Chanjun Park
+	Teaching Small Language Models to Reason	2022	Lucie Charlotte Magister Jonathan Mallinson Jakub Adámek Eric Malmi Aliaksei Severyn
+	Resprompt: Residual Connection Prompting Advances Multi-Step Reasoning in Large Language Models	2023	Song Jiang Zahra Shakeri Aaron Chan Maziar Sanjabi Hamed Firooz Yinglong Xia Bugra Akyildiz Yizhou Sun Jinchao Li Qifan Wang
+ PDF Chat	TypedThinker: Typed Thinking Improves Large Language Model Reasoning	2024	Danqing Wang Jianxin Ma Fei Fang Lei Li
+ PDF Chat	Self-Discover: Large Language Models Self-Compose Reasoning Structures	2024	Pei Zhou Jay Pujara Xiang Ren Xinyun Chen Heng-Tze Cheng Quoc V. Le Ed H. Denny Zhou Swaroop Mishra Huaixiu Zheng
+ PDF Chat	Can Language Models Learn to Skip Steps?	2024	Tengxiao Liu Qipeng Guo Xiangkun Hu Jay J. Cheng Yue Zhang Xipeng Qiu Zheng Zhang
+	Chain-of-Thought Prompting Elicits Reasoning in Large Language Models	2022	Jason Lee Xuezhi Wang Dale Schuurmans Maarten Bosma Ed H. Quoc V. Le Denny Zhou
+ PDF Chat	Auto-Evolve: Enhancing Large Language Model's Performance via Self-Reasoning Framework	2024	Krishna Aswani H. J. Lu Prachi Patankar Priya Dhalwani I Leng Tan Jayant Ganeshmohan Simon Lacasse
+ PDF Chat	Reasoning with Large Language Models, a Survey	2024	Aske Plaat Annie Wong Suzan Verberne Joost Broekens Bas van Stein Thomas Bäck
+	ALERT: Adapting Language Models to Reasoning Tasks	2022	Ping Yu Tianlu Wang Olga Golovneva Badr Alkhamissy Gargi Ghosh Mona Diab Aslı Çelikyılmaz
+ PDF Chat	A NotSo Simple Way to Beat Simple Bench	2024	Soham Sane A.G. McLean
+	Complexity-Based Prompting for Multi-Step Reasoning	2022	Yao Fu Hao Peng Ashish Sabharwal Peter Clark Tushar Khot
+	Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning	2023	Murong Yue Jie Zhao Min Zhang Liang Du Ziyu Yao
+	Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance	2023	Yao Fu Litu Ou Mingyu Chen Yuhao Wan Hao Peng Tushar Khot

Works That Cite This (25)

Action	Title	Year	Authors
+ PDF Chat	Democratizing Reasoning Ability: Tailored Learning from Large Language Model	2023	Zhaoyang Wang Shaohan Huang Yuxuan Liu Jiahai Wang Minghui Song Zihan Zhang Haizhen Huang Furu Wei Weiwei Deng Feng Sun
+	LogiCoT: Logical Chain-of-Thought Instruction Tuning	2023	Hanmeng Liu Zhiyang Teng Leyang Cui Chaoli Zhang Qiji Zhou Yue Zhang
+ PDF Chat	Crystal: Introspective Reasoners Reinforced with Self-Feedback	2023	Jiacheng Liu Ramakanth Pasunuru Hannaneh Hajishirzi Yejin Choi Aslı Çelikyılmaz
+ PDF Chat	MoT: Memory-of-Thought Enables ChatGPT to Self-Improve	2023	Xiaonan Li Xipeng Qiu
+	Large Language Models Are Reasoning Teachers	2023	Namgyu Ho Laura Schmid Se-Young Yun
+	Reasoning with Language Model Prompting: A Survey	2023	Shuofei Qiao Yixin Ou Ningyu Zhang Xiang Chen Yunzhi Yao Shumin Deng Chuanqi Tan Fei Huang Huajun Chen
+	Sabiá: Portuguese Large Language Models	2023	Ramon Pires Hugo Abonizio Thales Sales Almeida Rodrigo Nogueira
+	A Survey of Reasoning with Foundation Models	2023	Jiankai Sun Chuanyang Zheng Enze Xie Zhengying Liu Ruihang Chu Jiaqi Liu Jiaqi Xu Mingyu Ding Hongyang Li Mengzhe Geng
+ PDF Chat	The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning	2023	Seungone Kim Segyeong Joo Doyoung Kim Joel Jang Seonghyeon Ye Jamin Shin Minjoon Seo
+	A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training	2023	Nitay Calderon Subhabrata Mukherjee Roi Reichart Amir Kantor

Works Cited by This (0)

Action	Title	Year	Authors