In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax

Aaron Mueller, Albert Webson, Jackson Petty, Tal Linzen

Type: Preprint

Publication Date: 2023-01-01

Citations: 0

DOI: https://doi.org/10.48550/arxiv.2311.07811

View Publication

Locations

arXiv (Cornell University) - View
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale	2022	Hritik Bansal Karthik Gopalakrishnan Saket Dingliwal Sravan Bodapati Katrin Kirchhoff Dan Roth
+ PDF Chat	In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax	2024	Aaron Mueller Albert Webson Jackson Petty Tal Linzen
+	Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?	2022	Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe Michael Lewis Hannaneh Hajishirzi Luke Zettlemoyer
+	Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations	2023	Chenglei Si Dan Friedman Nitish Joshi Feng Shi Danqi Chen He He
+	Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations	2023	Chenglei Si Dan Friedman Nitish Joshi Feng Shi Danqi Chen He He
+ PDF Chat	Exploring the Robustness of In-Context Learning with Noisy Labels	2024	Cheng Chen Xinzhi Yu Haodong Wen Jinsong Sun Guanzhang Yue Yihao Zhang Zeming Wei
+ PDF Chat	Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?	2022	Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe Michael Lewis Hannaneh Hajishirzi Luke Zettlemoyer
+ PDF Chat	Out-of-distribution generalization via composition: a lens through induction heads in Transformers	2024	Jiajun Song Zhuoyan Xu Yiqiao Zhong
+	Larger language models do in-context learning differently	2023	Jerry Wei Jason Lee Yi Tay Dustin Tran Albert Webson Yifeng Lu Xinyun Chen Hanxiao Liu Da Huang Denny Zhou
+	Do pretrained Transformers Really Learn In-context by Gradient Descent?	2023	Lingfeng Shen Aayush Mishra Daniel Khashabi
+	Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning	2023	Ruixiang Tang Dehan Kong Longtao Huang Hui Xue
+ PDF Chat	Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning	2023	Ruixiang Tang Dehan Kong Longtao Huang Hui Xue
+	Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing	2022	Linlu Qiu Peter Shaw Panupong Pasupat Tianze Shi Jonathan Herzig Emily Pitler Fei Sha Kristina Toutanova
+	On the Compositional Generalization Gap of In-Context Learning	2022	Arian Hosseini Ankit Vani Dzmitry Bahdanau Alessandro Sordoni Aaron Courville
+	On the Compositional Generalization Gap of In-Context Learning	2022	Arian Hosseini Ankit Vani Dzmitry Bahdanau Alessandro Sordoni Aaron Courville
+ PDF Chat	Parallel Structures in Pre-training Data Yield In-Context Learning	2024	Yanda Chen Chen Zhao Yu Zhou Kathleen McKeown He He
+ PDF Chat	Bayesian scaling laws for in-context learning	2024	Aryaman Arora Dan Jurafsky Christopher Potts Noah D. Goodman
+ PDF Chat	Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning	2023	Lean Wang Lei Li Damai Dai Deli Chen Hao Zhou Fandong Meng Jie Zhou Xu Sun
+	Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning	2023	Lean Wang Lei Li Damai Dai Deli Chen Hao Zhou Fandong Meng Jie Zhou Xu Sun
+	Syntactic Structure Distillation Pretraining For Bidirectional Encoders	2020	Adhiguna Kuncoro Lingpeng Kong Daniel Fried Dani Yogatama Laura Rimell Chris Dyer Phil Blunsom

Works That Cite This (0)

Action	Title	Year	Authors

Works Cited by This (0)

Action	Title	Year	Authors